Access to sanitary products Aberdeen pilot: evaluation report

Annex C: Full Methodology

Data collection: community partners

Administrative data

Data was recorded by CFINE on:

• how many and what type of products were purchased;
• cost of products;
• how many products, cash or cards, and what type of products, were distributed to partners and when.

Data was recorded by partners on:

• number of participants signed up;
• how often each participant is provided with products or cash/card;
• no./type of products provided or amount of money given.

Partners were provided with a template spreadsheet on which to log this information, to help standardise the information recorded. Organisations varied as to their systems for recording this data – in smaller organisations one person typically managed the recording of information, while in others a number of staff members and volunteers were involved. There was, therefore, variation in the level of detail recorded about products: some entries included full information on the number and type of products ( e.g. "Always Ultra pads with wings"), while others simply recorded "Tampons". Some information on products provided was recorded for 528 participants at community partners (see Table 22 below).

Initial data collection

Partners

Telephone interviews were conducted with a sample of the initial community partners after they had set up the pilot in their organisation. Interviews were conducted with the staff member responsible for coordinating pilot activities at eight organisations involved in the pilot (five third sector partners and three regeneration areas). Interviews took place between the end of August and beginning of October 2017, depending on how quickly the pilot had commenced in that organisation, and lasted between 15 and 30 minutes. Interviews covered issues partners had encountered around access to sanitary products prior to the pilot; decisions made about providing sanitary products and the rationale for these; and any issues arising with the coordination of the pilot thus far.

Participants

Pilot participants were asked to complete an initial questionnaire when they signed up to take part. Five partner organisations decided not to ask participants to formally sign up to the pilot or complete questionnaires because they considered their clients to be especially vulnerable. The questionnaire was agreed between the Scottish Government, CFINE and partners. Interviewer complete and self-complete versions were produced. Most partners chose to have staff or volunteers go through the questionnaire with participants (especially where literacy might be a barrier – no specific provision was made for non-English speakers however). The questionnaire recorded general information about the participant and their past experiences accessing sanitary products, including: how they heard about the pilot; age; ethnicity; household size; economic status; financial difficulties experienced; difficulties accessing sanitary products and impact of these difficulties; and awareness of and interest in reusable sanitary products.

A completed initial survey was submitted for 498 community participants (see Table 22 below). Many completed questionnaires did not include answers to all questions. In particular, many participants chose not answer questions on their financial situation.

End-point data collection

Partners

The end-point data collection focussed on the original community partners who had been running the pilot in their organisation longest. However one additional community partner was included because they signed up a comparatively large number of participants over a short space of time and their experience was considered to be particularly interesting.

Interviews were conducted with partner staff or volunteers at the end of the pilot. Three staff involved in co-ordinating the pilot at CFINE and the lead staff member at 10 partners were interviewed (six third sector partners, three regeneration areas, and one additional community partner). Two interviews and two paired interviews were also conducted with six volunteers at CFINE who had been involved with running the pilot in CFINE's food bank, two of whom had also been helped with the coordination of the pilot.

The majority of interviews were conducted face-to-face (14) and interviews lasted between 15 and 60 minutes. Interviews covered views on the central coordination and distribution by CFINE; how the chosen delivery model worked; what went well and what challenges they faced; and their experiences discussing the issue with participants.

Participants

Towards the end of the pilot, participants were asked to complete a brief questionnaire on their experience of and the impact of the pilot. Paper and online versions of the survey were created. Most community partners used the paper version, but one distributed the online version only and two made both versions available. Topics covered in the survey included:

• whether participants received the products they required;
• impacts of the pilot on them; and
• views on a future scheme.

An 'end-point' survey was completed by 109 participants who had signed up for the pilot (see Table 2). An additional 71 respondents who were not pilot participants completed the survey.

At the end of the pilot, we undertook qualitative data collection with a small sample of participants to explore their feelings about, and experiences of, accessing sanitary products and the pilot in more detail. The sample aimed to include participants who signed up with a range of the partner organisations. Five individual interviews, one paired interview and three focus groups were conducted with 28 participants in total from seven partners (four third sector organisations and three regeneration areas). Participants were recruited via partner organisation staff and we took advice from staff on whether individual interviews or a focus group would be preferable for participants.

Topics covered included:

• previous difficulties accessing products and implications;
• experience during the pilot, particularly product choice and dignified receipt (how they felt receiving free products; how they felt about method of receiving products; attitudes toward product cost vs. quality);
• impact of the pilot (managing menstruation, anxiety, embarrassment, ability to take part in school/college/other activities);
• views on a future scheme.

Data collection: schools, college and university

The schools, college and university collected the same administrative information as other partners (see above) .On sign up, college and university students were asked to complete a similar initial questionnaire to community partners, while school pupils were asked to complete a much shorter form that asked for information on age, ethnicity and previous difficulties accessing products.

We aimed to interview the staff member responsible for coordinating pilot activities at the college, university and each of the four schools towards the end of the pilot, and for students at the college, university and each of the three secondary schools to complete an online survey. However, the capacity of staff in the college and particularly some of the schools to engage with the evaluation was limited. A face to face interview was conducted with the lead at RGU in February 2018, and telephone interviews with the lead at two of the schools (one secondary and one primary) in March 2018. Interviews covered the same main topics as for other partners.

A slightly adapted online survey was created for participants at the college, university and schools. The survey was open to all students or pupils at the participating institution. For pilot participants it included questions on the same topics as for community partners. Those who had not signed up to the pilot were asked a different set of questions, including why they did not sign up, difficulties accessing sanitary products and impact of these difficulties; and awareness of and interest in reusable sanitary products. The survey routed respondents to different questions depending on whether they had heard about the pilot and signed up to the pilot. The online survey was distributed at the university, college, but only one secondary school.

An 'end-point' survey was completed by 27 college and university students and 4 secondary school pupils (at one school) who had signed up for the pilot (see Table 22). An additional 118 college students, 99 university students, and 18 school pupils who were not participants completed the survey.

Table 22 provides an overview of the three types of quantitative data collected about participants by partner organisation: the number of participants partners recorded data about products/card distributed for, the number of participants who completed an initial survey, and the number of participants who completed an end-point survey, as well as the total number of participants signed up to the pilot for comparison.

Table 22: Administrative and survey data collected by partner type and organisation

Partner Organisation	Total N participants signed up*	N participants product data recorded for	N participants completed initial survey	N participants completed end- point survey
CFINE (food bank)	209	202	157	12
Foyer	23	23	23	21
Homestart	12	8	12	9
Instant Neighbour	19	15	16	0
Balnagask	6	3	6	4
Cummings Park	20	20	16	8
Fersand & Fountain	23	13	11	0
Middlefield	36	33	30	5
Printfield	9	8	7	7
Seaton	14	14	13	3
St George's Church	208	147	170	40
Other community partners	65	42	37	N/A
Total community partners	644	528	498	109
Robert Gordon University	43	36	41	9
North East Scotland College	108	87	14	18
Secondary schools	120	100	77	4
Total school, college, university	271	223	132	31

* Includes only participants who formally signed up

'N/A' – indicates where the organization did not take part in that aspect of data collection

Analysis

Quantitative data analysis

Administrative data and data from the initial participant survey were collated and input into separate Excel spreadsheets by CFINE staff. The paper end-point participant surveys were collated by CFINE and passed to Scottish Government analysts for inputting. The online survey data was extracted from Questback in Excel format and the data from the paper surveys was also input into this spreadsheet. The data in the three spreadsheets was checked by Scottish Government analysts, and incomplete responses were removed from the dataset. ^[16] The data was analysed using Excel.

Qualitative data analysis

Interviews with partner staff and pilot participants were audio recorded, with permission, and transcribed either by the researchers working on the project or a member of SG staff who provides transcription services. All the end-point interviews were fully transcribed, while extended notes were taken from the recordings of the initial partner interviews. One focus group was not audio-recorded. Instead, notes were taken by the researchers during the discussion and written up fully immediately afterwards.

Thematic analysis was undertaken using the main topics outlined in the research objectives as a starting point. Partner and participant transcripts were analysed separately. Excerpts from transcripts were organised under the main topics set out in the research objectives. Once all transcripts had been coded into broad topics, more detailed coding took place within topics to organise extracts into sub-themes and identify similarities and divergences within the data. Two researchers worked together to assign and cross-check themes and sub-themes.

Ethical considerations

Ethical issues for this project were considered by the project team and a Scottish Government ethics review checklist was completed. In addition, Aberdeen City Council's Research Request form was completed and approval to conduct research in the schools was received. The key ethical issues considered were:

• Informed consent – partner staff: partners were informed by email about the purpose of the research, topics to be discussed, use of data and that their participation is voluntary. At the beginning of the phone call or interview, the researcher outlined this information and answered any questions the interviewee had, requested consent to audio-record the interview and confirmed consent.
• Informed consent – participants: participants received general information about the pilot and data being collected as part of the evaluation on signing up to take part in the pilot. A key concern was that participants could feel obliged to take part in the research because they were benefiting from the pilot. Researchers underlined to partner staff that participation in the pilot and evaluation is voluntary and that it was absolutely fine for beneficiaries to receive products but opt out of any/all aspects of the evaluation. Interviews with participants were set up through partner staff. Pilot participants who indicated to staff they were willing to take part in an interview or focus group were provided with an information sheet informing them about the purpose of the research, topics to be discussed, use of their data and that their participation is voluntary. A researcher talked through this information at the beginning of the interview, in particular underlining the voluntary nature of participation, and answered any questions participants had. Consent was then confirmed and participants were asked to sign a consent form.
• Discussing a sensitive topic: the research topics were viewed as potentially stressful or sensitive for participants to discuss ( e.g. menstruation; struggling to get by on a low income). This was a key consideration in the design of topic guides and conduct of the interviews. The topic guide focussed on the essential data required for the evaluation and took care to frame questions so participants did not feel pressured to disclose personal information if they did not want to. Interviewers were conducted by a Scottish Government researcher or a postgraduate researcher, who were trained in qualitative research methods. At the outset of interviews, researchers emphasised that participants could choose to not answer any of the questions or stop the interview at any point.
• Non-disclosure of identity and personal information: CFINE and partner organisations collected and stored some personal data about participants to manage the pilot provision. Respondents were informed about the personal data collected and what would happen to it. Personal data was stored securely and separately from research responses. The Scottish Government stored and analysed anonymised data only, and any extracts from interviews with participants are reported anonymously. It was more difficult to fully anonymise partner staff in the reporting. Partner staff were given the opportunity to check they were happy with any interview extracts included in the report and how they were attributed ( e.g. 'staff member, third sector organisation').