Access to sanitary products Aberdeen pilot: evaluation report

2. Pilot delivery and evaluation methodology

2.1 Overview of delivery arrangements

The pilot provision was initially targeted at those on a low income via third sector and community partners. In addition to provision via CFINE's food bank, five third sector organisations initially agreed to provide access to sanitary products.

• HomeStart support families facing a variety of challenges, including ill health, disability, domestic violence and substance misuse.
• Instant Neighbour provides a range of services, including emergency food parcels through their food bank and good quality 2 ^nd hand furniture and electrical items, sold at low cost in their shops.
• Aberdeen Foyer helps people build confidence and develop their skills and talents to make major and lasting changes in their lives. They deliver linked up services – education, training, mentoring, counselling, employability support and health improvement initiatives to people of all ages.
• Grampian Women's Aid support women, children and young people who have experienced domestic abuse. They provide a free and confidential service which includes advice and information, support and temporary refuge accommodation.
• Aberdeen Cyrenians provide services to meet all the varying needs of people affected by homelessness.

The pilot also operated across regeneration areas of the city, working with:

• Balnagask Community Centre
• Cummings Park Community Centre
• Fersands and Fountain Community Project
• Middlefield Community Project
• Printfield Community Project
• Seaton Community Project and the Rehab Project
• Tillydrone Community Flat

In addition, CFINE worked with North East Scotland College ( NESCol) and Robert Gordon University ( RGU) to provide sanitary products for students, and Aberdeen City Council to provide products for pupils in three secondary schools. Two of the secondary schools are in regeneration areas and one is not. A primary school was also added to the pilot later. The pilot provision in schools, the college and university was open to all students.

Some additional local authority, third sector and community partners were added to the pilot at a later stage. A full list of community partners is provided in Annex B. The local authority, third sector and community organisations involved in the pilot provision for those on a low income are referred to as 'community partners' throughout the report.

CFINE and partners initially focussed on providing access to a range of products directly. As noted in the last chapter, providing access to products in ways that respect the dignity of participants and provide choice were key concerns. However, for the purpose of the pilot, participants were generally asked to 'sign up'. This was to enable data to be collected about uptake and participants. The sign up process involved participants being provided with information about the pilot, signing a statement regarding the collection and use of data, and completing a questionnaire. Partners were encouraged to use their judgement and provide products without the sign up process if participants preferred not to formally sign up.

To test options where participants were provided with the means to buy products themselves, partners were also given the option of beginning to offer cash to participants in October and a pre-paid card in December. These options either involved the partner organisation offering participants £3 in cash for that month instead of products, or a 'Love to Shop' gift card, which can be spent in a number of high street shops, with a value of £3.

2.1.1 Community partners

Delivery started in the original community partners between August and September 2017. The additional community partners started delivery between December 2017 and January 2018. Methods for signing up participants and distributing products or the means to purchase products varied depending on how the organisation works with clients. Generally, sign up took place in a private room and products were picked up from the same location or another designated location within the building. However, other examples included a worker taking products out with them on a visit or inclusion in a food parcel.

2.1.2 College and university

In the college, the Student Association led on the pilot, and delivery started in mid-September 2017. Students signed up at the Student Association's reception and collected the products they required at a designated pick-up point within the Student Association's central hub. There was a schedule of drop in sessions for signing up and collecting products that was released on a monthly basis.

In the university, the Student Union delivered provision. Delivery started in mid-September 2017. Students could sign up at the Student Union reception and access products from a designated collection area. Reception staff also emailed those who have signed up to remind them to access products each month.

2.1.3 Schools

The secondary schools started delivery between November and December 2017, while the primary school started delivery in December 2017. Delivery in the secondary schools was shared between guidance staff and school nurses. Products were made available at designated location/s in each school (for example the school office or nurse's office) or on request from specific teachers. In relation to respecting dignity in schools, options where pupils did not have to ask for products were considered. One secondary school also agreed to trial making products available in baskets in the school toilets to test a method of accessing products that did not require pupils to speak to someone.

2.2 Number of participants taking part in the pilot

Community/third sector partners stopped formally signing up participants at the end of February. As of end February (end March for schools), just over 1,000 participants had signed up to take part in the pilot: 209 via CFINE's food bank, 590 via the other community/third sector partners, 43 at RGU, 108 at NESCol and 133 at the schools involved (see Table 1).

Table 1: Pilot sign up by month and partner (from CFINE)

Partner Organisation	Sept	Oct	Nov	Dec	Jan	Feb	Total
CFINE (food bank)	79	21	25	26	29	29	209
Grampian Women's Aid	4	4	4	0	0	0	12*
Foyer	23	0	0	0	0	0	23
Homestart	10	2	0	0	0	0	12
Instant Neighbour	15	0	2	2	0	0	19
Deeside Family Centre	13	13	14	7	0	0	47*
Regeneration areas
Balnagask	6	0	0	0	0	0	6
Cummings Park	12	2	1	3	2	0	20
Fersands & Fountain	13	0	0	0	10	0	23
Middlefield	19	1	2	10	2	2	36
Printfield	8	1	0	0	0	0	9
Seaton	13	0	0	0	0	1	14
Tillydrone	5	5	5	0	0	0	15*
Additional partners
St George's Church	N/A	N/A	N/A	110	90	8	208
Other additional partners	N/A	N/A	N/A	30	94	22	146
College, university and schools
Robert Gordon University	35	7	0	1	0	0	43
North East Scotland College	39	15	33	0	20	0	107
Secondary school 1	0	0	9	6	8	7	30
Secondary school 2 (regen. area)	0	0	33	11	11	8	63
Secondary school 3 (regen. area)	0	0	0	27	0	0	27
Primary school (regen. area)	N/A	N/A	N/A	N/A	13	0	13
Total	294	71	128	233	279	77	1082

'N/A' – indicates where organisations had not yet started signing up participants

*Approximate total – organisation did not ask participants to formally sign up

As community partners made the pilot available across a range of different services and to those who dropped in to community centres or food banks, it is not possible to provide any reasonable estimate of the target population for each organisation. Similarly, we do not consider it appropriate to provide the number of participants as a proportion of the female population of the schools, college and university. It is not clear how widely knowledge of pilot was disseminated, therefore whether not signing up is reflective of lack of awareness.

2.3 Methodology

2.3.1 Overview of methods

Ethical issues for this project were considered by the project team and a Scottish Government ethics review checklist was completed. In addition, Aberdeen City Council's Research Request form was completed and approval to conduct research in the schools was received.

A mixed methods approach was taken to evaluation of the pilot. A range of monitoring data was collected by CFINE and partners. In addition the evaluation surveyed participants at the start and end of the pilot, and qualitative interviews were conducted with a small sample of participants and administrators at a sample of the partners. The full methodology can be found in Annex C and an overview of how the methods related to the research questions can be found in Annex D.

CFINE and all partner organisations recorded information about the number of participants signing up and products, cash or cards distributed. Pilot participants were asked to complete an initial questionnaire when they signed up to take part. The questionnaire recorded general information about the participant and their past experiences accessing sanitary products. Data collection focussed on the original community partners and educational institutions that had been running the pilot in their organisation longest.

2.3.2 Community partners

• Telephone interviews were conducted with administrators at eight partners after they had set up the pilot in their organisation to find out how the pilot activity was progressing.
• Seventeen interviews were conducted with partner staff or volunteers at the end of the pilot. Three staff involved in co-ordinating the pilot at CFINE, administrators at 10 partners, and six volunteers at CFINE were interviewed. Interviews covered views on how the chosen delivery method worked, what went well and what challenges they faced.
• Towards the end of the pilot, participants were asked to complete a brief questionnaire on their experience of and the impact of the pilot. Paper and online versions of the survey were created; most community partners used the paper version.
• At the end of the pilot, we also undertook qualitative data collection with a small sample of participants to explore their feelings about, and experiences of, accessing sanitary products and the pilot in more detail. Individual or group interviews were conducted with 28 participants from seven partners.

2.3.3 Schools, college and university

• A lighter touch approach was taken to data collection in the schools, college and university as their capacity to engage with the pilot was more limited and their provision started later.
• College and university students completed the same initial questionnaire as community participants, while school pupils were asked to complete a much shorter form that asked for information on age, ethnicity and previous difficulties accessing products.
• An online version of the end-point survey was created for use in the schools, college and university. This survey open to all students, with the same set of questions for pilot participants, and different questions for those who had not signed up to the pilot, including why they did not sign up and previous difficulties accessing sanitary products. The survey was distributed at the college, university and in one school.
• Telephone interviews were conducted with the lead at RGU and two schools (one secondary and one primary). Interviews covered the same main topics as for other partners.

2.3.4 Critical discussion of methodology

A number of considerations and limitations should be borne in mind when reading the findings presented in this report. Limitations to be aware of include:

• The lack of existing data on how many people have difficulty accessing sanitary products and who is affected mean it is difficult to assess whether the pilot was reaching the right people and to understand how participants compare to non-participants.
• Due to this lack of baseline data, it was not possible to use objective measurements to assess the impact of the pilot – all impacts discussed in the report are based on participants' self-reports in questionnaires and interviews.
• As discussed in more detail later in the report, many of the partner organisations had limited capacity to dedicate to the pilot, and particularly to the data collection aspects. School staff in particular struggled with capacity to implement the pilot activities. This meant that the data collected was variable across organisations and it was not possible to conduct interviews with all of the partner administrators we had hoped. Many completed participant questionnaires also did not include answers to all questions.
• A discussed further in Section 4.4 of the Findings, a significant proportion of those who signed up to the pilot initially did not return monthly for products. This created an additional barrier to finding out about participants' views of the pilot. The end-point survey was completed by a relatively small proportion of participants and may not be representative of all those who participated in the pilot.
• Additionally, periods, sanitary products and struggling to manage on a low income are sensitive topics to raise with participants. It is likely that, in general, the pilot did not reach those who felt most uncomfortable with speaking to someone they do not know very well about these issues. In particular, it is likely that participants who agreed to take part in a qualitative interview were those who were more comfortable speaking about these topics.

A central issue for this pilot and the evaluation is the issue of dignity and positioning people as recipients of help or research participants. In general, to maintain the dignity of those that provision is aimed at, it is preferable for them not to have to identify themselves as 'in need'. However, in order to gather information as part of the evaluation it was necessary to ask participants to sign up to take part, and then to answer questions about their experiences of being in need. The evaluation methodology therefore put constraints on the models of provision that could be tested during the pilot, particularly as regards dignified provision.