Fair Start Scotland employability service - year 4: online survey results

2. Approach

2.1 Methods

This research was undertaken by the social researchers at the Scottish Government as part of the ongoing evaluation of the Fair Start Scotland (FSS) service.

Potential survey participants were identified using a dataset supplied by local FSS providers that contained selected characteristics and contact details of individuals who joined the service in Year 4.

The survey questions were prepared by Scottish Government researchers. Each survey contained between 5 and 9 questions including a mix of closed- and open-ended questions. There were two common questions across the three surveys asking survey participants if there was anything that the service could have done better to support them into findings and staying in work^[6] and asking for any additional comments about the service that individuals might want to share.

The invitation to take part in each of the three online surveys was emailed to all individuals who joined FSS in Year 4 and who were eligible. Each survey remained active for three weeks and invited individuals were emailed up to two reminders to complete the survey. Each survey took around 5-10 minutes to complete. Data was collected between July and August 2022. Between 249 and 361 individuals completed each survey. The response rate was 10% for Survey 1 and Survey 3 and 11% for Survey 2. Table 1 presents further information on how many participants were invited to take part, how many declined and how many took part in each of the surveys.

For the closed-questions, where survey participants were asked to select one or more response option, we calculated how many survey participants selected each of the options. Free-text responses to the open-ended questions were analysed using thematic analysis to identify key themes across the responses for each of the open-ended question.

* Please note that some Year 4 FSS participants, who met the eligibility criteria for more than one survey (e.g. they were parents and they received in-work support as part of the FSS) received invitation to take part in all the surveys for which they were eligible. Due to the anonymity of the participants it is not possible for the researchers to indicate how many individuals completed more than one survey.

Further details of the methodology used in this study is presented in Appendix 1. Survey questions for each survey are presented in Appendix 2.

2.2 Strengths and limitations

All surveys were online, with invitations to participate issued to all service participants who joined in Year 4 of service delivery for whom email addresses were available. No explicit sampling frame was adopted – instead, we sought to survey everyone in each large cohort directly.

Using an online survey format allowed us to contact a relatively large population quickly and efficiently. However we are aware that using email addresses to contact participants and directing them to an online survey may mean that those people without good access to technology and/or those who have lower levels of digital literacy were at risk of exclusion from the survey.

This type of approach where people self-select whether to participate or not, may result in participation from those who have greater motivation to respond. There may be several influencing factors in this, including whether the participants had a more 'extreme' experience – good or bad. This selection bias may mean that results observed are not fully representative of the experience of service participants as a whole. However, the responses to these surveys indicate there are a range of experiences captured.

To reduce respondent burden and maximise response rates, a restricted number of closed key questions were included, along with a limited number of free text open questions to gather additional information. We did not gather information on the characteristics of respondents as previous similar online surveys found this to be detrimental to response rates. We also did not intend to conduct analysis by sub-group – the purpose of this research was to give a broad overview.

The response rates achieved were 10 to 11% across the three surveys. The relatively low response rates and the likelihood some response bias is present means that results presented here should be considered indicative only and considered within the wider body of evidence and evaluation of the FSS service. All previous evaluation reports are available on the Scottish Government website.^[7]