Fair Start Scotland: evaluation report 4 - local area case studies - year 3

Appendix 1 Evaluation methodology

This evaluation focuses on providing an insight into how delivery of Fair Start Scotland is playing out in practice and seeks to inform future iterations of employability services in Scotland through understanding what works.

This evaluation report:

• Outlines the findings from the three case studies conducted in year three
• Details the findings from our survey and interviews across Scotland
• Brings together the findings from years 1, 2 and 3 to summarise the overall findings for the evaluation.

This appendix breaks down the evaluation objectives and methodology.

Evaluation objectives

The key objectives of the evaluation are to:

• Understand how Fair Start Scotland is being implemented across the different lot areas in Scotland
• Understand the experience of Fair Start Scotland for lead providers, partner organisations, participants and employers
• Identify what is working well and less well in the implementation of Fair Start Scotland
• Identify the lessons learned and recommend changes to consider for the remainder of the Fair Start Scotland contract period as well as shaping what the next iteration of employment support in Scotland might look like

Evaluation methodology

This evaluation has multiple components in terms of methodology in relation to:

• Scotland wide research – conducted primarily in Year 3
• Nine case studies across the three years of the evaluation.

Scotland-wide methodology

We conducted the following research in Year 3 of the evaluation (2021) with staff and stakeholders across Scotland in order to understand their experiences of Fair Start over the past three years:

1. Interviews with senior stakeholders - a range of senior stakeholders from across Scottish Government, Jobcentre Plus, and Lead Providers

2. Survey of Fair Start Scotland frontline staff – an online survey of Fair Start Scotland frontline staff across all nine contract case study areas

3. Survey of Jobcentre Plus frontline staff – online survey of Jobcentre Plus Work Coaches across Scotland.

The table below outlines the details of who we interviewed as part of the senior stakeholder interviews.

In 2017, as part of a previous piece of work, Rocket Science UK Ltd. had conducted a survey of Fair Start Scotland frontline staff and Jobcentre Plus work coaches to determine their early experiences of the Fair Start Scotland service. This year, we issued very similar surveys to current frontline staff and Jobcentre Plus work coaches to determine whether there had been any significant change in their experiences. The surveys were issued to all frontline staff and Jobcentre Plus work coaches via Scottish Government. In 2017, 172 responses were received from frontline staff. 147 frontline staff responded in 2021. In 2017, 90 Jobcentre Plus work coaches responded to the survey, and in 2021 96 Jobcentre Plus work coaches submitted responses. The full comparison of findings can be found in Appendix 2. Key points are drawn out in relevant sections of this report.

Case study methodology

We have carried out the following tasks in each of the case study areas:

• Conducted desk-based area analysis of the socio-economic and employment trends in each of the localities to understand the local labour market context that Fair Start Scotland is operating in. This analysis was streamlined in Year 3 to enable a fuller analysis of the messages across Fair Start Scotland bringing together the three years of case studies
• Analysed the management and performance data from Fair Start Scotland in each of the localities to understand the profile and numbers of participants and outcomes achieved in the area
• Conducted interviews with participants in each locality to understand their experience of and views on Fair Start Scotland. The total number of participants in each case study area interviewed is outlined in the table overleaf.

• Conducted interviews with employers in one locality to understand their experience of and views on Fair Start Scotland. Because of the impact of Covid-19 arranging employer interviews in years 2 and 3 of the evaluation has proved difficult as employers had staff on furlough or it was difficult to get hold of very busy staff in HR roles, so the number of employer interviews was limited.
• Conducted interviews with service staff including managers and frontline staff delivering Fair Start Scotland locally to understand their experience of and views on the service
• Conducted interviews with staff in partners of Fair Start Scotland providers in the locality to understand their experience of and views on Fair Start Scotland
• Conducted interviews with relevant other stakeholders in the area to understand their experience of and views on Fair Start Scotland.

The table overleaf provides a breakdown of the Year 3 case study participant interviewees as has been provided in prior years’ reports.