Youth Employment Scotland Fund (YESF): evaluation report

2 Method

Introduction

2.1 The primary research for the evaluation consisted of four main phases of research with stakeholders, Local Authorities, employers and young people who have been involved in the YESF and these are described in this chapter.

2.2 The research tools are provided as a technical annex to the report.

Stakeholder consultations

2.3 The study team conducted consultations with strategic and operational stakeholders from the following organisations:

• Scottish Government
• Department for Work and Pensions
• Scottish Chambers of Commerce
• Federation of Small Businesses in Scotland
• Skills Development Scotland
• Third Sector Employability Forum
• Scottish Council for Voluntary Organisations
• Jobcentre Plus
• The Scottish Local Authorities Economic Development ( SLAED) Group

2.4 The purpose of the consultations was to gain an understanding of consultees' views of the YESF; its successes and weaknesses; its strategic fit; complementarity/ duplication with other employability initiatives; perspectives on the marketing of the YESF; and delivery challenges.

2.5 We consulted with ten stakeholders and their views are included throughout the report. A list of the individuals consulted is provided in Annex 1.

Local Authority engagement and consultation

2.6 Local Authority engagement and consultation was a key element of the study. The researcher contacted all 32 Local Authorities by telephone and email to request YESF monitoring data. This included information relating to the young people who have participated in the YESF, the employers who provided jobs and the financial details of the funding received by the Local Authorities and paid as ERIs to employers. We provided a pro-forma to the Local Authorities to help them gather and provide the data.

2.7 Twelve Local Authority areas were selected by purposive sampling for more in-depth qualitative work with the council, young people and employers. The 12 Local Authority areas included all three Ayrshires as one area, as the YESF was delivered in a co-ordinated way across all three. The sample included urban and rural areas, areas with high and low jobseeker claimant rates of 18 to 24 year olds, areas with high and low levels of YESF uptake and areas with different employer profiles.

2.8 The evaluation team interviewed staff from ten of the 12 Local Authorities covering topics such as views on the YESF delivery model and the marketing/awareness raising activities used to promote the scheme to young people, employers and others. The interviews also covered the strategic fit and complementarity/duplication with wider employability offers, funding arrangements and particular strengths and weaknesses of the YESF, including challenges in delivery, monitoring and evaluation.

Research with employers

2.9 Employers who had participated in the YESF were contacted to take part in a telephone survey. A total of 50 employers took part in the telephone survey from the sample of 12 Local Authority areas (three Ayrshires included as one) which represented employers by geography, take-up levels, unemployment levels and business types and sizes.

2.10 The telephone survey covered the following topics: the business's background, how the employers found out about the YESF, what they thought of the arrangements, the level of support provided, the outcomes for the business, and the difference that the ERI made.

2.11 Employers within the sample of Local Authority areas who did not take part in the telephone survey were sent an email and invited to take part in an online survey using SurveyMonkey ( www.surveymonkey.com). The online survey covered the same themes as the telephone survey. The remaining 18 Local Authorities who were not included in the sample of Local Authority areas were contacted and asked to circulate the online survey to employers who had participated in the YESF within their area. One hundred and thirty three (133) employers completed the on-line survey, and a further five employers were consulted with as part of the case study development, meaning that a total of 188 employers participated across the two surveys and the case studies.

2.12 Following the completed telephone interviews, five employer case studies were developed. They were designed to illustrate how the programme has assisted employers to provide young people with sustainable employment and/or improve their skills and qualifications. A combination of information obtained from the telephone survey and the in-depth telephone conversation was used to develop the final case studies.

Research with young people

2.13 Young people from the 12 Local Authority areas who had undertaken a YESF job were contacted to take part in a telephone survey. Forty young people participated in the telephone survey which covered topics such as background information about the young person, how they found out about the job, what attracted them to participate, the experience they gained, the outcomes and impacts and their views on their future career and support needs.

2.14 The remaining young people within the sample of 12 Local Authority areas who had not been surveyed by telephone were emailed an online survey covering the same themes as the telephone survey. The remaining Local Authorities who were not included in the sample were contacted and requested to circulate the online survey to young people involved in the YESF in their area. A total of 74 young people responded to the on-line survey. This means that survey information was gathered from a total of 114 young people across 20 Local Authority areas representing just over 1% of the total jobs provided through YESF. Whilst it is a small percentage, the total number of 114 young people is adequate to give robust findings as part of a mixed methods approach.

2.15 Following the completed telephone interviews, five young people case studies were developed with young people of different ages, in different locations, who have completed the ERI period of the job at different types of employer by type, sector, size, etc. The purpose of the case studies is to demonstrate how the programme has assisted young people to achieve sustainable employment and/or improve their skills and qualifications.

2.16 Further insights into some of the themes identified were obtained from an online discussion group of young people. Survey respondents opted in to the discussion forum and took part in moderated discussions around key topics relating to the fund. These discussions were delivered using the online focus group platform, Liveminds ( www.liveminds.co.uk).

Financial Analysis

2.17 The evaluation did not include financial analysis as the data was not available for the evaluation. The study was conducted before the programme closed and so the financial data was not final.

2.18 Whilst there is no financial analysis, we do know that Local Authorities absorbed the costs of set up, administration, compliance and delivery into existing budgets as no funding was available to cover these costs. This means that the YESF investment levered in significant additional resource however it has proved very difficult to place a figure on this given the variations in local delivery models.

Methodological challenges

2.19 The researcher faced some methodological challenges during the evaluation and worked closely with the Scottish Government and SLAED to respond to and overcome these challenges.

2.20 The YESF involved an approach to monitoring which sought to minimise the burden on Local Authorities and employers. The Scottish Government collected data on job starts from Local Authorities throughout the programme. Local Authorities gathered data on the characteristics of the young people, the employers, the jobs and the outcomes in different ways and to varying degrees. In addition, the final phase of the YESF was not complete at the time of the research. Both of these factors mean that the detailed data on characteristics is not as comprehensive as it might be and has not been straightforward to aggregate and analyse. Whilst there are good, robust findings based on this data, it must be caveated that it is not complete.

2.21 The majority of Local Authorities were not able to provide contact details for all of the young people and the employers who had participated in the YESF in their area which reduced the total population that the survey reached. Despite this, the telephone and on-line surveys were completed by a good sample of young people and employers and provided robust data.

2.22 The intention was to conduct a programme of focus groups with young people in local areas but through the work with Local Authorities it became clear that this was not feasible within the timescales and budget. Instead, we boosted the number of young people taking part in qualitative interviews and facilitated an on-line discussion forum.

2.23 These challenges caused some delay in the study progress and meant we had to adapt our methods, they did not however detract from the robustness of the evaluation and the findings.