Developing the Young Workforce (DYW) regional groups: formative evaluation

Executive Summary

Introduction

SQW was appointed by the Scottish Government to conduct a formative evaluation of the Developing the Young Workforce (DYW) Regional Groups. The study was carried out between December 2017 and April 2018. It involved in-depth exploration of what was working well and where challenges remained in four of the Groups: Ayrshire; Edinburgh, Midlothian and East Lothian; Inverness and Central Highland; and North East.

Methodology

There were three main stages involved in the evaluation. The work began with a desk review and scoping consultations with members of the National Delivery Group, as well as the Chairs and Executive Leads for each of the four Regional Groups. This was followed by the main evidence gathering stage, which involved surveys and consultations with employers, schools and Regional Group board members within each of the four areas. The final stage involved reporting the headline findings back to the four Regional Groups and the National Group via a series of workshops, which were used to test and further refine the analysis and interpretation of the results.

Policy context

• The Scottish Government set up the Commission for Developing Scotland's Young Workforce in January 2013 in response to the rapid rise in youth unemployment following the recession and concerns that relatively few employers were offering work experience or employment opportunities to young people. The Commission was led by Sir Ian Wood and tasked to develop recommendations to improve young people's transitions from education to employment.
• The Commission identified 39 recommendations – one of which was the establishment of regional industry-led groups to facilitate better engagement between employers and education.
• The DYW Regional Groups were established in response to this recommendation. They aim to provide leadership, a single point of contact and support to facilitate engagement between employers and education at a regional level.
• The DYW Regional Groups operate in a complex policy and delivery landscape amongst a wide range of other initiatives aimed at improving the labour market outcomes of young people.

Profile and operation of the Regional Groups

• The consensus amongst Regional Group Board Members was that the governance arrangements for the Regional Groups were working well, with 92% saying that they were effective or very effective.
• The Groups were found to be doing particularly well at creating a single point of contact for employers to engage with education, and for schools / colleges to engage with employers – one of their primary objectives.
• Most Regional Group Board Members were clear on the strategic objectives of DYW and the National Group, but were less positive about the guidance and level of feedback from the National Group.
• The strengths and skills of the executive teams were highlighted as key success factors for the Regional Groups. They were described as highly driven and essential for bringing pace and momentum to the work of the Groups.
• The funding received by DYW Regional Groups has been used for staffing and other overheads, marketing, campaigns, events and project activity. It has also been used to leverage significant additional investment from the Chambers themselves, employers, schools and other funding sources.

Monitoring and reporting

• The Scottish Government identified an overarching set of KPIs for the DYW programme of activity. Following their formation, each DYW Regional Group agreed an individual set of KPIs with the National Group. These were based on the National KPIs and tailored to suit their specific areas of focus.
• The four groups included within the evaluation were found to be reporting against a combined total of 30 unique KPIs, many of which were measuring similar things in slightly different ways.
• The monitoring and reporting arrangements for the Groups were found to work well at the regional level, mainly due to the comprehensive CRM systems used by the Chambers, but were less effective for enabling reporting at the national level.
• It was not possible to accurately assess how well or otherwise each of the groups was performing based on analysis of their KPI reports due to issues in the way data has been collected and reported, and the lack of comparability.
• National and Regional consultees reported that they would welcome a more streamlined and consistent set of KPIs for the DYW Regional Groups to report against. This would form the basis for on-going monitoring and a possible future impact assessment.

Engaging employers

• Improving the work-readiness of young people and giving back to the community were more commonly cited reasons for employers to engage in DYW activities than tangible business benefits.
• The nature of employer engagement activity varied by region according to their priority areas of focus. Around 90% of employers surveyed reported having undertaken at least one type of activity organised through the Regional Group – most frequently careers and skills fairs.
• Employers reported engaging in one off stand-alone activities more frequently than recurring activities that were more deeply embedded within the curriculum.
• Whilst a significant proportion of employers reported having not experienced any barriers to participating in DYW activities, others faced challenges in aligning their activities with school / college timetables and releasing staff.
• Employers valued the work of the Regional Groups and reported positive experiences of engagement with the Groups. Across all regions, employers felt that the work of the Regional Groups had helped to break down barriers to engagement with schools and simplify the landscape for them.

Engaging schools and colleges

• Schools and colleges reported similar motivations to employers for engaging in DYW activities. These mainly focused on improving the life chances of young people through developing their work-readiness, employability and job outcomes, rather than meeting statutory obligations.
• Schools and colleges reported a number of barriers to engaging with employers, including a lack of resources (time and budget), timetabling issues and competing policy priorities.
• Feedback from schools and colleges on the quality of the activities delivered through the Regional Groups was very positive. Several also commented on the enthusiasm, commitment and professionalism of the DYW staff teams.
• Potential areas for improvement highlighted by schools / colleges included: better targeting of hard-to-reach students, more opportunities to share best practice, and increased resources for schools to deliver.

Evidence of impact

• The evaluation found evidence of changed behaviour amongst employers that had participated in DYW activities, including offering more and / or better quality work experience placements and taking on more young people and apprentices.
• The majority (80%) of Regional Group Board Members that participated in the survey reported that the behaviour of employers in their region had changed for the better as a result of engagement with the DYW Regional Group.
• Almost all (96%) of the schools and colleges surveyed reported that engagement with the DYW Regional Group had led them to deliver more and / or better quality employer engagement activities.
• The majority (85%) of schools and colleges that participated in the survey reported that the activities delivered following engagement with the DYW Regional Groups had a positive impact on their students.
• Whilst the evidence gathered on the potential for impact was encouraging, it was widely acknowledged by consultees that it was still early days for the Regional Groups and therefore too soon to make a full assessment of impact.

Recommendations

The evaluation identified 13 recommendations for consideration by the Scottish Government, as well as National and Regional Group Members and stakeholders.

Recommendation 1: The strategic objectives of the DYW Regional Groups should be reaffirmed. This will provide a timely reminder of the long-term system change that the Groups are seeking to achieve, which will help inform decision making around priority areas of activity.

Recommendation 2: The restated strategic objectives should be aligned to the new streamlined KPIs that Regional Groups will be required to report against and should include guidance on future priority areas of activity.

Recommendation 3: The research tools used for the evaluation should be made available to all 21 Regional Groups to enable them to assess what they are doing well and where challenges remain in a consistent way. They should be encouraged to develop an action plan to address any areas identified for improvement through this process.

Recommendation 4: There are likely to be common areas identified for development and improvement across the Regional Groups and so consideration should be given to establishing a series of themed working groups to develop common approaches to addressing these. These should be led by the Regional Groups themselves, with the Scottish Government and National Group providing minimal input beyond establishing the process.

Recommendation 5: The Scottish Government should consider separating out responsibility for funding and development of the network. The Regional Groups are likely to be more willing to engage in open and developmental conversations if this was clearly decoupled from funding award and contract management processes.

Recommendation 6: The channels for communicating information from the National Group to the Regional Groups should be reviewed with a view to being strengthened. As part of this process, Link Members should liaise with the Chairs of each of the Regional Groups that they have been assigned to jointly agree an approach to future communication and engagement.

Recommendation 7: The new set of streamlined KPIs will need to be tightly defined, with clarity on how they should be measured and reported, if they are to be effective in enabling consistent reporting at the national level.

Recommendation 8: The National Group should take on a greater check and challenge role with the Regional Groups on their performance against the refreshed KPIs.

Recommendation 9: Consideration should be given as to how best to quantify the extent of employer investment that is being leveraged by the Regional Groups in a consistent way. This should form part of the guidance issued alongside the refreshed KPIs.

Recommendation 10: Consideration should be given as to whether the Regional Groups are doing enough to improve the work-readiness of young people given that this is the most frequently cited motivator for engagement by employers, schools and board members.

Recommendation 11: The Scottish Government should clarify their expectations for schools and colleges in relation to engaging employers. This could include guidance for local authorities in terms of their role in supporting the work of the Regional Groups. It could also involve placing a greater emphasis on the DYW agenda within the inspection framework for schools/colleges.

Recommendation 12: The next phase of development of the Network, including future priority areas of activity for the Regional Groups to focus on, should be informed by the views of young people.

Recommendation 13: A suite of standard tools / resources should be developed and shared with the Regional Groups to enable them to measure the outcomes and impact of their engagements with employers, schools, colleges and young people in a consistent way.