Developing the Young Workforce (DYW) regional groups: formative evaluation

8. Evidence of impact

Chapter Summary

• The evaluation found evidence of changed behaviour amongst employers that had participated in DYW activities, including offering more and / or better quality work experience placements and taking on more young people and apprentices.
• The majority (80%) of Regional Group Board Members that participated in the survey reported that the behaviour of employers in their region had changed for the better as a result of engagement with the DYW Regional Group
• Almost all (96%) of the schools and colleges surveyed reported that engagement with the DYW Regional Group had led them to deliver more and / or better quality employer engagement activities.
• The majority (85%) of schools and colleges that participated in the survey reported that the activities delivered following engagement with the DYW Regional Groups had a positive impact on their students.
• Consultees were also confident that the activities being delivered through the Regional Groups were having a positive impact on young people.
• The evidence gathered on the potential for impact was encouraging, although it was widely acknowledged by consultees that it was still early days for the Regional Groups and therefore too early to make a full assessment of impact.

Introduction

8.1 The evaluation was primarily a formative exercise, designed to capture lessons on what is working well and where challenges remain following the initial set up phase of the Regional Groups. However, some early evidence of impact was collected and this chapter reports on this, covering emerging evidence of impact on employers, schools and young people.

Employers

Employers reported offering more work experience placements, and taking on more young people and apprentices, as a result of engagement with the DYW Regional Groups.

8.2 One of the objectives of the evaluation was to assess the extent of cultural change being achieved through the activities of the DYW Regional Groups. Cultural change can be difficult to measure as it often involves subtle / imperceptible changes to the way in which an organisation operates or responds to opportunities – it can best be evidenced through changed behaviour.

8.3 The evaluation found evidence of changed behaviour amongst employers that had engaged in DYW activities through the Regional Groups. Figure 8-1 shows that these changes focussed on:

• Offering more and / or higher quality work experience placements – 39% of employers surveyed reported offering more work experience placements and 18% were offering higher quality placements
• Offering more and / or higher quality work inspiration activities – a quarter of employers reported that they were offering more work inspiration activities (such as attendance at careers and skills fairs, delivering careers talks and participating in enterprise activities) and 12% were offering higher quality inspiration activities
• Recruiting young people – almost a quarter of all employers surveyed reported that they had either started recruiting young people (7%) or recruited more young people (16%) as a result of engagement with the Regional Group
• Recruiting apprentices – 13% of employers surveyed had recruited more apprentices as a result of the engagement and 4% had started recruiting apprentices.

Figure 8‑1: What has your organisation done differently as a result of engagement with the DYW Regional Group? (Please select all that apply)

Source: SQW survey of employers
Base: 231

Board Members were confident that the behaviour of employers had changed for the better as a result of engagement with the Regional Group.

8.4 The majority (80%) of the Regional Group Board Members that participated in the survey reported that the behaviour of employers in their region had changed for the better as a result of engagement with the DYW Regional Group (Figure 8-2), including 20% thought that it had changed a lot for the better.

Figure 8‑2: How far has the behaviour of employers in the region changed as a result of engagement with the DYW Regional Group?

Source: SQW survey of Regional Group Board Members
Base: 25

In addition to changed behaviour, employers reported wider benefits from engagement with the DYW Regional Groups.

8.5 As referenced in Chapter 6 (Figure 6‑1), employer motivations for engaging with the DYW Regional Groups mainly centred around the potential benefits for young people, rather than themselves. Nonetheless, several employers consulted through the evaluation reported benefits to their organisation from engagement. One employer referenced the opportunity to raise their profile amongst the local community as being key benefit from engagement with the Regional Group. Another cited the opportunity to build their networks through attendance at regional events with other employers. A further benefit cited by one employer was the development opportunity for staff. DYW activities were considered aspirational, resulting in a strong "feelgood factor" for those involved.

Schools and colleges

The DYW Regional Groups have resulted in an increase in the volume and quality of employer engagement within participating schools and colleges.

8.6 Of the 71 schools / colleges that responded to the survey, 96% reported
that engagement with the DYW Regional Group had led them to deliver
more and / or better quality employer engagement activities.
Figure 8‑3 shows that:

• The greatest increases were reported in high volume activities, such as careers talks, networking with employers and careers and skills fairs
• Around a quarter of schools and colleges reported that they were delivering more or better quality enterprise activities as a result of engagement with the Regional Group – this has been a key focus for many of the Groups
• Schools and colleges also reported delivering more and / or better quality activities aimed at exposing young people to workplaces, such as work placements and employer visits, as a result of engagement with the Regional Groups
• Activities aimed at developing the employability skills of young people (such as CV workshops and mock interviews) also featured as having increased in volume and / or quality.

Figure 8‑3: Has your engagement with the DYW Regional Group led you to deliver more and / or better quality employer engagement activities for your students?

Source: SQW survey of schools and colleges
Base: 71

Most Board Members the work of the DYW Regional Groups has changed the behaviour of schools and colleges for the better.

8.7 The majority (92%) of the Regional Group Board Members that participated in the survey reported that the behaviour of schools and colleges in their region had changed for the better as a result of engagement with the DYW Regional Group (Figure 8-4), and over a third (35%) reported that it had changed a lot for the better.

Figure 8‑4: How far has the behaviour of schools and colleges in the region changed as a result of engagement with the DYW Regional Group?

Source: SQW survey of Regional Group Board Members
Base: 23

Schools reported a range of wider benefits from engaging in DYW activities.

8.8 As with employers, schools and colleges reported that their primary motivation for engaging with the DYW Regional Groups was to improve the work-readiness and employability of young people. However, several reported wider benefits to their organisation from engagement. These mainly related to:

• Improved profile within the local community – several schools reported building and strengthening their relationships with a wide range of local partners, including employers, other schools and colleges, training providers and third sector organisations through engagement with their Regional Group. These connections were reported to be resulting in additional benefits to the schools beyond DYW, for example through access to different sources of funding and support.
• Diversifying the curriculum – one school that received funding through the DYW Ayrshire Innovation Fund reported that it had enabled them to introduce an alternative learning environment for young people who were at risk of not achieving their National 5s and who were more likely to pursue vocational routes / pathways to employment.
• Resources – one school reported that they had gained a template for delivering a skills workshop from an employer intervention organised through the DYW Group, which they can use with other cohorts of young people.

Young people

Schools and colleges reported that the activities delivered following engagement with the DYW Regional Groups had a positive impact on their students.

8.9 The majority (85%) of participating schools and colleges reported that the activities delivered following engagement with the DYW Regional Group had a positive impact on their students (Figure 8‑5) and 44% said that the activities had a high positive impact. Whilst this is not a substitute for feedback from young people themselves, it is a useful proxy measure of the potential for impact amongst young people who have engaged.

Figure 8‑5: Overall, how would you rate the impact of the activities delivered following your engagement with the DYW Regional Group on your students?

Source: SQW survey of schools and colleges
Base: 68

Consultees reported a wide range of outcomes and benefits to young people from engagement in DYW activities.

8.10 Consultees reported a range of benefits to the young people that had participated in activities organised through the DYW Regional Groups. These mainly focussed on:

• Engagement – one consultee reported a "marked improvement" in attendance rates amongst young people participating in DYW activities. Another cited an example of a group of young people who had given up their Saturday to attend a community event. This was particularly notable as the group had previously had poor attendance rates and were considered at high risk of disengagement.

  "The students participating in the nail bar are experiencing enjoyment from school again. They are coming in with a smile on their face. Their work at the nail bar has given them a sense of ownership and belonging to the school that was lacking. They are very proud of what they have achieved. They are much more confident, happy and engaged in school life." School consultee

• Employability skills – including improved confidence, team working skills, communication, negotiation and business skills.

  "It has made young people much more marketable and employable. It has given them something substantial to put on their CV to demonstrate their skills to employers." School consultee

• Qualifications – one school reported an increase in the proportion of young people gaining a National 5 in literacy as a result of engagement with the initiative supported through the DYW Ayrshire Innovation Fund.

• Transitions – one school reported that young people engaging in DYW activities were more likely to sustain their post-school destinations. This was attributed to them making more informed choices through greater exposure to different options.

• Employment opportunities – one consultee reported that engagement with the DYW Regional Group had opened up part-time employment opportunities for a number of young people in their area, particularly during peak tourist season. For example, a local café had taken on several young people they had engaged through work experience placements.

  "The DYW Regional Group has created a lot of opportunities for young people in the region. Schools are now engaging directly with large employers that they would not otherwise have been able to. This has resulted in more varied and higher quality work experience placements and employment opportunities." Local Authority consultee

The consensus was that it was too early to make a full assessment of the longer term impacts on young people from engagement in DYW activities.

8.11 Whilst consultees were generally confident that the activities being delivered through the Regional Groups were having a positive impact on young people, it was acknowledged that this was "purely speculative" at this stage as:

• It was too early to tell – the Regional Groups have only been in operation for up to three years, and part of this time was spent setting up, and so the longer term impact of the activities they are delivering have yet to be fully realised

  "There is clear evidence that they have developed their employability skills, but the extent to which that will translate into improved job outcomes remains to be seen." School consultee

• There are no mechanisms in place to track impact – several consultees commented on the lack of tracking measures in place to assess the longer term benefits and impacts to young people from engagement in DYW activities

  "Pupils and skills practitioners have a greater awareness of the local labour market and the skills required to succeed. There are likely to be spill over benefits over the coming years, but we do not have the capacity to track progress / outcomes at the moment." Employer consultee

• It is difficult to attribute change – the wide range of factors that influence a young person's life and learner journey make difficult to isolate the specific impact of the interventions delivered through the DYW Regional Groups.

  "It will be particularly difficult to measure this long term as it is hard to attribute positive destinations for young people to these types of activities." School consultee