Developing the Young Workforce (DYW) regional groups: formative evaluation

9. Conclusions and recommendations

9.1 This report has taken a detailed look at the operation of the DYW Regional Groups, through in-depth exploration of what is working well and where challenges remain in four of the Groups. This final chapter provides summary conclusions and resultant recommendations for consideration by the Scottish Government, the DYW National Group and the DYW Regional Groups.

Policy context

9.2 The DYW Regional Groups form one part of the Government's approach to improving the labour market and employment outcomes of young people in Scotland. They are operating in a complex policy and delivery landscape amongst a range of other initiatives with similar objectives. Their remit is to provide a single point of contact and support to facilitate increased engagement between employers and education. The long-term vision is for cultural change within the skills system, with many more employers engaged in education, provision better aligned to employer need and all young people given exposure to the world of work whilst in education.

9.3 When the strategic framework for the Regional Groups was launched more than three years ago, the messaging was focussed on addressing the rapid increase in youth unemployment following the recession. Since then, the labour market context has changed, with more young people choosing to stay in education for longer and youth unemployment at its lowest level in more than a decade. It is therefore a good time to re-emphasise the strategic priorities of the Regional Groups, with a greater emphasis on the long-term goal of achieving cultural change within the system.

9.4 This would help address feedback from Regional Group Board Members that they would like greater clarity from the National Group on their priority areas of activity. This is in part a response to them being asked to contribute across much of the policy landscape (and to do so alongside others and with limited resources). It is also in part due to the wide range of Key Performance Indicators (KPIs) they are currently required to report against, some of which they consider to be outside their direct influence (such as take up of Modern Apprenticeships).

9.5 The Scottish Government has developed a new streamlined set of KPIs that all the Regional Groups will be required to report against in future. Alongside a relaunch of the strategic objectives of the Groups, these should provide a firm basis for prioritising future activity.

Recommendation 1: The strategic objectives of the DYW Regional Groups should be reaffirmed. This will provide a timely reminder of the long-term system change that the Groups are seeking to achieve, which will help inform decision making around priority areas of activity.

Recommendation 2: The restated strategic objectives should be aligned to the new streamlined KPIs that Regional Groups will be required to report against and should include guidance on priority areas of activity.

Profile and operation of the Regional Groups

9.6 The governance arrangements for the Regional Groups were found to be working well. The Groups were reported to have been particularly successful in creating a single point of contact for employers to engage with education, and for schools / colleges to engage with employers – one of their primary objectives.

9.7 Regional Group Board Members were clear on how their activities contributed to the wider DYW policy agenda and to the strategic objectives of the National Group. However, they were less positive about the level of guidance and feedback received from the National Group. The main communication mechanism is through each Group having a dedicated Link Member on the National Group. The consensus was that, whilst the Link Member model was good, there was a lot of variability in how it was being implemented across the regions, particularly in relation of the level of engagement and support provided by Link Members.

9.8 The Scottish Government hosts meetings of the executive leads for each of the 21 Regional Groups. Whilst these were reported to be useful for networking, the consensus was that they were not conducive to collaborative working and sharing of best practice as they were too large, making it difficult to get into the detail. The fact that they were hosted by the funding body for the Groups (the Scottish Government) seemed to be creating an environment where individual Groups were keen to impress and showcase the work they were doing. This was found to be contributing to a culture of competition between the Groups, which was acting as a barrier to collaborative working.

9.9 Related to this, the evaluation found some resistance on the part of the Regional Groups to adopt tools and approaches that had been developed elsewhere, particularly where they felt these did not fit with their priority areas of activity or the needs of the region. One route to securing buy-in to national approaches would be to provide the Regional Groups with the opportunity to contribute to their development.

Recommendation 3: The research tools used for the evaluation should be made available to all 21 Regional Groups to enable them to assess what they are doing well and where challenges remain in a consistent way. They should be encouraged to develop an action plan to address any areas identified for improvement through this process.

Recommendation 4: There are likely to be common areas identified for development and improvement across the Regional Groups and so consideration should be given to establishing a series of themed working groups to develop common approaches to addressing these. These should be led by the Regional Groups themselves, with the Scottish Government and National Group providing minimal input beyond establishing the process.

Recommendation 5: The Scottish Government should consider separating out responsibility for funding and development of the network. The Regional Groups are likely to be more willing to engage in open and developmental conversations if this was clearly decoupled from funding award and contract management processes.

Recommendation 6: The channels for communicating information from the National Group to the Regional Groups should be reviewed with a view to being strengthened. Link Members should liaise with the Chairs of each of the Regional Groups that they have been assigned to jointly agree an approach to future communication and engagement.

Monitoring and reporting

9.10 A key area for exploration through the evaluation was the effectiveness of the monitoring and reporting arrangements of the Regional Groups. These were found to be working well at the regional level, with Board Members generally happy that they were getting the right level of information required to inform decision making.

9.11 However, the process was not working as well at the national level due the range and diversity of KPIs that the Regional Groups were reporting against. This was making it impossible to read across the Groups to report on what was being delivered nationally or to fully understand relative performance, and provide appropriate check and challenge on performance against these.

9.12 A separate piece of work is underway to address this issue and develop a streamlined set of KPIs that all Regional Groups will be required to report against. Whilst this was welcomed by the Regional Groups, there was concern raised around the definition of the terminology within the new KPIs – "strategic partnerships" was identified by some as being particularly problematic. There were also concern about how the new KPIs could limit the potential for local discretion on the priority areas of activity for the groups.

Recommendation 7: The new set of streamlined KPIs will need to be tightly defined, with clarity on how they should be measured and reported, if they are to be effective in enabling consistent reporting at the national level.

Recommendation 8: The National Group should take on a greater check and challenge role with the Regional Groups on their performance against the refreshed KPIs.

Recommendation 9: Consideration should be given as to how best to quantify the extent of employer investment that is being leveraged by the Regional Groups in a consistent way. This should form part of the guidance issued alongside the refreshed KPIs.

Engaging stakeholders

9.13 The evaluation found the motivations for engagement with the DYW Regional Groups to be broadly aligned between schools, employers and Regional Group Board Members. These mainly focussed on improving the work-readiness, employability and job outcomes of young people, rather than individual, organisational or statutory objectives. This is a positive finding as it aligns with the policy expectation. However, it does raise a question around the extent to which employers will continue to engage in the absence of tangible business benefits, and also whether they would be willing to pay for such activities in future.

9.14 The Groups were found to have been successful in engaging a high volume of employers in DYW activities. This was helped by having access to Chamber networks and contacts from the outset. However, there is a question around how many 'new' employers they have managed to get on board, particularly SMEs. The Groups got off to a slower start in terms of engaging schools / colleges, although they have all made good progress on this front and were found to be working with most of the schools / colleges within their respective areas. Schools referenced competing policy priorities as a barrier to engaging in DYW activities, as well as challenges navigating the cluttered landscape of provision.

9.15 A key challenge for the Regional Groups has been balancing the needs and priorities of both employers and schools, which are not always directly aligned. There also appears to have been a lack input from young people into what guidance and support they would like from employers to help them progress.

Recommendation 10: Consideration should be given as to whether the Regional Groups are doing enough to improve the work-readiness of young people given that this is the most frequently cited motivator for engagement by employers, schools and board members.

Recommendation 11: The Scottish Government should clarify their expectations for schools and colleges in relation to engaging employers. This could include guidance for local authorities in terms of their role in supporting the work of the Regional Groups. It could also involve placing a greater emphasis on the DYW agenda within the inspection framework for schools/colleges.

Recommendation 12: The next phase of development of the Network, including future priority areas of activity for the Regional Groups to focus on, should be informed by the views of young people.

Evidence of impact

9.16 Whilst it was too early to make a full assessment of impact, the evaluation did find evidence of changed behaviour amongst employers, schools and colleges as a direct result of engagement with the Regional Groups, as well as some evidence of positive impacts on young people. It also found that the Groups were taking different approaches to measuring impact locally. There is potential to introduce more consistent approaches to ensure both quality and consistency in how impact is being measured locally, and to enable collation of the results to report at the national level.

Recommendation 13: A suite of standard tools / resources should be developed and shared with the Regional Groups to enable them to measure the outcomes and impact of their engagements with employers, schools, colleges and young people in a consistent way.