Scottish Mentoring and Leadership Programme: interim report - qualitative process and impact assessment

Executive summary

Background

The Scottish Mentoring and Leadership Programme (SMLP) aims to support and improve the outcomes of care experienced young people, and those in ‘tough realities’ (such as those on the cusp of care, young carers, and those living in SIMD 1 or 2 areas). SMLP is one of several initiatives aiming to improve the wellbeing of children and young people. It is predicated on the idea that young people can be empowered by developing their relationships with family, friends and adults in the wider community, and that more relational forms of support will enhance well-being and develop young people’s and their families’ capabilities, thereby improving their short, medium and long-term outcomes.

There are two strands to SMLP: MCR Pathways and Columba 1400 Young People’s Leadership Academies (Columba 1400 YPLAs):

• MCR Pathways is a school-based mentoring scheme. The scheme engages S1 and S2 pupils in group work and then matches them with a mentor from the local community, who meets with them in school, once a week, from S3 onwards. The intervention aims to improve attainment and staying on rates for S5 and S6 pupils, as well as supporting young people to develop key life skills. By doing so, the aim is to increase the number of young people in full-time work, college or university after leaving school (known as “positive destinations”).
• Columba 1400 YPLA provides values-based leadership experiences for young people. These are delivered in three phases over the course of S2/3 (starting with group work in school, followed by a residential trip to Skye at their purpose-built community and international leadership centre, then a group project back at school). It aims to support young people to develop leadership skills, create better relationships and improve confidence and wellbeing so they can lead lives of purpose, meaning and contribution.

Following early evidence from an independent evaluation that MCR Pathways has had a positive impact on outcomes and capabilities, the Scottish Government and the Hunter Foundation agreed to invest additional funding to expand MCR Pathways and roll it out over a six year period (from 2021/22 to 2026/27). Alongside this, Columba 1400 would deliver values-based leadership training to young people via their YPLAs.

It was envisaged that the two schemes would complement each other in many schools, with schools involved in MCR being encouraged to participate in Columba 1400 as well and vice versa.

Aims and methodology

The aim of the evaluation is to assess the effectiveness of SMLP. There are three main strands to the evaluation, which explore:

• The delivery of the programme
• The impact on ‘hard’ outcomes (attendance, attainment, staying-on rates and positive post-school destinations) and on capabilities which young people have said matter to them (such as increased social confidence, increased health and well-being, and better relationships).
• Value for Money (VfM) - assessing the costs against monetised benefits (including the longer-term benefits to wider society).

This report uses qualitative data to explore both process and outcome findings, addressing the evaluation questions in relation to the reach, implementation and impact of the programme (see Appendix 1 for the Evaluation Framework and Theory of Change). It will later be complemented by a quantitative impact assessment (see next steps below).

The findings in this report are based on visits to four secondary schools; two taking part in MCR Pathways (for S1 to S6 pupils) and two taking part in Columba 1400 YPLA (for S2 and S3 pupils). The schools were selected with the aim of achieving a mix in terms of local authority area, geographic location, rurality, proportion of care experienced pupils, proportion of pupils living in the most deprived areas (SIMD1), school size, and whether the schools were non- denominational or Roman Catholic.

In total, 75 qualitative depth interviews were conducted with pupils, parents/carers, MCR mentors, Columba 1400 delivery staff and teachers, as well as with local authority and national stakeholder representatives.

Key findings

Impacts

There is qualitative evidence that both interventions – MCR and Columba 1400 – have had a positive impact on the young people who have taken part in them.

Increased confidence was identified as one of the main impacts of both interventions, particularly in relation to increased social confidence and self-belief. For those participating in the MCR intervention, this increase in confidence was attributed, in part, to encouragement from mentors and MCR Co-ordinators. These individuals were described as ‘patient’, ‘non-judgemental’, and ‘reliable’ which allowed them to build strong relationships with pupils. For those participating in the Columba 1400 intervention, the group work, in particular, was felt to have encouraged young people to speak to their peers and helped them feel more comfortable in group settings.

The other capabilities identified as having been most impacted by the MCR intervention included increased academic confidence, increased attainment/achievement and confidence in doing well at a job in future. These were felt to be closely linked with increased social confidence.

There were, however, also some pupils for whom mentoring seemed to have made little difference, either because they perceived themselves to be doing well across the capabilities already or because they still appeared to be struggling with confidence and were not comfortable opening up to their mentor.

The impact of the Columba 1400 YPLA was described as ‘powerful’, ‘affirming’ and ‘transformational’. Increased social confidence and self-belief underpinned many of the other capabilities that pupils felt the YPLA had improved. These included belief in their abilities, increased social capital, and feeling more involved in school decisions. Broader impacts of the intervention were also noted, including the embedding of a mentoring and leadership culture in the schools (e.g. through encouraging wider discussions about values) and benefits for staff who took part (e.g. developing their leadership skills).

Reach

Both interventions were generally considered to be reaching the young people in schools who would benefit most from them.

Reaching the care experienced young people who are the most disengaged from school, however, was identified as a key challenge for MCR Pathways. As a school-based scheme, it was recognised that MCR cannot be expected to reach young people who are not at school. Nevertheless, this highlights the additional disadvantage faced by this group.

In relation to the Columba 1400 YPLA, a reflection among staff in some schools who reported a lack of engagement from care experienced young people, was that these young people might be targeted more directly in the future to encourage greater participation.

Implementation

It was generally agreed that both of the interventions had been implemented as intended.

Aspects of the MCR Pathways implementation that were seen to be working particularly well included:

• Co-ordinators being members of school staff so they were better able to raise awareness of the intervention, work closely with other staff members and build positive relationships with pupils, so avoiding duplication of work already being done by Pupil Support Assistants. This was also felt to help Co-ordinators build relationships with pupils.
• Co-ordinators introducing the scheme to young people, which had helped ensure it was communicated effectively.

MCR taking the lead on Co-ordinator and mentor recruitment, since it was reported that local authorities would struggle to find the resource for this and MCR have the experience and expertise to do this well. The main implementation challenges identified in relation to MCR Pathways were the cost to local authorities of employing school Co-ordinators in a challenging financial environment, and attracting and retaining the right staff for temporary contracts.

Aspects of the Columba 1400 Young People’s Leadership Academy that were seen to be working particularly well included:

• The positive relationship between Columba 1400 and schools facilitated by the flexibility of the intervention (e.g. school staff considered Columba 1400 staff to have a clear understanding of their school’s needs and to have tailored the YPLA accordingly).
• The professionalism and skills of Columba 1400 staff in getting to know the young people, understanding their needs and building trust from an early stage.
• The timing (i.e. taking place over S2/S3), pace and content of the intervention.

The main implementation challenge for the Columba 1400 YPLA was the process of selecting young people to take part, due to the limited number of places and the target criteria (set by the Scottish Government) of 10 young people who are care-experienced or in tough realities. Engaging care experienced young people was felt to be a particular challenge because either there were not any in the year group to begin with, or because of their suitability to take part in the YPLA at that point. This underscored the importance of schools being responsible for the selection of young people as they know pupils’ individual circumstances and what is appropriate for them, while also factoring in the target criteria.

Implementation of the SMLP programme overall

Despite the individual strands being implemented as intended within schools, the research found that the speed and scale of rollout of the SMLP overall has adapted over time and has been considerably more limited than originally intended. Stakeholders indicated that the main reason for this was the much less favourable fiscal climate which had developed since the start of the programme. It was suggested that the original funding model, which required matched funding from local authorities/schools, was no longer sustainable.

In addition, the original vision of the two strands complementing each other within schools has not materialised. Cost is, again, likely to be a major factor with schools/local authorities feeling that they cannot justify funding two interventions in the same school. One national stakeholder questioned how much this mattered if both interventions were delivering what schools wanted (albeit separately).

Suggestions for improvement

The suggestions for improvement given by participants of this research were largely practical considerations specific to each intervention.

For MCR Pathways these included:

• Additional support for mentors (e.g. opportunities in the early stages to have a one-to-one conversation with an experienced mentor, greater signposting of resources, evening training sessions and more use of online breakout rooms, more information on what school is like for young people nowadays, and where to find information to advise young people on post-school opportunities).
• Providing more information to parents on the types of activities involved in the intervention and a little more information on their child’s mentor.
• Establishing formal end of year meetings between MCR and school staff to review the impact of the intervention.
• More active promotion of MCR Pathways in their area by Co-ordinators.
• Involving young people in Co-ordinator recruitment.
• Ensuring Co-ordinators can access school IT systems.
• Mentors and mentees being able to meet outside school settings.

For Columba 1400, these included:

• Giving more information to school staff and parents to explain what the YPLA would involve.
• Having a greater number of sessions in phase one (which comprises group work in preparation for the residential).
• Although the five day residential in Skye was generally considered to be the right length, there was a view that spending slightly more time at the residential would be beneficial.

Next steps and recommendations

The next steps for the evaluation are to conduct an interim quantitative impact assessment and an interim Value for Money assessment of MCR Pathways (reporting in March 2025). Following further qualitative fieldwork (on both the MCR and Columba 1400 strands) in autumn 2025 and quantitative/Value for Money analysis in winter 2025/26, the final evaluation report is due in March 2026.

Recommendations for possible additional steps for Scottish Government include:

• If the forthcoming quantitative analysis and Value for Money assessments of MCR Pathways are positive, review the funding model (including exploring alternative sources of funding both within and outwith the Scottish Government) to enable MCR Pathways to be rolled out further and sustained.
• Conduct an evaluability assessment of the Columba 1400 YPLA to assess the feasibility of a quantitative impact and Value for Money assessment, which could then inform future funding decisions.