Publication - Progress report
Scottish Mentoring and Leadership Programme: interim report - qualitative process and impact assessment
The Scottish Mentoring and Leadership Programme (SMLP) supports disadvantaged youth through MCR Pathways, a mentoring program improving education and life skills, and Columba 1400, which fosters leadership and confidence. The program has enhanced young people's wellbeing and outcomes.
Appendix 1: Theory of Change and Evaluation Framework
Theory of Change
Theory of Change text
The resouces needed to input into the SMLP are:
- Mentoring programme
- Pathways Coordinator
- Engagement with schools
- Leadership Academy
The activities intended to be delivered under the SMLP are:
- Mentor training
- 1-2-1 mentoring sessions
- Coordinator group and 1-2-1 sessions
- Young people who would benefit supported to participate
- Relationship planning with school
- Whole school day activity
- Residentials
- Community projects
The programme is expected to achieve the following ‘being capabilities’:
- Mentor / employer benefits: mentor skills and confidence; perceptions of young people
- Increased social confidence: communication skills; self-esteem
- Enhanced social capital: social network; family relationships
- Increased academic confidence: growth mindset
- Increased workplace knowledge and skills: understanding world of work; workplace confidence; employability skills
- Higher aspirations: young person; family
- Improved health and wellbeing: happiness; anxiety
The programme is expected to achieve the following ‘doing capabilities’:
- Mentor actions: sustained mentoring relationships
- Increased engagement in learning: staying on rates; attendance
- Increased attainment / achievement: improvements in literacy / numeracy
- Stronger young person voice in schools: participation in school decisions; leadership skills in practice
- Improved relationships with peers, families and key adults: relationship with mentor, relationship with teachers; family wellbeing
- Subject / course choice: subject choice; qualification level
- Engagement in wider activities: participation in extracurricular activities; participation in employability activities
SMLP is expected achieve the following impacts:
- Increased attainment / achievement: improvements in literacy / numeracy; SQA results; wider achievements
- Sustained positive destinations: progression to further / higher education; sustained employment; training
- Reduction in the Attainment Gap: literacy; numeracy; wellbeing
- School benefits: ripple effects for teachers and other pupils
- Employer benefits: employee skills and confidence; engagement with young people
Evaluation framework
| Evaluation questions (process) | Primary source(s) of evidence | 2022/3 (Year 1) | 2023/4 (Year 2) | 2024/5 (Year 3) | 2025/6 (Year 4) |
|---|---|---|---|---|---|
| Has the Programme been implemented as intended? | Qualitative fieldwork with stakeholders (SG, Hunter Foundation, MCR, C1400, LA Education Leads, school staff) | - | Oct 23-Jan 24 | - | Oct 25-Jan 26 |
| Has the Programme been implemented as intended? | Analysis of delivery monitoring information from MCR and C14002 | - | Dec 23- Jan 24 | - | Dec 25-Jan 26 |
| What has worked well and what can be improved in the implementation? | Qualitative fieldwork with young people, parents/carers, mentors, MCR Pathways Co-ordinators, employers, and other stakeholders (SG, Hunter Foundation, MCR, C1400, LA Education Leads,school staff) | - | Oct 23-Jan 24 | - | Oct 25-Jan 26 |
| To what extent has the Programme embedded mentoring and leadership across the school system? | Qualitative fieldwork with stakeholders (mainly LA Education Leads, school staff) | - | Oct 23-Jan 24 | - | Oct 25-Jan 26 |
| To what extent has the Programme engaged with and supported young people? | Qualitative fieldwork with young people, parents/carers, mentors, MCR Pathways Co-ordinators, C1400 staff, school staff | - | Oct 23-Jan 24 | - | Oct 25-Jan 26 |
| To what extent has the programme developed a positive relationship with DYW, SDS and the Promise? | Qualitative fieldwork with representatives from DYW, SDS, the Promise, MCR, C1400 | - | Oct 23-Jan 24 | - | Oct 25-Jan 26 |
| How is the programme delivery changing and improving over time? | Qualitative fieldwork with young people, parents/carers, mentors, MCR Pathways Co-ordinators, employers, and other stakeholders (SG, Hunter Foundation, MCR, C1400, LA Education Leads, school staff) | - | Oct 23-Jan 24 | - | Oct 25-Jan 26 |
| Are participants being reached as intended? | Analysis of delivery information from MCR and C1400 | - | Dec 23- Jan 24 | - | Dec 25-Jan 26 |
| Are participants being reached as intended? | Qualitative fieldwork with stakeholders (SG, Hunter Foundation, MCR, C1400, LA Education Leads, school staff) | - | Oct 23-Jan 24 | - | Oct 25-Jan 26 |
| Evaluation questions (impact) | Primary source(s) of evidence | 2022/3 (Year 1) | 2023/4 (Year 2) | 2024/5 (Year 3) | 2025/6 (Year 4) |
|---|---|---|---|---|---|
| To what extent has the programme supported the capabilities of young people? (i.e. The 'being' and 'doing' capabilities from the logic model) | Qualitative fieldwork with young people, parents/carers, school staff | - | Oct 23-Jan 24 | - | Oct 25-Jan 26 |
| To what extent has the programme supported the capabilities of young people? (i.e. The 'being' and 'doing' capabilities from the logic model) | Analysis of feedback data from MCR (short 4 question survey for mentors and YP to complete at every session currently being developed, will probably include questions on agency, problem solving and self-worth) and C1400 YPLA survey (self-rated confidence, knowing own values, knowing what is important to me, feeling positive about future, motivation, leadership, working with others) and C1400 WEMWBS (well-being) | - | 6 monthly from Autumn '23 to Autumn '25 | 6 monthly from Autumn '23 to Autumn '25 | 6 monthly from Autumn '23 to Autumn '25 |
| To what extent has the programme contributed to academic attainment of participating young people? | Quantitative analysis of data on attendance (SG data), attainment (SQA via ADR Scotland), staying- on rates (SG data) and positive destinations post-school (SDS data) for intervention group and control group | May-July 23 | May-July 23 | May-July 25 | - |
| Did the Programme achieve the expected outcomes? | Quantitative analysis of data on attendance, attainment and staying-on rates for intervention group and control group | May-July 23 | May-July 23 | May-July 25 | - |
| Did the Programme cause the difference? | Quantitative analysis of data on attendance, attainment and staying-on rates for intervention group and control group | May-July 23 | May-July 23 | May-July 25 | - |
| Have different groups been impacted in different ways, how and why? | Quantitative analysis of data on attendance, attainment, staying-on rates and positive destinations post-school for intervention group and control group; analysis of differences within the intervention group (e.g. by gender, level/type of care experience) | May-July 23 | May-July 23 | May-July 25 | - |
| Have different groups been impacted in different ways, how and why? | For the 'how' and 'why'? elements: qualitative fieldwork with young people, parents/carers, mentors, MCR Pathways Co-ordinators, C1400 staff, school staff | - | Oct 23-Jan 24 | - | Oct 25-Jan 26 |
| Are young people achieving sustained positive destinations as a result of the Programme? | Quantitative analysis of positive destinations post-school | May-July 23 | May-July 23 | May-July 25 | - |
| Evaluation questions (value for money) | Primary source(s) of evidence | 2022/3 (Year 1) | 2023/4 (Year 2) | 2024/5 (Year 3) | 2025/6 (Year 4) |
|---|---|---|---|---|---|
| What are the costs of the programme and how does the cost of its work compare to initial expectations? | Cost data from SG, MCR, LAs and schools related to the funding and running of the programme: operational costs, staff costs, materials costs, direct funding awards. | - | May-Sept 23 | - | May-Sept 25 |
| How does the programme’s effectiveness compare to other interventions that could achieve similar outcomes? | Value for Money analysis based on quantitative impact data, costs and benchmarking against other interventions (where data available). (See note 1 for more details). | - | Aug-Dec 23 | - | Aug-Dec 25 |
| How can we robustly translate the programme’s initial impacts (e.g. on positive destinations) into longer term monetisable benefits for themselves and others? | Cost Benefit Analysis will focus primarily on academic/employment destinations associated with mentoring using data collated from educational datasets. The value for money analysis will apply different modeling approaches to estimate the lifetime monetary benefits to the individual depending on whether the impacts of mentoring were associated with academic or vocational qualifications/apprenticeship placements. Other outcomes like attendance can be monetised using indicative cost savings to the exchequer related to truancy. (See note 2 for more details). | - | Aug-Dec 23 | - | Aug-Dec 25 |
| How do the monetised benefits compare to costs, and how should we account for other important benefits that are too difficult to monetise? | Cost Benefit Analysis will focus primarily on academic/employment destinations associated with mentoring using data collated from educational datasets. The value for money analysis will apply different modeling approaches to estimate the lifetime monetary benefits to the individual depending on whether the impacts of mentoring were associated with academic or vocational qualifications/apprenticeship placements. Other outcomes like attendance can be monetised using indicative cost savings to the exchequer related to truancy. (See note 2 for more details). | - | Aug-Dec 23 | - | Aug-Dec 25 |
| Where and how can we monetize improved capabilities that do not immediately lead to direct economic benefits | Cost Benefit Analysis will focus primarily on academic/employment destinations associated with mentoring using data collated from educational datasets. The value for money analysis will apply different modeling approaches to estimate the lifetime monetary benefits to the individual depending on whether the impacts of mentoring were associated with academic or vocational qualifications/apprenticeship placements. Other outcomes like attendance can be monetised using indicative cost savings to the exchequer related to truancy. (See note 2 for more details). | - | Aug-Dec 23 | - | Aug-Dec 25 |
Contact
There is a problem
Thanks for your feedback