Scottish Mentoring and Leadership Programme: interim report - qualitative process and impact assessment

Chapter 3: Columba 1400

This chapter focuses on the Columba1400 Young Peoples’ Leadership Academy and explores the perceived reach, implementation, and impact of the intervention. It draws on the perspectives of pupils from two secondary schools who have taken part, some of their parents/carers, the school staff involved, Columba 1400 staff, local authority representatives and national stakeholders.

Summary

• Columba 1400 YPLA was widely considered to be reaching the young people who would benefit from it most and engaging them at the right stage of secondary school. However, it was also felt that care experienced young people could be targeted more directly to encourage them to take part.
• It was generally agreed that the intervention is being implemented as intended. The relationship between Columba 1400 and schools, the professionalism and skills of Columba1400 staff, and the design and flexibility of the scheme were all aspects that were felt to be working particularly well.
• The selection process was highlighted as a challenging aspect of the scheme due to the limited number of places and target criteria (set by the Scottish Government) of 10 young people who are care-experienced or in tough realities. This underscored the importance of schools being responsible for the selection of young people as they know their individual circumstances.
• Some practical suggestions for improvement included giving more information to school staff and parents to explain what the YPLA would involve and having a greater number of sessions in phase one (which comprises group work in preparation for the residential). Although the five day residential in Skye was generally considered to be the right length, there was a view that spending slightly more time at the residential would be beneficial.
• Participants were very positive about the Columba 1400 YPLA overall, describing the impact it had as ‘powerful’, ‘affirming’, ‘life-changing’ and ‘transformational’.
• The capability that most strongly resonated with young people was increased social confidence. This was a broadly and strongly felt impact of the intervention. Increased confidence also underpinned many of the other capabilities that pupils felt the YPLA had improved (such as belief in their abilities, increased social capital, and feeling more involved in school decisions).
• Broader impacts of the YPLA included the embedding of a mentoring and leadership culture in the schools (for example, through encouraging wider discussions about values) and benefits for staff who took part (for example, developing their leadership skills).
• The potential of the Columba 1400 YPLA to help build out and deepen support for young people was recognised. However, several barriers to this happening were identified (including a lack of funding and a lack of strategic coordination).

Overview of Columba 1400 Young People’s Leadership Academy

Columba 1400’s Young People’s Leadership Academies provide values-based leadership experiences for young people. It is delivered in three phases over the course of S2/3 (see Chapter 1 for an overview of each phase). Some schools have ongoing engagement with Columba 1400 and have taken part multiple times with different year groups.

Implementation

Understanding of intervention aims

Evaluation participants broadly understood the Columba 1400 intervention to be about supporting young people to develop self-awareness, confidence and self-efficacy by developing values-based leadership skills.

“It’s about building confidence in young people. Young people that might have barriers; that they’re not attending or school isn’t for them or, kind of, friendship issues. So that they’re able to go out and succeed once they leave us.” Local authority representative

It was also perceived, by teachers and Columba 1400 staff in particular, to be about embedding a leadership and mentoring culture in the wider school, with the Columba 1400 young people cohort playing a key role in bringing this about.

“This is a journey for them, to help them understand what’s important to them, and for them to grow in confidence and self-awareness, and unleash those amazing skills and leadership that’s in them, to lead themselves, to lead others, but to also lead impact, at whatever level that means for them.” Columba 1400 staff

The Columba 1400 YPLA was also seen to play a role in the broader aims of the SMLP to reduce the poverty-related attainment gap. This was through levelling out the opportunities for young people, both at school and beyond. The YPLA was commonly described as a form of “scaffolding” around the young person.

Initial engagement with schools

Initial engagement with schools comes about in a range of ways, such as:

• Schools initiating contact with Columba 1400 (e.g. because they heard about the it through word of mouth, saw it advertised, or because members of staff took part in the Columba1400 Head Teachers’ Leadership Academy (HTLA).
• Existing engagement in related local initiatives (such as MCR Pathways or the Clackmannanshire Family Wellbeing Partnership).
• Columba 1400 reaching out to local authorities proactively.

Motivation to take part was particularly strong among headteachers/deputy headteachers who had direct experience of Columba 1400 through the HTLA.

“Professionally, for me, it is the best thing I have ever done, and I would say it is the best thing I have ever done for kids in 34 years’ teaching". Teacher, Columba 1400 school

Existing involvement with MCR Pathways was also identified as an encouraging factor for one school, as this positive experience provided reassurance around working with an external organisation.

A challenge identified during the initial engagement was in relation to procurement rules. One local authority representative explained that a tendering process had to be initiated due to the number of schools in the area participating and the total cost being above their threshold for non-competitive direct awards. Requirements of the tender process meant that the local authority would have to gather bids, for what was considered to be a bespoke intervention, despite having already had the taster sessions with Columba 1400. The local authority representative described this as time consuming and disruptive to the delivery of the scheme. A Columba 1400 member of staff said that this local authority is now procuring a framework for potential suppliers which will last for four years. A handful of other local authorities have done the same and they predict that others will follow suit.

Looking ahead, evaluation participants also highlighted funding as a potential barrier to future engagement of schools in the YPLA. It was noted that most schools use the Pupil Equity Fund (PEF) to cover costs, with some schools receiving more than others^[21]. Where funding has been an issue, Columba 1400 staff said they would share information with the schools about other sources of funding. Nevertheless, for any schools unable to cover the costs with PEF monies, it was recognised that this might be a barrier to participation.

Are participants being reached as intended?

The Columba 1400 YPLA was widely considered, particularly by Columba 1400 staff and school staff, to be reaching the young people who would benefit from it most.

Initial engagement with young people

The YPLA was open to the whole year group. Pupils were made aware of it through an assembly where a representative from Columba 1400 told them about the YPLA (either in person or via video call).^[22] Additionally, pupils were made aware of it through notices in class or during registration. For those registering an interest, a taster session was run by Columba 1400 staff to give pupils a deeper understanding of what the YPLA involves. Pupils were then invited to submit a written application, and teachers would then select the final 16 who would take part. This process is discussed in more detail below.

Columba 1400 staff felt this initial engagement phase, prior to pupils submitting an application and being selected, was a key factor in ensuring there is meaningful engagement with young people at the right time.

Schools reported that levels of interest among young people tended to be high initially, although numbers typically dropped between the taster session and application stages. The young people who took part said that their main motivations were to improve their self-confidence, to make friends, or to try something different by going on a trip. The prospect of being away from home was an initial worry for some but did not appear to pose a significant barrier among those interviewed who went to Skye.

“I had really bad social anxiety and couldn’t talk to people. I wanted to be more comfortable talking around people and talking to people. I used to struggle a lot. [Columba 1400] helped a lot. The second I heard about it, and some of the experiences, I was instantly intrigued and went to speak to [the teacher] about it”. Pupil, S4, Columba 1400 graduate

Young people who took part did not report having many concerns or hesitations about signing up, other than some practical questions (such as whether dietary requirements would be catered for, or among those who suffered from travel sickness, what the journey would be like) which were addressed during phase one. It should be noted that the research did not include any young people who were either not selected or decided not to participate in the YPLA, so these could have been barriers for them.

Potential barriers to engagement were highlighted by staff. Teachers noted that not knowing specifically what the YPLA involved may have been off-putting for some young people. They understood why Columba 1400 didn’t give too much away and felt that young people enjoyed the element of surprise (for example, the search and rescue task). However, they suggested that more information for teachers in the initial engagement stages might enable them to help young people overcome any hesitations about taking part (particularly among school staff who were less familiar with the intervention having not participated in the HTLA or previously accompanied pupils on the residential).

“Out of the whole cohort, we were expecting to be overwhelmed [by applications], but there wasn’t as many as we thought, which could be because it was hard to sell. That was the biggest challenge for me. Once we had the group it was quite straightforward.” Teacher, Columba 1400 school

It was also recognised that the residential element was a barrier for those in challenging circumstances for whom going away would be impractical or inappropriate. Columba 1400 staff noted that the place-based academy (a new approach which involves activities within the local community and does not include a residential element) has been developed to engage with these young people.^[23]

The application process for pupils

Tailoring the approach to applications for individual schools and young people was also identified as a key factor in reaching those pupils who would benefit most. While the application process is designed to be accessible, Columba 1400 staff recognised that it may not suit all young people. Pupils are therefore encouraged to speak to Columba 1400 staff and teachers, so that they can be supported through this stage. One teacher recalled supporting a young person with their application, knowing that they were an individual who would benefit, but for whom the written aspect of the application was a barrier.

“If [the application form] really is a barrier, I make sure that they know they can come and speak to me at the end of that session and say, ‘I really don’t feel comfortable doing this, but I really want to do it’. To which I’ll say, ‘right, you need to go and speak to [the school staff]’. And that’s happened before, and that young person’s gone on the programme and [been] amazing. But sometimes that barrier of writing is just too much for them.” Columba 1400 staff

Selection

Following the application process, school staff select 16 young people to take part. The selection is guided by SMLP target criteria, set by the Scottish Government, which stipulate that 10 of the 16 young people should be care experienced or in ‘tough realities’ (such as those on the cusp of care, young carers, and those living in SIMD 1 or 2 areas).

Data provided by Columba1400 showed that:

• In year one of SMLP funding, of 160 young people taking part across 10 schools, 100 were care experienced or in tough realities (63%).
• In year two, of 346 young people taking part across 21 schools, 208 were care experienced or in tough realities (60%).

However, the composition of the group was said by respondents to be a further consideration in selection. Columba 1400 staff said they advise schools that selecting a “spectrum” or “screenshot” of the year group was also important in creating a productive group dynamic, maintaining a high quality of engagement, and ensuring the value of the experience to the young people.

Indeed, although it was broadly understood that each cohort should include 10 young people who are care experienced or in tough realities, the selection process was described as being “more fluid” compared to MCR Pathways which has more of a focus on young people in tough realities (including, but not limited to, care experienced young people).

School staff described their approach to selection as involving identifying pupils who:

• Were at risk of under-achieving
• “Don’t see their own potential”
• Were at risk of disengaging with school, or
• Are disadvantaged and/or who the school felt would benefit from respite from home (e.g. young carers).

Schools reported working with Pupil Support teams to identify priority young people among those who had applied, and one school staff member summarised this approach as considering “individual stories”.

The selection process was identified as one of the most challenging aspects of the YPLA for schools, particularly in relation to choosing 16 from the overall number of applicants and in understanding or meeting the target criteria. Care experienced young people were considered to be the hardest group to reach, due to both their individual circumstances (e.g. if disengaged from school or experiencing difficulties at home) and their relatively low numbers in the year group. School staff said they relied on guidance from Columba 1400 staff, as well as conversations with Pupil Support teams and families, to help with this.

“We weren’t sure how to interpret that at first, but [Columba 1400] said that anyone who struggles with friends or socially would count, so that helped us get the numbers up. I don’t think we would have been able to fill the spaces based just on care experience as we don’t have those numbers in the cohort.” Teacher, Columba 1400 school

Challenges relating to the target criteria were also recognised by Columba 1400 staff and, for them, this underscored the importance of schools knowing the circumstances of individual pupils when making their selections.

“You need to get it right for the young people. You know, it’s not picking your most vulnerable, vulnerable young people who can barely function. That’s unfair. You're setting a young person up to fail at that point.” Columba 1400 staff

“Young people have to be at the right stage to do it. You wouldn’t drop somebody into a lecture theatre who has got a reading age of four and chuck loads of paper at them, so it’s [about] where they are with their social-emotional literacy levels, to be able to work as part of the team […] It’s about, ‘How are you creating the conditions to really reach the young people and support them through their needs?’.” Columba 1400 staff

Improving reach

Among evaluation participants, there was a broad view that the more open approach to recruitment - inviting the whole year group to apply while providing supplementary support for those who need it - is the correct one. There was widespread trust in school staff to identify the right young people for the YPLA.

“I would want it done in a way where someone isn’t being selected or extracted for a programme […] Yes, we would be targeting/supporting for care experienced young people and ensuring their needs are catered for at the residential. Where I have an unease is that we don’t want to extract people from a period of learning and say ‘this is because…’ If something is universally offered but then something more intensive support such as coaching and mentoring [is needed], that’s what we should be doing.” Local authority representative

Nevertheless, participants suggested ways in which reach could be improved in future. Reflecting on the lack of engagement from care experienced young people in the first cohort taking part, one school said they would target this group more directly to encourage uptake next time. As noted above, school staff felt they would be more able to promote the YPLA and encourage uptake among target groups if they were more familiar with the detail of it.

It was also recognised that the young people taking part in the YPLA could be ambassadors to promote the experience among future cohorts. School staff noted that the young people who participated had been encouraging others to take part. Among young people themselves, when asked what they would say to someone thinking about taking part, the unequivocal response was that they would encourage others to apply.

"100%, definitely go for it. They don’t know what they’re missing out on […] If I was closer with them, they’d know how I never really spoke to people before, and they’d see how well I’d improved." Pupil, S4, Columba 1400 graduate

Columba 1400 staff also mentioned some partnership work with Heriot Watt University which is looking at how young people could be trained as ambassadors to make a positive difference in their local communities.

Has the YPLA been implemented as intended?

Overall, the YPLA was considered to have been implemented as intended. Participants indicated a clear understanding of the intervention’s aims (as outlined above), and, as one school put it, the YPLA “does what it says on the tin” in terms of implementation.

What’s working well

The relationship between the school and Columba 1400 staff was one aspect of the scheme that was considered to be working well. School staff felt that Columba 1400 staff had a clear understanding of each school’s needs, while Columba 1400 staff reported that relationships with schools were positively sustained through partnership facilitators working closely with the school and getting to know the staff and young people. One school reflected that having the same Columba 1400 staff in phase one and the residential also helped to build familiarity and trust in the process. The importance of this continuity was highlighted by Columba 1400 staff too.

The Columba 1400 staff were highly regarded by young people and by school staff. Young people described Columba 1400 staff as approachable and “friendly”. Teachers described them as “fantastic” and “impressive” in terms of how quickly they get to know the pupils and understand their needs in the early stages. Although Columba 1400 staff mentioned that there is a lot of administrative work for schools, which can be burdensome, the school staff interviewed for this research said they felt well supported.

“It was great, it was constant contact from the Columban staff about, “this is what you have to do next”, but they provide everything for you.” Teacher, Columba 1400 school

The intervention’s flexibility was also seen as a positive feature. It was felt that the scheme was tailored to the needs of each school, meaning that young people were being supported in the right way. One national stakeholder described the YPLA as always “evolving” in this way. In phase one in particular, Columba 1400 emphasised how getting a clear understanding of the school was vital for ensuring good preparation for the residential, allowing them to tailor delivery depending on each school’s style of working (e.g. having all the information upfront or given at relevant points in the process), and the pupils’ individual circumstances (for instance, based on any additional support or medical needs, their gender, or religion).

“[The programme] works best when we get an understanding of what the school needs.” Columba 1400 staff

The timing, pace and content of the YPLA were all viewed positively by evaluation participants. From the school perspective, S2 was considered a good point at which to engage pupils, as it was less intrusive on studies while allowing a few years for their leadership skills to benefit the school. Columba 1400 staff also acknowledged this, though they suggested that there was also value in engaging older pupils in the scheme, when they are more mature and able to have deeper group discussions.

Young people who had taken part were very satisfied with the delivery of the YPLA. They indicated that aspects they particularly enjoyed were the mix of indoor and outdoor activities, opportunities for teamwork and learning about values. The search and rescue activity was frequently mentioned as a particular highlight of the residential.

“I think it is all like really, like well thought out, and the planning of stuff was just perfect, and the times and everything like that.” Pupil, S4, Columba 1400 graduate

Parents also noted that they were not expected to provide anything, which made the scheme feel more inclusive and accessible.

What could be improved?

As mentioned previously, teachers felt that more information about the YPLA content for schools would help them manage the expectations of young people, ensuring they know what they are signing up for and encourage uptake. Some parents also said that they would have welcomed more information about the scheme at the start. At the same time, however, they were supportive of young people hearing about it directly from Columba 1400 and deciding for themselves whether to take part or not.

“It would give us a clearer understanding of what they actually did, what the aims were, because as I say, I think probably most teenagers, they maybe don't verbalise it particularly well." Parent of S4 pupil, Columba 1400 graduate

In terms of the YPLA delivery, participants offered some practical suggestions for improvement. Some phase one sessions had taken place online due to the COVID-19 pandemic, and it was broadly agreed that returning to face-to-face would better maintain engagement (and this is what C1400 are doing). One young person also suggested that there could be more sessions as part of phase one to explore the values more fully ahead of the residential.

Participants were very positive about the residential itself, although one suggestion for improvement included running more outdoor activities. This was raised by young people who found the reflective/writing exercises challenging, although they understood the purpose of them. The residential was generally felt to be the right length. However, some young people and staff mentioned that the residential could be extended to a full week to allow more time to fully explore both the Columba 1400 values and individuals’ values.

“Making it a bit longer because there are six core values at Columba 1400, and we had five days, and we did a value each day. We did two on the last day, I think. So, I just felt some [may have been] a bit rushed. I think if we could just stay for even like seven days it would be slightly better.” Pupil, S4, Columba 1400 graduate

Phase three was felt to be a more challenging part of the YPLA for some schools. Staff and pupils noted that engagement in the phase three project was not maintained by all who went on the residential, and Columba 1400 staff mentioned that staffing changes within schools can also disrupt progress. The timing of the residential relative to the school year can also be a factor; one school had returned from the residential just before the summer holidays and had lost a bit of momentum on their project by the time they returned.

However, Columba 1400 staff highlighted that young people did not necessarily have to engage in phase three for the impact of the intervention to be felt, as it could be demonstrated in other parts of their lives (see impacts).

How is YPLA delivery changing and improving over time?

Reflecting on how the intervention has changed over time, Columba 1400 staff mentioned that they have made the following improvements:

• Putting more time into phase one to help better prepare young people for the residential.
• Involving school staff more in phase one and having separate sessions with them. This was introduced to give teachers the opportunity to ask questions that they would not want to ask in front of pupils and to increase teacher engagement with the scheme.
• Introducing phase three while on the residential in Skye, rather than afterwards. This was perceived by Columba 1400 staff to have helped schools who were not sure how to approach the phase three project. It enabled them to make a start with Columba 1400 staff there to provide initial guidance in-person.
• While not covered as part of this research, the place-based academy was considered to be a further enhancement, as it allows the intervention to engage more young people for whom the residential element is a barrier.
• Forming different partnerships to deepen the support provided to young people (see the building out and deepening section of this report below for more details).
• Introducing a leadership passport. This is a document summarising a young person’s values and achievements to support conversations with careers advisers or other key adults.

To what extent has the YPLA supported the capabilities of young people?

Participants were very positive about the Columba 1400 YPLA overall, describing the impact it had as “powerful”, “affirming”, “life-changing” and “transformational”.

“Pupils come out of it knowing what they could do. Some who would never want to lead a group now want to, have made an effort in the project and have more of a voice now. It's built them up as individuals and fed into school leadership and community too.” Teacher, Columba 1400 school

In particular, increased social confidence and communication skills were common themes when discussing the overall impacts of the intervention. Pupils described feeling more confident in social situations as a result of taking part. They attributed this increased confidence to the amount of group work that they experienced, both in the sessions leading up to the residential and during the residential itself, which encouraged them to speak to their peers and helped them feel more comfortable in group settings. Staff also mentioned that some pupils had been chosen to take part because they were struggling with confidence – and they had observed significant changes in those pupils.

“I learned to be myself around a bigger group of people, rather than a small one, and it made me feel more confident talking to people. And the tasks helped that as well, because they put us into groups, so, I might have been in a group with people I didn’t really know that well but then, because of that, I got to know them and ended up talking to them. I think I’ve became a lot more comfortable around people.” Pupil, S4, Columba 1400 graduate

Pupils also highlighted how the Columba 1400 YPLA had improved their self-esteem and sense of belief in their abilities. They described having more confidence in themselves to contribute at school and to socialise or to try new things outside of school. Parents/carers and staff also observed more of a self-assuredness in the young people who, before taking part in Columba 1400, had lacked this:

“One of the boys had really low self-esteem before going. The difference in him, you see the confidence in him in classes, in the corridors he is chatting to people, he is not shying away now. He is out there; you see him and you know that he is there. So, for him, I think that is the biggie. He has come out of the corner and he is taking part in things.” Teacher, Columba 1400 school

In the next section, we look at each of the ‘being’ and ‘doing’ capabilities in turn. As outlined in the introduction, we explored each capability with pupils and parents in the form of statements (see Appendix 3 for the list of statements and related capabilities). Pupils who took part in these interviews were either in S4, having taken part in the residential the previous year and almost completed their phase three project, or in S3, having returned from the residential more recently and being part way through their phase three project. School staff, stakeholders and mentors were shown the logic model (see Appendix 1) and asked to pick out the capabilities where they felt the intervention had made the most difference and where it had made less of a difference.

Being capabilities

Increased social confidence

Young people who identified increased social confidence as an area where the YPLA had made a positive difference, described how they felt anxious in social situations, or avoided them altogether, prior to taking part. Working in groups encouraged them to get to know their peers and adults (teachers and Columba 1400 staff) which, in turn, made it easier for them to be in social situations beyond the YPLA. Since returning from Skye, pupils said they were finding group work at school easier and were also feeling more comfortable with different types of interactions outside of school (for instance, socialising with friends or ordering in a restaurant). Girls, in particular, mentioned this as an area that the YPLA had made a key difference to. As this pupil and her parent put it:

"I’m more confident around my peers and much more self-assured around them. I used to be quite shy and nervous, but now I’m comfortable talking to people in different environments." Pupil, S4, Columba 1400 graduate

"She was at her first house party on Halloween, which is something I just don't think she would have really imagined herself doing before." Parent of S4 pupil, Columba 1400 graduate

Related to this, pupils said they felt more comfortable speaking with other people as a result of taking part in Columba 1400 and had more confidence in their ability to do this. Boys in particular highlighted this as a key difference, and it was again attributed to the format of the YPLA which included a lot of groupwork activities to encourage young people to share their views. There was also a sense among those who identified with this impact that the YPLA had helped reduce barriers to speaking, such as fears around saying the wrong thing, making mistakes, or being judged by others. Several pupils commented that they would not have felt comfortable taking part in an interview for this research before the YPLA.

“It has given me a lot more confidence […] in group scenarios. Whereas usually, if I was in a group myself, I'm quite a shy person, I would maybe just let someone else take control, but now I feel like I would be […] more involved in the discussions.” Pupil, S4, Columba 1400 graduate

Columba 1400 staff attributed this impact to the facilitation of the YPLA and the environment within which it is delivered, which they felt empowered young people to try new things.

“It’s creating the environment in a massively deliberate way to help effect change, to help them feel safe enough, and then provide the right experiences that they can start taking those risks and trying things out that they wouldn’t try before.” Columba 1400 staff

Where increased social confidence was not something that pupils or parents/carers felt the YPLA had made a difference to, this was generally because it was not considered to be an issue for that young person to begin with.

Enhanced social capital

Enhanced social capital was evident among pupils who felt that their friendship network had widened as a result of taking part in Columba 1400. This was partly because they had made new friends as part of the YPLA, but also because they had built up confidence in their ability to make new friends in the future. Some pupils did not feel that taking part had resulted in them having more people to talk to but felt that the YPLA had given them confidence to talk to other people, such as peers or teachers, and not to bottle things up as much.

“I made a few friends there and I realise that people that you don't know as well are nicer to you, in my case anyway. So, anyone that I'm not really close with, we can actually talk a lot easier.” Pupil, S3, Columba 1400 graduate

A member of staff from Columba 1400 highlighted an instance where the intervention had enabled a pupil to positively influence existing friends. They described a situation in one school where, prior to taking part, a pupil’s behaviour in the local community was causing problems. Since returning from the residential, the pupil had not only stopped offending themselves, but had influenced their friends to make better choices too. The headteacher also cited this example.

Albeit to a lesser extent, there was also evidence among those interviewed that relationships at home had improved for some pupils as a result of the intervention (though, when asked to pick out what the ‘key differences’ were for them, this was not something they highlighted). In one case, both a pupil and their parent/carer felt that their relationships at home had improved. From the young person’s perspective, he felt he was spending more time with his parents, while the parent felt he had been more engaged with family time. In another case, while a pupil did not identify this as an area where the YPLA had had an impact (viewing their relationships at home as already good), their parent/carer did feel that they were opening up more:

“Certainly, in the family dynamic, we sit and we talk more than we did. Columba1400 has been the catalyst for this.” Parent of S3 pupil, Columba 1400 graduate.

More broadly, some pupils felt that their time away on the residential helped them appreciate their parents and relate to them more, which had encouraged them to help out at home. Columba 1400 staff provided further anecdotal evidence of this, describing how the YPLA had helped some pupils to realise the importance of their family relationships. This was linked to young peoples’ experiences of getting to know their teachers better during the residential (e.g. using their first names) and viewing them more ‘as people’.

“Because we were with the teachers and seeing that they are just people like us […] It just made me realise that [adults] were just like us, so I started helping out more in the house and stuff.” Pupil, S4, Columba 1400 graduate

Others noted that their relationships with siblings had improved as they could communicate better than before and this was linked to having more confidence to speak and share their opinions.

A greater self-awareness among young people was also reported. The process of identifying values and hearing about others’ values was considered to have helped them to develop better listening skills and to be more respectful and empathetic towards others.

"I'm a better person - kinder, more sensitive, nicer." Pupil, S3, Columba 1400 graduate

Improved health and wellbeing

Improved health and wellbeing was identified as an impact by staff and pupils alike. Some pupils noted that the experience of going away to Skye and being in nature had been beneficial to their mental health, and others described feeling generally happier. This was linked to positive memories of the trip, the ongoing support from peers and teachers, and feeling engaged and “upbeat” about the phase three project (which pupils were still involved in at the time of research).

Some pupils reported feeling less anxious because of taking part. They felt the opportunities to speak in public had made them less self-conscious and more able to manage their anxiety (particularly in social situations but also in terms of managing stressful events like exams). Staff also highlighted that the time away helped the young people to focus on relaxation, being calm and developing strategies to help manage problems.

“I used to feel anxious about everything, even coming to school sometimes, but now I know I can speak to teachers any time. I still have some anxiety but it’s nowhere near as bad. Columba 1400 helped because of talking to people one on one and speaking publicly. We were pushed out our comfort zone.” Pupil, S4, Columba 1400 graduate

Other pupils still felt quite anxious and did not see the YPLA making a difference in this way. However, they were not sure if that was something that it could have helped with anyway, as they saw it as a more “personal” thing. These pupils did not mention any expectations that the YPLA could have helped them with their anxiety (citing other motivations for taking part such as to make friends or try something new). In contrast, those who felt the intervention had helped them to feel less anxious tended to highlight this – particularly in relation to social situations – as a reason for applying.

There was also evidence of the experience helping young people develop strategies for managing their mental wellbeing. One pupil described feeling better in the mornings while in Skye as they were going to bed earlier and were not allowed their phones, and so had adopted these habits since returning home.

Increased academic confidence

Although identified as a key difference among teachers and Columba 1400 staff more so than among pupils, increased academic confidence was identified by several evaluation participants as something the YPLA had made a difference to. From the staff perspective, it was felt that the intervention had given pupils a more positive mindset towards their studies.

“Negativity is not there, because usually you get to fourth grade, I'm going to fail this, I'm going to fail this. But they have all got the kind of ‘can do’ [attitude].” Teacher, Columba 1400 school

Columba 1400 staff echoed this, viewing the YPLA’s impact in this area as providing tools for young people to persevere, to work through challenges using their values, and to build resilience.

From the pupil perspective, while academic confidence was part of it, the impact was broader as they felt the YPLA had given them more belief in themselves and in their abilities generally. This was linked to the values-based approach which encouraged pupils to identify values that matter most to them.

“Doing values about yourself, finding out who you are, talking about things like that just helped me know myself better and have more belief in my abilities. Not just qualifications, but I just think I have more belief in my ability and belief in myself.” Pupil, S4, Columba 1400 graduate

Increased workplace knowledge and skills & higher aspirations

The impact of the YPLA on young peoples’ knowledge of the workplace was not as widespread as for other capabilities, as pupils typically reported either that they already had an idea about the kinds of jobs they might like to do before the YPLA or that it was not something they had come to think about yet. However, there was a clear sense that the intervention had helped pupils feel more confident about doing well at a job in the future and had contributed to a greater sense of self-belief (particularly among girls). This was again linked to their experiences working together as a team and speaking in group settings as part of the YPLA.

“It is always about confidence really. I kind of thought, if I’m going to be a teacher, I’m going to have to have that belief in myself, and I really got to see that through Columba 1400.” Pupil, S4, Columba 1400 graduate

On employability more generally, Columba 1400 staff highlighted how the YPLA helped young people develop the “soft skills” and emotional intelligence needed for the workplace, as well as the qualifications. Developing the impact of the YPLA in terms of employability was discussed as a growth area for the scheme (see the building out and deepening section of this report below).

Similarly, in terms of higher aspirations, this did not resonate as strongly with pupils as other capabilities did, as they did not tend to feel that the YPLA had helped them “aim higher” in terms of what they would like to do after school. But there was a sense that participation had helped them believe in themselves to do well in the future whatever they were aiming for (while for some it was just not something that they had thought much about yet). From the perspective of Columba 1400 staff, the YPLA had helped young people take responsibility for, and ownership of, their futures.

“[It’s about] young people making the right choice to be the best version of themselves that they can be. So, what’s your pathway, no wrong pathway, what do you need to do it, and how do you take responsibility to get you where you want to be, because you’ve got the skills and the amazingness inside to do that.” Columba 1400 staff

From school and Columba 1400 staff, there was also evidence that the YPLA has helped to support young people with their personal development. School staff reported high levels of engagement in the phase three project among pupils, and although this was not universal, staff felt the intervention had supported and engaged young people in different ways, such as providing them with tools to be more resilient, to “work through complexities” and to make better decisions in terms of social networks.

“You can see that they are thinking, how do I use what I have learnt at Columba 1400 to get me through school and beyond.” Teacher, Columba 1400 school

Parents/carers echoed this, with one perceiving their child to have a more “aspirational” outlook after taking part in Columba 1400, particularly in terms of career options. Another parent felt their child was demonstrating more personal organisation skills and commitment to the phase three project than they had observed before in other settings.

Doing capabilities

Stronger young person voice in schools

There was widespread agreement among pupils, parents/carers, school staff and local authorities that the Columba 1400 YPLA had led to stronger young person voice in the schools. Phase three was the main driver of this. However, the extent to which this went beyond the intervention differed depending on how far through the YPLA the school was. For one school, where pupils had returned from Skye a few months before interviewing took place, the sense of increased involvement in school decisions among young people was more limited to their involvement in the phase three project only. For another school, where pupils had returned from Skye the previous year and were coming to the end of their phase three project, there was more evidence of young people getting involved in wider initiatives too (e.g. being class representatives or sitting on student councils) or feeling more confident to take leadership roles in the classroom:

“I feel like I’ve helped and contributed. I feel more confident doing speaking assignments than I would have before.” Pupil, S4, Columba 1400 graduate

There was also a broad sense among all participants that the YPLA had enabled young people to put their leadership skills into practice. In relation to the phase three project, participants highlighted their commitment to this, and there was a clear sense of duty and ownership over it. There was also a sense of empowerment, as young people felt they had more of a say and were being allowed to make decisions as a group to drive the project forward (rather than being told what to do by an adult).

“The impact I would say is that the young people feel more empowered and that they’ve got a voice and any ideas that they do have about their community are going to be listened to and, actually, they come to fruition.” Columba 1400 staff

There was broad recognition among evaluation participants that the Columba 1400 YPLA had played its part in engaging with and supporting young people to develop these leadership skills. But it was also recognised that there was a responsibility on the school to continue to nurture this engagement with values-based leadership beyond the YPLA, both among young people and staff.

“Columba 1400 has that ability to build. I’m still using the values-based leadership that I learned. Thinking longer-term is more for practitioners than Columba1400. You have to have headteachers trained, as well as young people, so you’re talking the same language.” Local authority representative

Improved relationships with peers, families and key adults

As well as improved relationships at home (see above), pupils and parents also identified improved relationships with peers as something that Columba1400 had made a difference to. Pupils who identified with this felt that they were getting on better with their friends because they were able to talk to them more. Pupils also reported that their social networks had expanded through making new friends or that they had prioritised different friendship groups that made them feel happier.

“I think the new friends that I have now are a lot better to me, so I am better as a result of that.” Pupil, S3, Columba 1400 graduate

There was evidence that the collaborative nature of the Columba 1400 YPLA, which encourages young people to engage with their teachers on more of a peer-to-peer basis, had a positive impact on pupils’ relationships with their teachers too. For some, this meant being able to relate to teachers more and see them as human beings while, for others, it meant finding teachers more approachable and easier to talk to than they had before. This was especially the case for the teachers they went to Skye with, but in some cases, pupils felt that they would feel comfortable opening up to other teachers that they trusted. School staff also echoed this view.

"The relationship we built with them up there, because you are cut off from everyone, and you have to get on, you need to make the effort, and they were just a fantastic group of kids. It is still going on since we came back." Teacher, Columba 1400 school

In other cases, pupils did not feel the YPLA had made a difference to their relationship with teachers or their ability to talk to them, which was generally because they felt they got on well with teachers already.

Engagement in wider activities

While some pupils reported that they were already quite active outside of school, others said that they had either signed up for more things, were socialising more outside of school, or felt more confident in the activities they were already engaged in as a result of the Columba 1400 YPLA. This was more evident among S4 pupils than S3 pupils, which could reflect the fact that S4 pupils had had more time to engage in wider activities since returning from the residential the previous year (compared to S3 pupils who had returned more recently). This was also mentioned more often by girls than boys.

“I started in a brass band which, I think, I was asked before, but I [had] said, no, because I didn't think I would be able to do it. It is, again, just about the confidence and stuff like that, it really helped.” Pupil, S4, Columba 1400 graduate

There was also some evidence of increased participation in employability activities, particularly among S4s, which pupils associated with having more confidence to ask questions about jobs, to take on more responsibilities as part of the activities they’re doing and to start getting prepared (e.g. by creating a CV). However, this impact was not widespread, with several participants saying that they were already doing this (perhaps because of their age and stage at school) or had not thought about it yet.

Increased engagement in learning

Increased engagement in learning did not resonate strongly with pupils as they typically reported that they had planned to stay on at school regardless and that their attendance record was already good. However, one pupil said that they had been considering leaving after S5 but had decided to stay on to further develop their leadership skills while another, who had not been sure, said that the YPLA made them realise the benefits of staying on.

“I was thinking of leaving at S5, not sure but think I might stay on longer. The whole point was about making us leaders so, if I left at S5, I wouldn’t have much to show. If I stay on, I can show to younger people.” Pupil, S3, Columba 1400 graduate

Staying on rates were highlighted as a key impact of the intervention by teachers and Columba 1400 staff. From the staff perspective, there were a number of pupils who might not have stayed on at school had they not taken part in the Columba 1400 YPLA. Staff also said that general engagement in learning had improved for pupils who were at risk of disengaging. It is possible that the difference between pupil and staff perspectives on staying on rates reflects the sample of pupils we interviewed.

“[One pupil] was potentially being removed from class because of bad behaviour and at risk of making bad decisions. They seem a lot more engaged now, which is really good. There could have been other factors but Columba 1400 was a big one. They have a more positive association with school now.” Teacher, Columba 1400 school

Reflecting on staying on rates, Columba 1400 staff suggested that this might not be the right path for all young people and that choosing an apprenticeship or college route instead would also be positive outcomes.

While improved attendance was also not widely identified as something that the Columba 1400 YPLA had made a difference to - as most participants reported good levels of attendance regardless - one pupil reported that reduced anxiety had enabled them to come to school more. Among some staff, this was noted as a key impact, but it was also noted that pupils had been chosen based on likelihood to engage, and so the cohort would have included those with better attendance generally.

Increased attainment / achievement

While increased attainment and achievement did not resonate strongly with pupils, some staff felt this was a key impact of the intervention. They acknowledged that it was difficult to quantify, but sensed that the YPLA had helped care experienced pupils and those in tough realities engage with school more. Some pupils were not clear on the link between Columba 1400 and attainment, but others felt that their experiences had helped them be more focused and determined in their school-work.

“One of my values was definitely determination, so that concentration, that drive is kind of applicable to my school-work.” Pupil, S3, Columba 1400 graduate

Subject / course choice

Views on the extent to which the YPLA made a difference to subject choice were mixed. Perhaps because they were focusing on the most recent choices they had made (in S2), this was not generally considered to be a relevant impact. A timing element was highlighted by one pupil who felt that, had the residential taken place before subjects were chosen, it might have influenced their decisions.

However, one pupil felt that the YPLA had given them confidence to speak to their school careers advisor about subject choices, which they would not have felt comfortable doing before C1400. There was also a sense of accountability and responsibility over subject choice and decisions for the future – which was attributed to the YPLA.

“[Pupil] had picked chemistry but isn’t enjoying it as much as he’d hoped, so now he’s thinking about conversations he can have to change subject but still get the subjects he needs for future jobs. He doesn’t defer to us as much now. He has his own ideas and opinions which he’s happy to voice as a result of the work they did with Columba 1400.” Parent of S3 pupil, Columba 1400 graduate

Columba 1400 staff highlighted this as an impact of the YPLA too, in that young people were being empowered through the intervention to make the right decisions for themselves, rather than being told what to do.

“Give them the scaffolds to help them use their own voice to decide what they want to do. It is about that ‘no wrong pathway’ and the conversations we have that help them and their confidence, their aspiration, their motivation, to make the right choices for them.” Columba 1400 staff

Impacts summary

The infographic below provides a visual summary of the ‘doing’ and ‘being’ capabilities in relation to the Columba 1400 YPLA. The statements are colour coded to illustrate those which resonated most widely or strongly (darker blue) and those that resonated less widely or strongly (lighter blue).

This summary should be read in conjunction with the section above, which provides further context and nuance to the ways in which participants felt the YPLA had made a difference to them.

Infographic text:

‘Being’ capabilities

Capabilities that resonated more strongly

• Increased social confidence: “I feel more confident in social situations”
• Increased social confidence: “I am more comfortable speaking to other people”
• Increased social confidence: “I feel better about myself”
• Increased academic confidence/higher aspirations: “I have more belief in my ability to do well in school/ gain qualifications”
• Improved health and wellbeing: “I feel less anxious”
• Improved health and wellbeing: “I feel happier”

Capabilities that resonated less strongly

• Increased workplace knowledge and skills: “I know about the kinds of jobs I might like”

Capabilities that were somewhere in between

• Enhanced social capital: “I have more people I can talk to about things”
• Increased workplace knowledge and skills: “I feel more confident about doing well at a job in future”
• Higher aspirations: “I am aiming higher in terms of what I would like to do after school”
• Enhanced social capital: “My relationships with the people I live with are better”

‘Doing’ capabilities

Capabilities that resonated more strongly

• Stronger young person voice in schools: “At school, I have been able to put into place some of the leadership skills I have learnt”
• Stronger young person voice in schools: “I feel more involved in school decisions”

Capabilities that resonated less strongly

• Increased engagement in learning: “My attendance at school has improved”
• Increased engagement in learning: “I plan to stay on longer at school than I did before”
• Increased attainment/ achievement: “I am doing better in my schoolwork”

Capabilities that were somewhere in between

• Improved relationships: “I am getting on better with my friends”
• Improved relationships: “I find it easier to talk to teachers / I am getting on better with my teachers”
• Increased engagement in learning: “I am doing more things that might help me get the kind of job I want in the future”
• Improved relationships: “My relationships with the people I live with are better”
• Subject/ course choice: “I am thinking more about the subjects I need to choose at school to help me get the kind of job I might want in the future”
• Engagement in wider activities: “I am doing more things that might help me get the kind of job I want in the future”
• Engagement in wider activities: “I am doing more activities out of school”

Broader impacts

To what extent has the YPLA embedded mentoring and leadership across the school system?

It was commonly mentioned by school staff that the YPLA had helped to embed mentoring and leadership practices across the school system.

One school’s participation in the YPLA had helped embed more of a mentoring culture through changes to registration groups, encouraging discussions about values and shifting the approach to discipline away from “punitive” towards more “restorative” approaches.

“The Columba 1400 cohort are helping with [registration changes, which are] encouraging more of a mentoring culture. This change is based on Columba 1400 and trying to get more leadership opportunities across the school.” Teacher, Columba 1400 school

Benefits for school staff involved in the YPLA were also identified as a ripple effect of the scheme and were considered to be the result of effective partnership working between Columba 1400 and the school. Columba 1400 staff reported that they often receive feedback from teachers about the (sometimes unexpected) impact that the YPLA has had on them professionally in terms of how they view themselves as leaders.

Young people felt that they had been able to put leadership skills into practice, at least to some extent, and felt that they had more of a voice in school than they did before (see Doing capabilities Stronger young person voice in schools above). Parents also described it as “refreshing” to see the young people more involved in the school and hoped this would continue in future.

“Moving forward, the group that have been involved, I’d like to think they feel more invested in the school and so will contribute to the school’s improvement in other areas in the future […] sometimes a small group can be a catalyst for change”. Parent of S3 pupil, Columba1400 graduate

Building-out and deepening

Columba 1400 staff highlighted examples of working with other organisations to offer more bespoke support within schools, particularly around employability. An example given was a partnership with Heriot Watt University which provided training to Columba 1400 staff on having helpful conversations with young people who are struggling to plan their next steps (e.g. course choice or staying on at school).

“[The training] was about supporting our staff with knowledge around effective pathways, how to have conversations with young people effectively to support them along the lines of leading themselves effectively, building on their confidence, removing that glass ceiling, there’s no wrong path, that journey.” Columba 1400 staff

Columba 1400 staff also mentioned the ongoing support of, and involvement with DYW’s ‘World of Work’ annual event and conversations with SDS about the leadership passport; a document that summarises a young person’s values and achievements to support conversations between young people and careers advisers or other key adults. This has been introduced at the end of phase three of the YPLA since April 2023.

Local authorities and national stakeholders felt there was huge potential for the Columba 1400 YPLA (and SMLP more broadly) to help provide links with other services or organisations outside of schools to support young people – but had not seen clear evidence of this happening yet.

Challenges relating to funding post-pandemic and the number of different agendas across the education system were identified as barriers to this happening. One rural local authority representative suggested that having a “connector” role (with a specific remit to bring work together) would help with this building-out element, especially across complex geographical areas.

“I’m sure it’s happening in pockets but [it’s] getting a national drive on it, I suppose.” Local authority

Local authorities noted some evidence of organisations trying to deepen support for young people beyond the Columba 1400 YPLA, highlighting examples of young people going through the leadership programme and then going on to be advocates or leaders in other things, such as the Youth Parliament. Again, the potential for harnessing the YPLA for a “lifetime development approach” was recognised, but it was felt that a national strategic approach would be required to achieve this.

Measuring longer-term impact

This phase of the evaluation was not able to consider longer-term impacts of Columba 1400 on the young people taking part^[24]. However, some evidence of the potential longevity of impacts was provided by school staff and local authority representatives, who described the ongoing impacts of having taken part in the HTLA themselves (one participant said that it continues to inform their work years later). Based on these personal reflections, coupled with the immediate impacts that had been observed from the YPLA by school staff, there was confidence among these respondents in the programme having similar long-term impacts on the young people taking part.

Columba 1400 staff also shared longer-term impact stories, highlighting sustainable partnerships with schools, and the feedback they receive about the systemic cultural shifts happening in schools as a result of taking part. Nevertheless, measuring the longer-term impact of the programme on young people was identified as something that Columba 1400 continues to work on.

“A lot of the impact, kind of is a bit subjective, you know, how do you absolutely nail down and measure things? And our challenge as well is, how do we measure the impact that we’ve had, given the interventions that are around a lot of these young people over their journey?” Columba 1400 staff

The research will revisit this in later stages. However, there will be limitations on robustly measuring the long-term impact of Columba 1400 YPLAs on young people within the scope and remit of this evaluation. (See Evaluation aims above).