Young people's experience of education and training

3. Methodology and approach

Introduction

3.1 This chapter details the main stages involved in the design and delivery of the research covering:

• Establishing the Insights Panel
• Research design
• Project branding and communications
• Delivery of the workshops
• Online task
• Analysis and synthesis of results
• Insights Lab
• Total participation and engagement.

Establishing the Insights Panel

3.2 An Insight Panel made up of 12 young people aged between 17 and 23 was established to co-design the research and provide input and guidance to the SQW / Young Scot team at key points throughout the study. This section provides an overview of the profile of the Insights Panel and their role in supporting the study.

The Insights Panel was made up of young people from across Scotland, with representation from both urban and rural areas.

3.3 Figure 3-1 shows the distribution of Insights Panel members across Scotland. It shows that around half were from in and around the central belt (mainly Edinburgh and Glasgow) and the remainder were spread across the North and South of the country ^[1] .

Figure 3-1: Location of Insights Panel members

Source: Produced by SQW 2017. Licence 100030994
Contains Royal Mail data © Royal Mail copyright and Database right [2016]
Contains National Statistics data © Crown copyright and database right [2016]

The Insights Panel was made up of young people from a range of different backgrounds and learner journeys.

3.4 It would not have been possible for the Insights Panel to be fully representative of all young people in the 15-24 age range. However, the intention had been to ensure broad representation from a range of different backgrounds and learner journeys. Figure 3-2 provides an overview of the profile of the panel in terms of age, gender, ethnicity and education / employment status, demonstrating the balance of gender, and the predominance of young people over the age of 18, all of whom would have made some transitions.

Figure 3-2: Demographics of the Insights Panel

Source: Insights Panel Equality and Diversity Monitoring Forms

Most Insights Panel members had previous experience of co-design projects.

3.5 The timescales for the study meant that it was not possible to take an open recruitment approach to the Insights Panel. Members were recruited, therefore, via Young Scots' existing networks and contacts. The majority had been involved in co-design projects in the past and therefore had an understanding of the process, which meant that they were able to contribute immediately.

3.6 Several members of the Insights Panel were active across a number of youth engagement fora and could therefore be considered very 'engaged'. However, most had also faced issues, challenges and barriers in their lives and learner journeys and brought insights into approaches that might / might not work for less engaged groups.

The Insights Panel provided input and guidance throughout the study.

• First meeting – the first meeting of the Insights Panel took place over a residential weekend in January 2017. The focus for the first session was on the development of the research tools to be used in the workshops.
• Basecamp – to enable ongoing communication and engagement throughout the project, an online platform was established using Basecamp. This was used to share the final research tools, update the Panel on workshop dates and locations and confirm arrangements for future meetings.
• Second meeting – the meeting in February 2017 focused on finalising the research tools for the workshops and creating an introductory video featuring Insights Panel members. This video was played at the start of each workshop.
• Delivery – Panel members were made aware of each workshop and were welcome to attend and assist with facilitation. Workshops in Dumfries and the Borders were organised and co-facilitated by Insight Panel members.
• Moderation of findings and agreement on key messages – the Insights Panel met again in April 2017 to review the data and materials gathered from the workshops and to discuss and agree on the themes emerging from these. They were given an introductory tutorial to qualitative data analysis, before being tasked with creating a coding framework for the next stage of analysis.

Research design

Quotas – a series of quotas relating to the profile of workshop participants were developed based on analysis of demographic information for the age group and the findings from the document review.

Research tools – the plan for the workshops, including the research tools, were co-designed with the Insight Panel. The three-hour workshop session plan that was used can be found in Annex C.

Figure 3-3: Worksheets for participants to show their learner journey

Project branding and communications

The project was given the title LevelUp. The logo and hashtag above were used in all communications relating to the project.

3.7 LevelUP was based on a gaming reference, with the suggestion being that you complete each level of your learner journey before progressing / unlocking access to the next. It also incorporated the concept of equality of opportunity ( i.e. by 'levelling up' the playing field) and the idea that you can 'reboot' or restart your learner journey at different times. The hashtag #LevelUpScot was used on all social media communications relating to the project. The hashtag had a lively Twitter feed, which could be viewed without the need for a Twitter account. Workshop participants contributed to this, with one writing about her experience in an online blog.

Delivery of the workshops

SQW, Young Scot and the Insight Panel's networks of contacts were used to engage stakeholders and recruit workshop participants.

3.8 An email communication was issued to partners and stakeholders providing an overview of the study and an invitation for the young people with whom they were working to participate in the workshops; these were followed up with phone calls. Levels of engagement were mixed, with a noticeable lack of engagement from universities but high levels of engagement from employability training programmes.

A total of 14 workshops were delivered across Scotland, engaging 145 young people aged 15-24.

3.9 The demographic profile of workshop participants is outlined in Annex D. An above average proportion were in the younger age groups, due to the challenges experienced in engaging university students and those in employment, who tend to be older than those on training programmes. A relatively high proportion of participants reported having a disability, with 12 per cent having some form of mental health condition. Representation from BME and care-experienced groups were in line with the population as a whole.

Online task

Young people were able to contribute to the study via an online task, however engagement with this was lower than anticipated.

3.10 An online task was set up to provide an alternative option for young people to participate in the study, particularly those who were unable to attend any of the workshops. It was designed as a single, open question with several prompts for young people to think about (see Figure 3-4).

Figure 3-4: LevelUP Online task

Share your learner journey! In doing this, you may want to think about:

1. What have been the key stages in your journey?

2. What big decisions have you had to make?

3. What support have you received to help plan and navigate your journey?

4. What have been the main influences on your journey, both positive and negative?

5. What (if anything) would have made your journey better?

3.11 The online task was posted on the Young Scot website and promoted to Young Scot Reward Card holders. It was also circulated to stakeholders working with young people in the target age range, including those who were unable to participate in the workshops. A total of 30 responses were received, from which the relevant responses were combined with the notes from the workshops and included within the analysis.

Analysis and synthesis of the results

A moderation meeting with the Insights Panel was held to discuss and agree on the emerging themes from the workshops.

3.12 For each workshop, a 'hot report' of headline messages was produced within three days of the event, followed by a fuller write-up of the main discussion points. These documents, alongside the raw materials produced by workshop participants, were reviewed by the Insights Panel with a view to identifying emerging themes. These were used as the basis of a coding framework for the analysis and interpretation of the results.

Insights Lab

3.13 A one-day event ( "Insights Lab") was held in central Edinburgh on 25 May 2017. The event was attended by 45 young people ^[2] and 11 policy officials from the Scottish Government. The aim of the event was to bring together policy officials working on the Learner Journey review with young people to explore the support available to them to make decisions about their future and to progress in their learner journey.

3.14 The young people participated in two workshop sessions with policy officials, during which they were invited to share their own learner journey experiences and to reflect on what was working well, what was not working or could be improved, and their ideas for change. In the final session, the ideas from the two workshop sessions were collated to provide overall suggestions for change from the day. These were then voted on by the young people to identify their top priorities for change.

Total participation and engagement

Over 200 young people participated in the research, contributing a total of 1,100 volunteer hours.

3.15 Research participants include Insight Panel members (12), workshop participants (145), those who completed the online task (30) and attendees at the Insights Lab (45). Taking account of those who participated in more than one activity, this resulted in a total of 210 unique research participants. The depth of engagement was high across all of the activities (with the exception of the online task) generating a combined total of 1,100 volunteer hours – an average of five hours per participant.

3.16 The study was qualitative in nature, aimed at gaining in-depth insights into the learner journey as it is experienced by young people in Scotland. As noted above, participants contributed an average of five hours each to the study. It would not have been possible to achieve this depth of engagement, whilst also attaining a sample that was fully representative of all young people in Scotland. However, the profile of participants (see Annex D and Annex E) shows that young people from a broad range of backgrounds were included within the study.

3.17 A recent academic study ^[3] found that 80% of the key themes from qualitative research are discoverable within three focus group sessions, and that 90% are discoverable within three to six focus groups. The fieldwork phase of the current study involved 14 three-hour workshops and a one-day event. We are therefore confident that the themes emerging from this represent an accurate representation of young people's experiences of the education and training system in Scotland.