Get into Summer 2021: qualitative evaluation

1. Introduction and methods

About the Get into Summer offer

The global COVID-19 pandemic, and consequent restrictions to supress the virus, have had an enormous impact across Scottish society. The impacts on families and children in particular have been documented in a series of Scottish Government evidence summaries,^[1] the latest of which (published in June 2021) highlighted persistent effects on anxiety and stress, especially (though not only) for older children and young people, and lower levels of mental wellbeing among parents with young children. It also reinforced earlier findings on the disproportionate impacts of the pandemic for particular groups of young people, including those with additional needs and disabilities, young carers, young people living in the most deprived areas of Scotland, minority ethnic young people, and LGBTQ+ young people. A strong theme of a need for more 'recovery' support was identified across the research and evidence reviewed.

This evidence on the negative impacts of the pandemic for children and young people combines with long-standing evidence that inequalities for children and young people from different socio-economic backgrounds are exacerbated during the school holidays. Children from low income families are less likely to have access to opportunities for structured activities during the holidays, less likely to exercise, and are more likely to experience social isolation.^[2] This 'holiday experience gap' has been found to have broader impacts on educational outcomes, further contributing to the well-documented attainment gap between children from high and low income families.^[3]

The Scottish Government wanted to take action to help address both the 'holiday experience gap' and the particular negative impacts of the pandemic for children and young people across Scotland. In March 2021, £20 million funding was announced to support an enhanced summer activity offer for children and young people over the school holidays.^[4] £15 million was made available to local authorities and £5 million to 18 national organisations (referred to in this report as the local authority and national partners respectively). Funding was primarily intended to cover targeted provision to support key target groups^[5], but also enabled some universal opportunities, open to all children and young people. Key priority groups for the targeted provision including:

• Children from low-income households
• Families with a disabled child or adult, or child with Additional Support Needs (ASN)
• Children from families who have been shielding during the pandemic
• Minority ethnic families
• Children and young people with care experience
• Young carers
• Larger families (with three or more children)
• Young mothers (aged under 25)
• Families with children under one
• Children in need of protection
• Children supported by a child's plan.

The Scottish Government provided guidance to local authority and national partners, setting out the rationale and broad parameters for using the funding, but with considerable flexibility as to how funding was used within these parameters.^[6] The Scottish Government marketing team also supported provision by creating an overarching national brand, 'Get into Summer', with supporting materials and advertising through the YoungScot website. This 'branding' was intended both to support local promotion through national promotion, and to reduce any potential stigma some children, young people or families might feel around participating by emphasising the Scotland-wide nature of the programme.

The primary aim of Get into Summer was to mitigate the negative impacts of the pandemic on children and young people and to boost their wellbeing by giving them opportunities to socialise, play and reconnect. In turn, it was hoped that this focus on wellbeing, play and reconnection would also contribute to supporting school-aged children with the transition back to the classroom, post-holidays. Overarching guiding principles for the programme included: co-creation with children, young people and families to ensure the offer met their needs; building on existing provision and knowledge of what works; and working in partnership with service providers to ensure a joined-up approach to development and delivery. It was envisaged that the scheme would fund a very wide range of activities, including sports, arts, cultural activities, and spending time in nature.

A logic model for the programme, informed by the Scottish Government's guidance and setting out the intended inputs, outputs and outcomes, was developed to help steer this evaluation by Ipsos MORI in conjunction with the Scottish Government and wider programme stakeholders.

Evaluation aims and questions

The Scottish Government commissioned Ipsos MORI Scotland to evaluate the Get into Summer programme. The overall aim of the evaluation was to assess whether the programme achieved its intended short-term outcomes and to identify what worked well and what could be improved, in order to inform the planning and delivery of any future holiday provision and support, both locally and nationally. More specifically, it aimed to:

• Understand the impact of participation for children, young people and families, including on their general wellbeing
• Understand what worked for children, young people and families, and what could be improved, particularly in those areas that children and young people identified as important to them (including areas discussed by interviewees in this research, and those identified as part of a YoungScot 'Jam event', in which the Scottish Government worked directly with young people to explore their priorities for Scotland's future as it recovers from the impacts of the pandemic^[7])
• Identify whether there were specific groups of children, young people and families who benefited more or less from the Get into Summer programme and its approach (and why), whether there were particular enablers or barriers affecting this, and whether specific groups might need further support beyond the summer to reconnect with peers or otherwise boost wellbeing
• Understand the range of support provided by national and local delivery providers, what worked well, and what could be improved from their point of view, including with regard to joint working and the approach to funding and governance of the programme
• Understand any other variations in what worked well in practice (for example, by geography, or type of activity) and any particular enablers or barriers affecting this, and
• Identify any unintended outcomes (positive or negative) of the programme.

Methods

The evaluation included three main elements: in-depth interviews with children, young people and families who took part in Get into Summer activities; in-depth interviews with local authorities, national organisations, and partners who were involved in planning and delivering activities; and analysis of local authority and national partners' monitoring reports submitted to the Scottish Government in September 2021.

Sampling and recruitment

Children, young people and families were recruited via a sub-sample of 11 Get into Summer activity partners, including two national partners and nine local authorities.^[8] These partners, and the specific activities focused on within each, were selected to try and ensure diversity in terms of activities offered, key target groups they were aimed at, and geography. Participants were recruited with the help of programme organisers, who were briefed over the phone and provided with written information by Ipsos MORI to share with Get into Summer attendees, inviting them to take part. Individual families then contacted the research team directly, who checked eligibility and arranged a suitable time for an interview.

Depending on the age and preferences of the child(ren) or young person who had participated in Get into Summer and their parent or carer, the research team either spoke to the child(ren) or young person on their own (6 interviews), the parent on their own (10 interviews), or conducted a joint interview with both (27 interviews). In total, 42 children and young people and 37 parents or carers were interviewed across 43 interviews, with interviews covering the experiences of 63 children and young people who had taken part in Get into Summer activities.^[9] Table 1 shows the profile of participating families, in terms of the age and gender of the child(ren), and whether they were from one of a number of key target groups for Get into Summer.

The research team also interviewed 30 partners involved in the planning or delivery of Get into Summer, recruited from across 10 local authorities and nine national partners.^[10] Interviews with local authorities included both those involved in planning activities, and partners involved in delivering activities. Most partners were interviewed individually, while two participated in paired depth interviews.

Table 1.1. Profile of children, young people and families covered by interviews ^[11]

Criteria		Number
Age of child or young person	0-7 years old	19
	8-11 years old	29
	12+ years old	14
Gender of child or young person	Boy	36
	Girl	27
Low-income family[12]		22
Child or young person with care experience		6
Disabled child / child with ASN or disabled adult in the household		20
Families with English as a Second Language (ESOL)[13]		5
Family member shielding		3
Large family (three or more children)[14]		9
Families with a child under two years-old[15]		6

Review of monitoring reports

Local authorities and national organisations were required to submit interim monitoring reports on delivery of Get into Summer to the Scottish Government in July 2021 and final reports by the end of September 2021. These reports asked respondents to record information on: delivery partners, activities, participants, outcomes, and reflections on what had worked well or could be improved.

The Scottish Government shared the final reports with Ipsos MORI for 31 (out of 32) local authorities. However, a number of these were received later than expected, after analysis and reporting had begun. In total, 29 out of 32 were systematically summarised for analysis, while the remaining two were reviewed for additional relevant information, particularly relating to reach. Reports from all 18 national partners were also received and systematically reviewed.

Data collection and analysis

Discussion guides were developed to ensure all relevant issues were covered in interview. In-depth interviews were conducted either by telephone or video call (depending on participants' preferences) between and August and October 2021. Families were given £30 to thank them for their time. All interviews were facilitated by members of the research team and were recorded for subsequent analysis.

Data from interviews and monitoring reports were summarised into separate thematic matrices^[16] developed by the research team, drawing on the research questions and logic model. These thematic matrices were then reviewed to identify the full range of views and experiences on each issue.

Scope and limitations

All research is subject to challenges and limitations. First, it is important to keep in mind that this evaluation took a largely qualitative approach. Given the available time and resources and the key aims of identifying what worked well and less well from the perspectives of children, families and partner organisations, this was felt to be the most appropriate design. The aim in qualitative research is not to achieve a sample that is statistically representative of the wider population, but to identify as much diversity of experience as possible. Estimates of prevalence based on qualitative data are therefore inappropriate – as such, this report avoids quantifying language, such as 'most' or 'a few' when discussing findings from qualitative interviews.

Overall, the range of programmes covered and recruitment quotas set enabled the evaluation to hear from families with a wide range of characteristics and circumstances, including those on low incomes, with disabilities or ASN, with English as a Second Language, and with care experience. However, it was not possible within the available time frame or resources to hear directly either from every one of the Scottish Government's 11 specific target groups, or from participants in all of the numerous types of activity funded through Get into Summer. This limits the ability of this evaluation to comment in detail on which specific activities might work best for different groups – although it does identify enablers and barriers across activities.

Another limitation is that although partners were asked about any groups that were more difficult to reach, the research did not include interviews with children, young people and families who did not take part in activities. While this does not negatively affect the quality of the data gathered, it should be kept in mind that there may be further barriers to participation that the evaluation could not identify, given non-participants were not included. It is also possible that data collected through partners interviews and included in monitoring reports may be skewed towards more positive aspects of delivery, although partners did also consistently include reflections on areas for improvement. The timing for reports was also, as far as possible, intended to reduce any recall bias around elements of programme delivery. Across the monitoring reports, however, there was significant variation in the level of detail provided. This posed particular issues for assessing reach, as discussed in chapter three.

Finally, the timeframe for this evaluation (interviews for which were completed by late October 2021) meant that it focused on the short-term outcomes from Get into Summer, as outlined in the logic model. While there is some consideration of the perceived likelihood of the programme having longer-term impacts, further research would be required to assess the extent to which this actually happens in practice.

Report structure and conventions

The remainder of this report is structured as follows:

Chapter 2: Overview of Get into Summer activities. This chapter describes the range of activities delivered through Get into Summer, and how this compared with the aims of the programme. It also considers the extent to which activities were completely new or enhanced existing provision, and evidence of co-creation of activities with children, young people and families.

Chapter 3: Participation and reach assesses evidence on who did and did not take part in Get into Summer activities, and enablers and barriers to participation among different target groups of children and young people.

Chapter 4: Perceived impacts for children and young people examines evidence for whether the short-term outcomes envisioned for Get into Summer (as set out in the logic model) were actually experienced by children and young people.

Chapter 5: Perceived impacts for parents and carers and the wider community considers short-term outcomes for parents and carers and for the wider communities within which the programme was delivered.

Chapter 6: Perceived impacts for partners. This chapter examines the perceived impacts of Get into Summer for partners, including impacts on partnership working.

Chapter 7: Key learning from Get into Summer. This chapter brings together both suggestions from interviewees and learning from across the previous chapters on what works in delivering inclusive, effective holiday provision, as well as findings on any persistent barriers to participation.

While any future provision might differ from Get into Summer in its precise aims or scope, it is hoped that these findings will nonetheless be helpful in informing design and delivery (nationally or locally) in terms of potential elements to replicate and barriers to take into account.

Case studies are included across the report to illustrate key points and diverse experiences. These are based on real experiences, but pseudonyms are used to protect anonymity. The research was carried out in accordance with the requirements of the international quality standard for Market Research, ISO 20252.