Attainment Scotland Fund evaluation: fifth interim report - year 6

Chapter 6 Discussion and Conclusions

6.1 The Attainment Scotland Fund (ASF) evaluation aims to provide learning about the overall implementation of the ASF and the extent to which the aims of ASF are being met in terms of closing the poverty-related attainment gap.

6.2 The current evaluation continues to make the best use of existing data to inform our understanding of factors that support improvement in closing the poverty-related attainment gap. It builds upon the five year impact report Closing the Poverty Related Attainment Gap: Progress Report 2016-2021 published in March 2021. A key difference from the Year 5 (2019/20) report is that in order to reflect the ongoing context of COVID-19 impacting across the academic year 2020/21, material has been gathered to reflect approaches across the year overall and with specific reference to the period of remote learning from January to March 2021. It is important to note that response rates to the Headteacher Survey 2021 were lower than usual, most likely impacted by COVID-19, and therefore the evidence gathered should be interpreted with some caution in terms of generalisability across the wider population.

6.3 This concluding section draws together key findings across the report to assess the extent to which there has been progress towards to overall aims of ASF across 2020/21.

Key findings

6.4 The overall findings presented in this report for Year 6 point to the continuation of many key trends in the evidence which have emerged over the previous years of the evaluation. This includes, for example, trends around collaborations, the use of data and evidence and the ongoing recognition of the important role of Attainment Advisors.

6.5 Where patterns of variability have previously emerged, this continued to be identified in evidence considered for Year 6. For example, headteachers from PEF-only schools, from schools with lower PEF allocation and from rural schools continued to report less positive perceptions of their experience of the ASF on some measures.

6.6 Approaches continued to adapt and respond, not least to the impact of COVID-19 which, as identified in the Year 5 report, remained the key factor influencing perceived progress in closing the poverty-related attainment gap. There was an ongoing focus on health and wellbeing, reflecting the importance of addressing increasingly prevalent mental health concerns impacting on children and young people as a result of COVID-19.

6.7 There was an increasingly evident focus on engaging families and communities with key learning and emerging outcomes identified.

6.8 The evidence shows that approaches are increasingly embedded, underpinned by understanding of the challenges facing pupils and their families, and supported by a positive culture and ethos in schools.

6.9 It is apparent from the evidence considered in the evaluation that the impact of COVID-19 on the 2020/21 year was considerable. However, despite the impact, the evidence continues to indicate positive progress towards the outcomes the SAC. This is, in large part, due to the response of schools and local authorities in adapting and responding to the challenges posed by COVID-19, and the role of ASF funding and flexibilities to support this response.

6.10 The analysis presented in Chapter 5, and previous analysis of quantitative data on attainment, indicates that on a number of measures, there is positive signs on closing the attainment gap, but the impact on children and young people's learning as a consequence of the COVID-19 pandemic is clear, although this is a varied picture on progress depending on the measure under consideration. Availability and comparability of data must be borne in mind, particularly with availability of ACEL data and the changed SQA approach to certification as a result of the pandemic, which affects comparability for school leaver attainment.

6.11 This is illustrated by data from Achievement of Curriculum for Excellence Levels (ACEL) in primary schools, where the proportion of pupils achieving expected levels in both literacy and numeracy increased steadily until 2020/21, the period of school building closures, when these figures decreased and the gap between pupils from most and least deprived areas widened.

6.12 Across SCQF NIF measures, the proportion of school leaver attaining one pass or more increased in 2020/21, while the gap between school leavers from the most and least deprived areas reduced.

6.13 Whilst measures of participation and positive destinations are consistently positive, with the proportion of 16-19 year olds participating in education, training or employment (the 'Participation Measure'), increasing year on year, with the exception of a slight drop in 2019, and reaching its highest ever levels in 2021. The participation gap between those who lived in the most deprived areas and those in the least deprived areas continues to show narrowing year on year, with the participation rate in the Challenge Authorities showing all nine authorities recording an increase between 2016 and 2021.

6.14 Whilst 87% of headteachers surveyed in 2021 perceived improvements in closing the poverty-related attainment gap as a result of ASF, this was slightly lower than in 2020 (90%) and 2019 (91%). Views on future expected improvements were rather more positive than in 2020, with 94% of headteachers indicating that they expected to see improvements over the next five years. This compared with 88% in 2020, although was still lower than the 98% who expected to see such future improvements identified in 2019.

6.15 The majority of local authorities who responded to the 2021 Local Authority Survey indicated positive perceptions of progress in the previous year as a result of ASF support. In addition, the Headteacher Survey 2021 impact of PEF was included in the Headteacher Survey 2021, which reflected highly positive perceptions of the specific impact of PEF amongst headteacher respondents who were also in receipt of Challenge Authority or Schools Programme funding.

6.16 There are ongoing differences between attainment data and perceptions of progress, with perceptions tending to be more positive.

6.17 Findings from the Headteacher Survey 2021 continue to suggest a number of factors associated with schools' experiences and positive perceptions of progress in closing the poverty-related attainment gap. This includes:

• Change of culture or ethos such as embedding the approach to equity;
• Tailoring use of ASF to local needs;
• Effective use of data and evidence;
• Engagement with families and communities.

6.18 The impact of COVID-19 continues to be evidenced across all aspects of the evaluation. As in Year 5, COVID-19 is considered as a theme throughout the report reflecting the inclusion of COVID-19 sub-evaluation questions underneath each main evaluation question (as indicated in Chapter 1). This has led to a fairly in-depth consideration of the impact of COVID-19 overall and specifically in terms of, for example, the impact of COVID-19 on use of data and evidence, and on collaboration in terms of the ASF.

Reflections on progress to date

6.19 The strategic aim of the Scottish Attainment Challenge is 'closing the poverty-related attainment gap between children and young people from the least and most disadvantaged communities'. This links to the National Performance Framework shared aim of improving outcomes for children and young people.

The Scottish Attainment Challenge logic model articulates four long-term outcomes:

Long Term Outcome 1: Embedded and sustained practices related to addressing the impact of the poverty-related attainment gap

Long Term Outcome 2: All children and young people are achieving the expected or excellent educational outcomes, regardless of their background

Long Term Outcome 3: An education system which is aspirational, inclusive in practice and approaches for all including teachers, parents and carers, children and young people

Long Term Outcome 4: Closing the attainment gap between the most and least disadvantaged young people

6.20 As identified in the Year 5 report, despite the considerable impact of COVID-19 across the 2020/21 academic year, the evidence presented for the Year 6 report continues to indicate positive progress towards the above long term outcomes particularly in terms of:

• Long Term Outcome 1 (embedded and sustained practices related to addressing the impact of the poverty-related attainment gap); and,
• Long Term Outcome 3 (an education system which is aspirational, inclusive in practice and approaches for all including teachers, parents and carers, children and young people).

6.21 The assessment provided in the Year 5 report pointed to the 'ongoing and long-term challenge which remains' in closing the poverty-related attainment gap between the most and least socio-economically disadvantaged pupils, suggesting that 'whilst the evidence suggests COVID-19 will impact on progress towards closing the poverty-related attainment gap, progress towards the long-term outcomes as the education system level continues'. The evidence considered for Year 6 points to a very similar overall assessment regarding the ongoing challenge which remains and the long-term nature of the challenge. It is apparent from the evidence considered that the impact of COVID-19 on the 2020/21 year was considerable. However, the ongoing progress in terms of understanding of the challenges facing pupils affected by poverty, embedding approaches to equity, and adapting approaches to respond to emerging evidence are all indicative of the extent to which there has been progress on a number of important aspects to date.

Evaluation and further research: reflections and future direction

6.22 While the initial impact of COVID-19 and school closures may be behind us, the full impact and effects of the pandemic on pupils – in terms of attainment, health and wellbeing, and many other fundamental areas, will continue to emerge and be felt for some time. Equally, the responses to this impact will continue to develop and evolve to meet the needs of pupils.

6.23 Evidence from Scotland and internationally shows that pupils from more disadvantaged backgrounds are likely to be disproportionally impacted by the pandemic in their learning and wellbeing. Work to close the poverty-related attainment gap will need to address this impact, while continuing to make progress across the programme activities.

6.24 As a result of the above, there is an ongoing need to consider the impact of COVID-19 in the ASF Evaluation, both in terms of its continuing impact as well as the approaches that are developed to mitigate the effects. The impact of COVID-19, and education recovery policies, will be features of the evaluation in years to come.

6.25 On 30 March 2022, the refreshed Scottish Attainment Challenge mission was launched, including changes to funding, a Framework for Accelerating Progress (including Stretch Aims), and an updated SAC Programme Logic Model. The refresh of the SAC Programme was underpinned by findings from the ASF Evaluation thus far.

6.26 The Refresh of the SAC Programme will necessitate a refocused ASF Evaluation Strategy, to ensure that progress in implementing the changes are evaluated, that we understand what is working and what is different in the programme, and ensure that we monitor progress towards the refreshed mission. However, we will continue to link between the existing evaluation evidence and any new approach to evaluation, ensuring continuity of learning.