Attainment Scotland Fund evaluation: fourth interim report - year 5

Chapter 6 Discussion and conclusions

6.1 As the introductory section of the report outlined, the evaluation aims to provide learning about the overall implementation of the ASF and the extent to which the aims of ASF are being met in terms of closing the poverty-related attainment gap.

6.2 In terms of the available evidence to draw on for this report, it is important to note that response rates to the Headteacher Survey 2020 and Local Authority Survey 2020 were impacted by COVID-19. However, the current evaluation methodology continues to make the best use of existing data to inform our understanding of factors that support improvement in closing the attainment gap at the five year point of the Programme.

6.3 This concluding section draws together the key findings to assess the extent to which progress has been made towards the overall aims of ASF. It also seeks to provides some reflections on progress to date, on the evaluation, and to highlight links to related activity in this policy area including the publication of ‘Closing the poverty related attainment gap - A report on progress 2016-2021’.

Key findings

Overall

6.4 The findings overall suggest broadly continuing trends across Year 5 of the ASF as previously indicated in the Year 4 report e.g. focus on continued development of collaborations, increasing use and capacity of data and evidence, and governance arrangements in place nationally and locally. The role of Attainment Advisors continued to be highly valued with strong recognition of the importance of Attainment Advisors in facilitating links between national, local and school contexts.

6.5 There was evidence of ongoing developments in approaches to achieving equity at both school and local authority level, for example with the Headteacher Survey 2020 findings indicating that the majority of schools had developed their approach to achieving equity during the first part of the 2019/20 year from their approach in 2018/19.

6.6 The evidence suggests that approaches were adaptive and responsive, and were increasingly embedded. The focus on health and wellbeing and on engaging families and communities were increasingly evident.

6.7 Understanding of the challenges experienced by pupils affected by poverty continues to grow.

6.8 The analysis presented in Chapter 5, and previous analysis of Achievement of CfE Level data, indicates that on a number of measures, there is progress in closing the attainment gap, although this is a varied picture depending on the measure under consideration. For the majority of measures, attainment of those from the most deprived areas has increased, although in some cases not at the same rate as those in least deprived areas.

6.9 Change in attainment in the Challenge Authorities (combined) is also a mixed picture; the gap has widened for more measures than it has narrowed, however, largely, this is not due to performance worsening, rather performance has improved but not kept pace with performance of those from the least deprived areas.

6.10 With respect to the Challenge Authorities (combined), the proportion of S3 pupils achieving Third Level or better in literacy and the proportion of school leavers with one or more pass at SCQF Level 5 or better are measures where attainment for pupils from the most deprived areas has decreased and the gap has widened.

6.11 Positive perceptions of success continued to be articulated by headteachers, with the great majority (90%) of headteachers surveyed in 2020 perceiving improvements on closing the poverty-related attainment gap as a result of ASF. Whilst perceptions of success on this measure were very similar to 2019, views on future expected improvements were rather less positive than in 2019 with a 10 percentage point drop recorded in headteachers’ perceptions of improvements over the next five years in 2020 (from 98% in 2019 to 88% in 2020). This finding may be related in part to schools who have already made good progress not expecting to see further progress (due to the gap being virtually closed from their perspective). It may also reflect a change in expectations due to the impact of the COVID-19 pandemic and resulting school building closures.

6.12 It is worth considering the data on the attainment gap in the context of the headteacher perceptions of success to date. The attainment data presents a national picture of performance for the pupils within the scope of each measure’s definition e.g. Achievement of CfE Level data report on P1, P4 and P7, but not the remainder of the Primary stages. Conversely, the Headteacher Survey results present a holistic view of headteachers’ perceptions at their school level. It is important to reflect on the range of data sources in order to consider on progress on closing the attainment gap.

6.13 At the local level, there is evidence of impact, for example in terms of health and wellbeing. This included evidence on soft indicators of health and wellbeing, with evidence cited of, for example, improved readiness to learn, decreased disruption in the classroom, and improved social and emotional competence. This is an area where there is potential for the evaluation to explore further in future.

6.14 The evidence suggests increasing capacity, knowledge, understanding, and changing culture reflecting systemic change. For example, the 2020 Headteacher Survey identified the following factors in schools’ experiences with positive perceptions of progress in closing the poverty related attainment gap:

• Change of culture or ethos – evidence of embedded approach to equity; and increased collaborative working;
• Improved understanding of barriers faced by pupils and families affected by socio-demographic disadvantage;
• Improved skills and knowledge in use of data and evidence, and application of data and evidence; and
• Developing approaches in terms of approach to equity overall, and in terms of developing approaches to engaging with families and communities.

Impact of COVID-19

6.15 In addition to the above, the evaluation has sought to tell the ‘story’ of change and adaptation which occurred throughout the course of the 2019/20 year as a result of the period of school building closures from March to June 2020 due to the COVID-19 pandemic.

6.16 The evidence considered has suggested the wide range of ways in which COVID-19 impacted on the ASF in terms of processes and progress during this period. Whilst some interventions and approaches were paused, others were adapted and adjusted. There was a focus on limiting and mitigating the impact of COVID-19 on pupils and families affected by socio-economic disadvantage whilst the likelihood of increases in socio-economic disadvantage as a result of COVID-19 was recognised. Health and wellbeing approaches were prioritised during this time. Local authorities’ and schools’ use of flexibilities of ASF in accordance with national guidance produced in May 2020 also featured strongly. The rapid pace at which this work progressed was particularly apparent across evidence sources.

6.17 A number of particular features suggested by the evidence during this period include:

• a strong use of data gathering/analysis (e.g. local equity audit) to respond appropriately to the challenges associated with the period of school building closures and responding to the pandemic;
• evidence of strong partnership working in response to COVID-19 and closure of school buildings, and particularly the role of third sector partnerships;
• the importance and value of family/link worker roles using existing knowledge and relationships with direct work with families and communities to meet needs (food, physical resources, signposting etc) was highlighted.

6.18 Whilst there was uncertainty regarding the extent of the impact of COVID-19, there was concern that whilst school building closures have impacted on all pupils, this would be particularly so for pupils affected by socio-economically disadvantage. Concerns were also expressed of the potential long term nature of the impact of COVID-19 on pupils affected by socio-demographic disadvantage, and of impacts on pupils at particular transition points.

6.19 There was a focus both on responding to immediate concerns but remaining view on long term focus on closing the poverty-related attainment gap. Despite COVID-19, there was also evidence of a remaining focus on sustainability.

6.20 The impact of COVID-19 on progress towards closing the poverty-related attainment gap is clearly identified as a factor. In terms of perceptions of impact of COVID-19 on progress, this featured strongly in the evidence. Nearly all (95%) of respondents to Headteacher Survey 2020 perceived COVID-19 to have impacted on their progress (61% a significant impact), and the majority of local authorities responding to the Local Authority Survey 2020 viewed COVID-19 to have impacted on progress.

Reflections on progress to date

6.21 The strategic aim of the SAC is ‘closing the poverty-related attainment gap between children and young people from the least and most disadvantaged communities’ and at a wider policy level links to the National Performance Framework shared aim of improving outcomes for children and young people. As outlined in the introductory section, the SAC logic model articulates the following long-term outcomes:

Long Term Outcome 1: Embedded and sustained practices related to addressing the impact of the poverty-related attainment gap

Long Term Outcome 2: All children and young people are achieving the expected or excellent educational outcomes, regardless of their background

Long Term Outcome 3: An education system which is aspirational, inclusive in practice and approaches for all including teachers, parents and carers, children and young people

Long Term Outcome 4: Closing the attainment gap between the most and least disadvantaged young people

6.22 The evidence continues to highlight positive progress towards long-term outcomes in a number of areas, particularly in relation to Long Term Outcome 1 (embedded and sustained practices related to addressing the impact of the poverty-related attainment gap) and Long Term Outcome 3 (an education system which is aspirational, inclusive in practice and approaches for all including teachers, parents and carers, children and young people). Evidence on closing the attainment gap between the most and least disadvantaged young people suggests that there is progress on some measures, and there continues to be positive reported evidence of perceptions of impact.

6.33 However, there is an ongoing and long-term challenge which remains in closing the poverty-related attainment gap between the most and least deprived pupils. Whilst the evidence suggests COVID-19 will impact on progress towards closing the poverty-related attainment gap, progress towards the long-term outcomes at the education system level continues.

Evaluation and further research: reflections and future direction

6.34 The impact of COVID-19 will continue to be considered in the evaluation of ASF for the academic year 2020/21. This will include work to ensure the evaluation takes account of the Equity Audit findings published in January 2021.

6.35 The evaluation will further consider the key messages emerging from the ‘Closing the Poverty-related attainment gap – a report on progress 2016 – 2021’, published alongside this report, to strengthen the evaluation for the 2020/21 year.