Attainment Scotland Fund evaluation: fifth interim report - year 6

Executive Summary

Introduction

E1. The Scottish Attainment Challenge (SAC) was launched in February 2015 with the strategic aim of 'closing the poverty-related attainment gap between children and young people from the least and most disadvantaged communities'. The Attainment Scotland Fund (ASF) prioritises improvements in literacy, numeracy and health and wellbeing of those children adversely affected by poverty in Scotland's schools. Achieving excellence and equity in education are the key aims.

E2. A programme of evaluation of the ASF is ongoing. The evaluation aims to provide learning about the overall implementation of the ASF and to assess progress towards the following long-term outcomes:

1. Embedded and sustained practices related to addressing the impact of the poverty-related attainment gap.

2. All children and young people are achieving the expected or excellent educational outcomes, regardless of their background.

3. An education system which is aspirational, inclusive in practice and approaches for all including teachers, parents and carers, children and young people.

4. Closing the attainment gap between the most and least disadvantaged children and young people.

E3. This report focuses on the Year 6 (2020/21) evaluation of ASF across Pupil Equity Funding (PEF), Challenge Authority and Schools Programme Funding streams. The refreshed Scottish Attainment Challenge, launched on 30 March 2022, has a new funding structure which will be taken account of in a revised evaluation strategy for 2022 onwards.

E4. The report provides a narrative on progress from a range of evidence sources including:

• Administrative data on Challenge Authority, Schools Programme and PEF allocations and spend;
• National Improvement Framework (NIF) quantitative measures on attainment and wellbeing;
• Challenge Authority and Schools Programme progress reports;
• Surveys of headteachers and local authorities undertaken on an annual basis to explore perspectives on ASF.

Key findings

E5. It is apparent from the evidence considered in the evaluation that the impact of COVID-19 on the 2020/21 year was considerable. This included operational challenges such as staff recruitment and challenges associated with staff absences; negative impacts on staff wellbeing; pupil absences; and impacts on delivery of initiatives with some paused. COVID-19 and the resulting impact on education was the key factor influencing perceived progress in closing the poverty-related attainment gap.

E6. However, despite the considerable impact of COVID-19 across the 2020/21 academic year, the evidence presented for the Year 6 report continues to indicate positive progress towards the long term outcome of closing the poverty related attainment gap. This is, in large part, due to the response of schools and local authorities in adapting and responding to the challenges posed by COVID-19, and the role of ASF funding and flexibilities to support this response.

E7. This evaluation shows the continuation of key trends from previous years – such as the focus on collaboration, use of data and evidence, and the recognition of the role of Attainment Advisors – as well as developing trends such as the increasingly evident focus on engaging families and communities. The evidence shows that approaches are increasingly embedded, underpinned by understanding of the challenges facing pupils and their families, and supported by a positive culture and ethos in schools.

E8. The quantitative measures on attainment show a generally positive, but varied, picture of progress with the impact of COVID-19 affecting both the measures and what the measures report. Data from Achievement of Curriculum for Excellence Levels (ACEL) in primary schools showed an increasing level of pupils meeting expected levels in both literacy and numeracy until 2020/21, a period of school building closures and other disruptions, when these figures decreased and the gap between pupils from most and least deprived areas widened. Across school leaver attainment measures, the proportion of school leavers attaining one pass or more increased in 2020/21, while the gap between school leavers from the most and least deprived areas reduced. However, care must be taken when interpreting 2019/20 and 2020/21 school leaver attainment figures due to the changed SQA approach to certification as a result of the pandemic, which affects comparability.

E9. Views on progress from schools and local authorities remain positive. Despite the impact of the pandemic, almost 9 out of 10 headteachers reported seeing improvements in closing the poverty related attainment gap, while 94% indicated that they expect to see improvements in the next five years.

E10. This report outlines the impact of the COVID-19 pandemic on schools; the response of schools and local authorities to address these challenges; and the progress that is being made to close the poverty-related attainment gap. Despite the impact of the pandemic, progress towards long term outcomes continues, particularly in factors such as collaboration, use of data and evidence, and positive culture and ethos. This progress is reflected in views from schools and local authorities, and in positive, although varied, attainment data. The challenge remains a long-term endeavour, particularly given the impact of the pandemic, but progress in closing the attainment gap continues to be evidenced.

The following paragraphs outline the key findings from each chapter of the report:

Governance and support (Chapter 2)

E11.There were broadly positive views of governance and support provided by Scottish Government, Attainment Advisors and wider Education Scotland teams. Calls were made for the Scottish Government to provide more opportunities for sharing learning and good practice.

E12. The Attainment Advisor role was viewed as offering strong support and challenge, bringing a national perspective, support around use of data and evidence, sharing good practice, and working directly with target schools.

E13. A range of processes and supports for schools were identified from Scottish Government, Education Scotland and local authorities, with particular recognition of the importance of the Attainment Advisor role in bringing a national perspective to support design and implementation of PEF.

E14. Headteachers responding to the 2021 survey were generally positive on Pupil Equity Funding (PEF) processes and the role of PEFin addressing the poverty-related attainment gap.

Funding (Chapter 2)

E15. Funding available to local authorities and schools has increased considerably over the duration of ASF.

E16. Challenge Authority and Schools Programme spend as a proportion of allocation have continued to broadly increase year-on-year from 2017/18 to 2020/21. PEF spend as a proportion of allocation has also increased year-on-year.

E17. There was some evidence of change of use of resources at local authority level over the previous year to focus on outcomes for the most socio-economically disadvantaged children and young people.

E18. Flexibilities in the use of ASF introduced in May 2020 to enable local authorities to respond to COVID-19 had been utilised by local authorities and by schools, for example to support access to remote learning and to address additional support for learners most impacted by COVID-19. Nearly 3 in 4 schools (73%) responding to the Headteacher Survey 2021 had used the greater flexibility, most commonly to increase staffing capacity; to provide digital resources and connectivity; to support those affected by poverty; and, to support increased family and community engagement.

Approaches to Closing the Poverty-related Attainment Gap (Chapter 3)

E19. There is continued evidence of the development of approaches to address the poverty-related attainment gap at local authority and school level.

E20. Approaches to addressing the poverty-related attainment gap had been influenced both by the first period of COVID-19 related school building closures (March to June 2020), and the second period (January to March 2021). Headteachers' experiences of the COVID-19 pandemic in the previous school year had also contributed to their approach during 2020/21.

E21. Engaging families and communities emerged as a strong focusin local authority and school approaches to closing the poverty-related attainment gap.

E22. The importance of understanding the challenges and barriers faced by children and young people affected by poverty as key to developing appropriate approaches was highlighted.

E23. Approaches continued to include both targeted and universal approaches. There were high levels of awareness of the range of potential approaches available to schools in closing the poverty-related attainment gap, and confidence in selecting the most effective approaches, was also evidenced.

E24. New circumstances affecting families as a result of COVID-19 were apparent, with increasing mental health needs being most commonly identified. Approaches had been adapted in response to new circumstances including, for example, a greater focus on mental health and wellbeing (and to a lesser extent on physical health and wellbeing); more support for remote learning and digital connectivity; additional staffing skills and capacity; and additional support for parents and families including help with access to financial support, food, clothing etc. and initiatives to reduce the cost of the school day. Using community links and partnerships in order to help reach more families in need of support were also commonly highlighted.

Progress Towards Short- and Medium-term Outcomes (Chapter 4)

E25. Progress towards short and medium-term outcomes was considered, with factors supporting and hindering progress identified. COVID-19 was the predominant factor, and had influenced progress in a variety of ways. This included operational challenges such as staff recruitment and challenges associated with staff absences; negative impacts on staff wellbeing; pupil absences and impacts on delivery of initiatives with some paused. However, factors supporting progress were also identified and included: improved ethos, culture and understanding; headteacher empowerment/empowerment at the school level; data and evidence; effective target setting; collaboration; prioritisation given to closing the poverty-related attainment gap; support and challenge provided by local authority central teams/lead officers; Attainment Advisor support; and allocation of PEF over two years.

Engaging families and communities

E26. Key learning around engaging with families and communities in the context of COVID-19 is emerging and includes: the importance of family learning/family link workers and teams; effective links to wider partnerships; opportunities for online communication; need for meaningful engagement and challenging assumptions of what support families may need.

E27. There is emerging evidence of the impact of specific roles to support engagement with families, as well as a broader range of impacts identified by stakeholders including strengthened relationships with families and communities; increased awareness within schools of poverty and its impact on pupils and families; increased supports in place for families affected by poverty; a greater appreciation of the role schools can play; and, a greater focus on pupil attendance and engagement.

Collaboration

E28. There continues to be evidence of increased collaboration as a result of the Attainment Scotland Fund.

E29. Collaborations were evident both within local authorities and outwith local authorities, with collaborations key to supporting and enabling sharing of practice, building capacity and improvement. Changes to collaborations over the previous year were primarily related to the ongoing impact of COVID-19. There was evidence of some networks and collaborations being paused, of the impact of staff resource constraints, and the need to adapt and develop new collaborations to focus on vital support work within schools rather than wider collaborations. There was also evidence of ongoing developments to support collaborations in a number of contexts and between different levels of the education system, including strategic and more practice-oriented approaches.

E30. Benefits of collaboration continue to be identified, including supporting a focus on improving outcomes for children and young people affected by poverty through effective collaboration across services and sectors.

Use of data and evidence

E31. Indicators related to the use of data and evidence within ASF activity point to an increasingly 'data rich' environment, with a wide range of tools utilised across levels of the system and evidence of ongoing developments in the collection, collation and analysis of data, including a wider range of data and evidence being drawn on to support decision-making.

E32. Data and evidence relevant to the local context featured strongly in local authority decision-making over the previous year, from local authority-wide to the school level. At local authority level, the use of nationally available tools (e.g. Insight, BGE Toolkit) continued to be highlighted, alongside bespoke local authority tools developed to support data analysis to support decision-making.

E33. Headteachers maintained broadly positive perceptions (as measured through responses to the Headteacher Survey 2021) related to the use of data and evidence on measures such as:

• use of data and evidence to inform the development of their school's approach;
• skills in measuring the impact of their school's approaches;
• identifying the most appropriate measure(s) to assess impact;
• measuring the progress and impact of approaches;
• the extent that ASF support had helped develop staff skills and knowledge in using data and evidence.

E34. Whilst broadly positive, headteachers were less positive on some of the above measures than pre-2020 Survey findings, and there continued to be some differences in perceptions according to ASF funding stream and rurality.

E35. Data and evidence are being used to drive improvement, from strategic-decision-making through to classroom practitioner, but also to ensure support is targeted at where it is most needed, or to effectively prioritise interventions.

E36. COVID-19 has impacted on use of data and evidence. Some data collections (such as ACEL) have been impacted due to the pandemic, and there are requirements to utilise existing data and evidence, and to develop new sources of data and evidence, in order to respond appropriately to ongoing challenges presented by COVID-19.

E37. Collaboration was a driver for improvement in use of data and evidence, with examples of data-focused collaborative approaches e.g. at school cluster and neighbourhood grouping level. The role of Attainment Advisors working across a range of contexts including with local authority and individual schools, often in close collaboration with local authority leads, was a key feature.

Progress Towards Long-term Outcomes (Chapter 5)

E38. The analysis of quantitative data on attainment and wellbeing shows some positive signs in closing the attainment gap but it is a varied picture on progress depending on the measure under consideration. The impact on children and young people's learning as a consequence of the COVID-19 pandemic is clearly seen in the attainment statistics.

Reported evidence of impact

E39. Perceptions of progress towards closing the poverty-related attainment gap were gathered at both local authority and school level.

E40. At local authority level, the majority of local authorities responding to the 2021 Survey perceived they had seen improvement in their local authority towards closing the poverty-related attainment gap over the 2020/21 year as a result of ASF. There were clear indications of the impact of COVID-19. However, there was recognition amongst some local authority respondents that whilst it may not be possible to evidence progress through Curriculum for Excellence levels, and there is variability of progress across cohorts and schools, there is identified progress at school level on a number of relevant indicators indicating a positive trajectory of travel.

E41. At the school level, 87% of headteachers responding to the 2021 Headteacher Survey reported seeing an improvement in closing the poverty-related attainment gap as a result of ASF supported approaches in the previous year, a small decrease on 2020 (90%) and 2019 (91%). Headteachers who had seen 'a lot' of improvement in 2021 highlighted: school's capacity to implement approaches relevant to local needs; teaching and staffing resources; and value of a focus on health and wellbeing.

E42. Headteachers who had not seen improvement primarily viewed adverse impact of COVID-19 as a factor, with some perceptions that PEF allocations were not sufficient for improvement to follow.

• 94% of headteachers responding to the 2021 Survey expected to see improvements in closing the poverty-related attainment gap over the next few years, an increase of six percentage points on 2020. This suggests that, after a decline in future expectations between 2019 and 2020, headteachers now feel more positive in terms of future expectations.
• There were also positive reports of improvements specifically as a result of PEF funding: 94% of headteachers in receipt of Challenge Authority or Schools Programme funding in addition to PEF who responded to the 2021 Survey reported seeing an improvement specifically as a result of PEF.

Factors influencing progress

E43. Findings from the Headteacher Survey 2021 continue to suggest a number of key factors associated with positive perceptions of progress in closing the poverty-related attainment gap: change of culture or ethos such as embedding the approach to equity; tailoring use of ASF to local needs; effective use of data and evidence; and engagement with families and communities.

E44. Headteachers who were most likely to have seen progress were those who indicated PEF had been used effectively to meet local needs; those who felt the approach to achieving equity has been embedded within their school community were the second most likely group to have seen such progress.

COVID-19 as an influencing factor

E45. COVID-19 emerged as the key factor influencing perceptions of progress towards closing the poverty-related attainment gap. All local authorities responding to the 2021 survey were of the view COVID-19 had impacted on progress. The disproportionate impact of the pandemic on the most vulnerable pupils was reinforced in responses, alongside increasing numbers of families moving into poverty.

E46. There was a continued view amongst headteachers of the impact of COVID-19 on perceived progress in closing the poverty-related attainment gap, with nearly all (95%) of Headteacher Survey 2021 respondents viewing COVID-19 and school building closures as having had at least some impact on their progress, although a lower proportion in 2021 felt the impact had been significant (54%) than in 2020 (61%). Headteachers from schools with lower PEF allocations, those in rural schools and PEF-only schools were more likely to perceive the impact COVID-19 and school building closures was significant.

Embedded and sustained practices

E47. There was evidence of sustained and embedded practices as a result of the fund. Just over half (54%) of Headteacher Survey 2021 respondents expected ASF supported improvements would be sustainable, an increase on the 2020 Survey (34%). Additionally, almost two thirds (65%) expected the focus on equity would be sustainable, with headteachers responding to the 2021 survey more positively than in both the 2020 survey (58%) and the 2019 survey (42%).

E48. Headteachers who felt progress and/or focus on equity will be sustainable pointed to a range of factors in support of their view including: staff skills and capacity developed with ASF support, developing capacity to use evidence to inform approaches and embedding use of evidence in monitoring progress as part of their school culture, developing whole school approaches to equity and embedding approaches to achieving equity through improved pedagogy. Headteachers who were less positive about sustainability of progress and/or focus on equity related to concerns about staffing levels and potential loss of skills should staffing levels reduce.

E49. Local authority perspectives were broadly positive that the focus on closing the poverty-related attainment gap would be sustainable, with continuation of funding the primary factor associated with ensuring sustainability of focus on equity alongside a range of other factors including: ongoing professional learning; continued focus on understanding the challenges associated with poverty for children and their families; focus on data and evidence; creating a culture of embedding sustainable approaches; and, a focus on collaboration.