Attainment Scotland Fund evaluation: third interim report - year 4

Executive Summary

Introduction

E1. The Scottish Attainment Challenge (SAC) was launched in February 2015 with the strategic aim of 'closing the poverty-related attainment gap between children and young people from the least and most disadvantaged communities'. The £750 million Attainment Scotland Fund (ASF) follows the duration of the SAC over the course of this Parliament, prioritising improvements in literacy, numeracy, health and wellbeing of those children adversely affected by poverty in Scotland's schools. Achieving excellence and equity in education are the key aims.

E2. A programme of evaluation of the ASF is ongoing. The evaluation aims to provide learning about the overall implementation of the ASF and to assess progress towards the following long-term outcomes:

1. Embedded and sustained practices related to addressing the impact of the poverty-related attainment gap.

2. All children and young people are achieving the expected or excellent educational outcomes, regardless of their background.

3. An education system which is aspirational, inclusive in practice and approaches for all including teachers, parents and carers, children and young people.

4. Closing the attainment gap between the most and least disadvantaged children and young people.

E3. This report focuses on the Year 4 (2018/19) evaluation of ASF across Pupil Equity Funding (PEF), Challenge Authority and Schools Programme Funding streams.

E4. The report provides a narrative on progress from a range of evidence sources including:

• Administrative data;
• National Improvement Framework (NIF) quantitative measures on attainment and wellbeing;
• Challenge Authority and Schools Programme progress reports;
• Surveys of headteachers and local authorities undertaken on an annual basis to explore perspectives on ASF.

Key findings

Approaches to closing the poverty-related attainment gap

E5. There has been a move away from a focus on individual interventions to approaches to close the poverty-related attainment gap. There is increasing evidence of a focus on broader approaches implemented around literacy, numeracy, and health and wellbeing to close the poverty-related attainment gap rather than at the intervention level. This includes local-authority wide developments such as whole school nurture approaches.

E6. Approaches to closing the poverty-related attainment gap are being refined based on improvement and use of evidence. Enhancements or adaptations in approaches to literacy, numeracy, health and wellbeing introduced to close the poverty-related attainment gap were based on the increasing use of a broad range of data, a focus on measuring impact, and a focus on building sustainability.

E7. Tailoring to local context and sufficient resources are key factors associated with success.The ability to implement approaches relevant to the local/school context and having sufficient teaching and staffing resources were the most commonly cited factors contributing to progress. Staff time/workload and reduction in other resources were seen as the main factors limiting progress.

Culture change and ethos

E8. There is evidence of systemic change in terms of culture and ethos towards development of a shared understanding of barriers faced by pupils and families, approaches to equity becoming more embedded within the school, a more collaborative system.

Long-term outcomes: Contribution to improvement and reduction of the poverty-related attainment gap

Perceptions of success

E9. Continuing positive messages in terms of perceptions of success:

• Nine out of ten (91%) of headteachers reported to have seen an improvement in closing the poverty-related attainment gap as a result of ASF supported approaches, an increase of 13 points since 2017;
• Nearly all headteachers (98%) expect to see improvement in closing the gap over the next five years.

E10. There are clear links between perceptions of success and positive culture change/ethos: headteachers most likely to report improvement in closing the poverty-related attainment gap were those who had seen a change in culture or ethos (such as embedding approaches to equity, broader strategies, more collaborative working) and where there was improved understanding of barriers faced by pupils and families.

E11. There are clear links between perceptions of success and use of evidence in developing and evaluating ASF supported approaches: headteachers most likely to have seen an improvement as a result of ASF supported approaches were those who feel confident using evidence to inform the development of their approach, and always use evidence to measure the impact of these approaches.

E12. There is variability across funding streams and urban/rural contexts including variations of perceptions from headteachers across the three funding streams in respect of reported progress in closing the poverty-related attainment gap, with Schools Programme respondents most likely to report an improvement, whereas PEF-only schools and particularly those with a lower PEF allocation were least likely to report an improvement.

Quantitative data on attainment and wellbeing

E13. On a number of NIF attainment measures, there is some progress in closing the attainment gap, although this is a varied picture depending on the measure under consideration. Overall, quantitative measures of the attainment gap do not yet show a consistent pattern of improvement.

E14. Reported impacts/perceptions of success currently present a more positive picture of progress than are emerging through the NIF attainment gap measures.

Funding and sustainability

E15. Funding continues to provide a focus. Whilst there are positive perceptions of the sustainability of focus on closing the attainment gap to some extent, it is recognised that funding is key in ensuring the resources to support improvements.

Conclusions and discussion

E16. The ASF seeks to support progress towards the strategic aim and associated long term outcomes of SAC, alongside the wider range of national programmes and initiatives which form the SAC. The ASF evaluation report has sought to bring together evidence to assess progress towards achieving these long term outcomes and the overall aims of the fund.

E17. The evidence has highlighted positive progress in a number of areas, including:

• development of embedded and sustained practices related to addressing the impact of the poverty-related attainment gap;
• positive developments across the education system towards aspirational, inclusive practice and approaches;
• positive reported evidence of perceptions of impact.

E18. However, the overall trends emerging from the Official Statistics and other data demonstrate the challenge which remains in closing the poverty-related attainment gap between the most and least deprived pupils. This reaffirms that the commitment to closing the poverty-related attainment gap is a long term one with a need to work in partnership to facilitate, broker and support action to maximise the progress towards closing the poverty-related attainment gap. The evidence gathered points to the importance of continued focus on Maximising Progress.

E19. Whilst it remains difficult to assess the reasons behind any observed improvement in attainment or closing the poverty-related attainment gap, and whether these changes have occurred as a direct result of the fund, the evaluation helps to support our understanding of factors related to these improvements, and the contribution the fund made to the realisation of these factors.

E20. A review of the ASF evaluation strategy is in progress. This will consider further analysis intended to strengthen future evaluation with a view to supporting the assessment of progress on closing the poverty-related attainment gap.

E21. Finally, the evaluation strategy review will consider how best to gather data on ASF Year 5 (2019/20) given the impact of Covid-19. It will also take account of the need to explore the impact of Covid-19 on both process and impact of the ASF in seeking to close the poverty-related attainment gap.