Closing the poverty-related attainment gap: progress report 2016 to 2021

Appendix 1: Data sources

ASF Evaluation reports

The ASF evaluation began in 2015 and follows the duration of the SAC. An evaluation of the first two years of the ASF was published in March 2018, with the Year 3 (2017/18) evaluation published in June 2019. The Year 4 evaluation report was published in October 2020, while the Year 5 publication report will be published in March 2021. The evaluation aims to provide learning about the overall implementation of the ASF and the extent to which the aims of the ASF have been met.

The evaluation has the following objectives:

• Assess the impact of the overall fund in improving attainment and health and wellbeing and reducing the difference between pupils from the most and least deprived areas
• Assess the extent to which the further aims of the ASF have been met: promote capacity for data-based self-evaluation and improvement, and encourage collaboration between schools and local authorities;
• Provide learning and increase the Scottish evidence base of what does and does not work to improve attainment and health and wellbeing, especially of pupils from the most deprived areas;
• Provide learning on what did and did not work well in the process of implementing the ASF across participating Challenge Authorities and schools and which factors helped and hindered the ASF achieving its outcomes.

Evidence in relation to the ASF aims has been gathered from a range of sources and evaluated against a set of research questions. These include:

• Administrative data: information gathered as part of the routine organisation of the ASF provides data primarily on the funding that local authorities and schools received from the different funding streams.
• Challenge Authority and Schools Programme progress reports: Challenge Authorities and Schools Programme Progress Reports in March and September each year.
• Local Authority Survey: all 32 local authorities are invited to take part in an online survey. The first wave took place in 2016 with Challenge Authorities only and subsequent waves have been undertaken with all local authorities. To note that the Local Authority Survey 2020 was impacted by COVID-19, with a final response rate of 15 authorities.
• Headteacher Survey: headteachers from a representative sample of schools are asked for their experiences of ASF, including the approaches that their school uses, the impact achieved, and the sustainability of the impacts and approaches. To note that the Headteacher Survey 2020 was impacted by COVID-19, with a final response rate of 27%.

Education Scotland 5 year professional review

Between October and December 2020, Attainment Advisors (AAs) worked alongside SAC local authority project leads to analyse data and evidence from their respective local authorities looking at the impacts of the SAC. The purpose was to provide qualitative and quantitative answers to three high level questions regarding the impact of SAC within each of the 32 local authorities:

1. How has the implementation of the SAC impacted upon the culture and systems of local authorities to ensure those cultures and systems are equitable for children, young people and families affected by poverty?

2. How has the SAC positively impacted upon, or contributed to, educational outcomes for children and young people affected by poverty?

3. Which lessons have been learned and what are the future priorities for the SAC?

Each Attainment Advisor completed a professional report which provided further detail and evidence around each of these three high level questions. They utilised data from a variety of different sources to triangulate the analysis. Some of these sources were more granular and specific to the individual local authorities (such as surveys and other data collections produced by those authorities), while others were more national in scope, such as Achievement of Curriculum for Excellence Levels (ACEL) statistics or National Improvement Framework (NIF) data. Harmonised data collection reports focusing on some or all local authorities and compiled by Education Scotland were also used, these included inspection reports and Challenge Authority inspection reports. The reports compiled by AAs were then further analysed by Education Scotland and its partners to provide a national professional report detailing the impact of SAC over the past 5 years across Scotland.

Education Scotland Inspection of Challenge Authorities report

In September 2017, Education Scotland announced that HM Inspectors would establish a new programme of inspections. These were designed to evaluate and report on the progress being made in improving learning, raising attainment and closing the poverty-related attainment gap by the nine local authorities designated as Challenge Authorities within the SAC.

HM Inspectors carried out the nine inspection visits between December 2017 and December 2018, with the report summarising this information published by Education Scotland in June 2019. During each inspection they met with local authority staff, including groups of headteachers, teachers and practitioners from early learning and childcare settings, primary, secondary and special schools. They also met with learners, parents, and other stakeholders. In addition, inspection teams observed practice in establishments and reviewed relevant documentation, with a particular focus on attainment outcomes for children and young people.

The inspection methodology included the issuing of questionnaires to allow HM Inspectors to gather the views of various stakeholder groups across a range of domains. The report itself contains a brief overview of the findings along with examples of best practice and high level descriptive statistics for the survey questions asked.