Attainment Scotland Fund evaluation: analytical plan 3 2024-2025

Attainment Scotland Fund 2022 – 2026 Evaluation: Analytical Plan Year 3 (2024/25)

Analytical Plan Year 3 (2024/25)

Purpose of the document

This document outlines the third annual evaluation plan for the Attainment Scotland Fund (ASF) 2022-26 and covers the 2024-25 school year, the third year of the refreshed Scottish Attainment Challenge. This is a companion document to the ASF Evaluation Strategy 2022-26, published in November 2022. Previous analytical plans were published in February 2023^[1] and December 2023^[2].

Who is this document for?

The document is for stakeholders with an interest in education in Scotland and beyond; including Education Scotland Attainment Advisors, policy officials, headteachers, teachers and practitioners, parents/carers, public bodies, and third sector organisations. It provides information about evaluative activity within the current school year.

The ASF Evaluation

The ASF Evaluation aims to provide learning about the overall implementation of the refreshed ASF and the extent to which progress has been made towards meeting intended outcomes articulated in the Scottish Attainment Challenge Logic Model in support of the refreshed programme mission.

The Evaluation Strategy for the Attainment Scotland Fund 2022-2026^[3] sets out the broad terms of the Scottish Government's approach to evaluating the ASF during this parliamentary term. The Strategy outlines four key strands of evaluation focus across the multi-year evaluation:

• Process evaluation to consider implementation of the refreshed Attainment Scotland Fund, including Strategic Equity Funding, Pupil Equity Funding and Care Experienced Children and Young People Funding;
• A thematic strand of evaluation which will respond to emerging system priorities and consider 'what works, for whom and in what circumstances'.
• Reporting on the National Improvement Framework (NIF) measures of the poverty-related attainment gap;
• Evaluation of the impact of ASF, through a combination of qualitative and quantitative research and analysis.

This Analytical Plan

This third analytical plan continues the approach to evaluating progress in closing the poverty-related attainment gap through the Attainment Scotland Fund set out in the Evaluation Strategy. The Evaluation will focus on the impact of the refreshed Scottish Attainment Challenge through research with a wide set of stakeholders, including children and young people, practitioners, school staff, and community groups. The implementation element will continue with a reduced emphasis given the stage of the Programme.

Key priorities in the Analytical Plan include:

• Commissioning impact evaluation research;
• Working to ensure the voice of children and young people is included in the Evaluation;
• Continued quantitative reporting on core National Improvement Framework (NIF) Measures of the poverty-related attainment gap.

Progress of the ASF Evaluation in Year 2 (2023/24)

We have worked with the Evaluation Advisory Panel^[4] throughout the second year of the new Evaluation Strategy taking a flexible and responsive approach to building an evidence base.

The key research activities/outputs from Year 2 were:

• Completing the Impact Feasibility Study with a sub-group of the Evaluation Advisory Panel, and taking forward recommendations from the Impact Feasibility Study through a commissioned survey and case study research.
• Publishing a summary report on the thematic evaluation alongside three individual thematic evaluation reports.
• Publishing a summary of evidence around improvement in attainment and health and wellbeing, and the gap between pupils from the most and least deprived areas - Attainment Scotland Fund Evaluation: Reporting on National Improvement Framework Attainment and Health and Wellbeing Measures 2024.
• Publishing a summary of national evidence on the impact of poverty on educational outcomes – the Attainment Scotland Fund Evaluation: Reporting on Wider Evidence Sources, 2024.
• Publishing Attainment Scotland Fund Evaluation: Findings from In-house Research on Implementation and Impact 2024, incorporating evidence from the SAC Leads Survey 2024 and interviews with key stakeholders.
• Collaborating with Scottish Attainment Challenge National Programmes on engagement with children and young people on their views on readiness to learn.

Evaluation Priorities for 2024/25 School Year

Our priorities for 2024/25 are to focus on the commisioned school based survey and case study research; reporting on the core NIF measures; continuing aspects of the process evaluation; and an ongoing exploration of existing thematic areas.

Whilst the process and impact strands remain, there is less separation between them as survey instruments and qualitative work will combine elements of process/implementation and impact evaluation. Similarly, we do not propose separate thematic reports going forward. Instead, we will consider themes within other aspects of reporting including the impact commission.

Process: The evaluation will continue to have a focus on the implementation of the Attainment Scotland Fund and its funding arrangements, developing insights on issues such as funding, guidance, Stretch Aims, and planning. This will be progressed through the school based survey and the case study research.

Impact: We will progress the plan for impact assessment, in line with the Impact Feasibility Study recommendations, through a school based survey and case study research with 12 to 14 schools. The main fieldwork will be undertaken between February and June 2025. The school based survey report will be published in July 2025 and the case study research report will be published in August 2025.

NIF Attainment and Health and Wellbeing Measures: In order to maintain trend data and comparability, the core NIF measures continue to form the basis of quantitative reporting with a publication scheduled in June 2025 on the poverty-related attainment gap. We will continue to explore an expanded analysis of the NIF Attainment and Health and Wellbeing Measures through building on the analysis undertaken in 2024 and the specific areas of consideration.

Thematic: We will have a continued focus on readiness to learn and children and young people's voice in decision-making. Children and young people's voice and readiness to learn will be explored within the case study research.

Additionally, exploratory work with Young Scot (one of the SAC National Programmes) and the #YSEquity Panel will develop recommendations on potential approaches to gathering children and young people's perceptions of the impact of ASF by Spring 2025. Once completed, we will seek to develop this work to capture children and young people's views.

The proposed focus on learning and teaching outcomes will be explored with the Evaluation Advisory Panel and through the school based survey and case study research.

Reporting: We will publish an Interim Evaluation Report in Autumn 2025 to provide an overall assessment of progress.

Our specific areas of activity for 2024/25 are:

Overview of the Analytical Plan

In order to address each of the four key evaluation strands over the duration of the Evaluation Strategy 2022-26, a high level overview of the analytical plan was developed (See Appendix A). The Evaluation Strategy was developed to ensure that the detail of analytical activity during each year would be determined in line with the adaptive and flexible approach proposed.

A timeline of proposed activity for Year 3, 2024/25, and a forward look to Year 4 (2025/26) is set out in the table below.