Attainment Scotland Fund 2022-2026 evaluation: analytical plan - year 2 2023 -2024
Second annual analytical plan for the Attainment Scotland Fund (ASF) evaluation 2022 to 2026 covering the academic year 2023 to 2024. This is a companion document to the ASF evaluation strategy 2022 to 2026 published in November 2022.
Analytical Plan Year 2 (2023/24)
Purpose of the document
This document outlines the second annual evaluation plan for the Attainment Scotland Fund (ASF) 2022-26 and covers 2023-24, the second year of the refreshed Scottish Attainment Challenge. This is a companion document to the ASF Evaluation Strategy 2022-26, published in November 2022. The first Analytical Plan was published in February 2023. [1]
The document is intended to be accessible to all those with an interest in education in Scotland and beyond; including teachers, support workers, parents/carers, public bodies, and third sector organisations.
The ASF Evaluation
The ASF evaluation aims to provide learning about the overall implementation of the refreshed ASF and the extent to which progress has been made towards meeting intended outcomes articulated in the Scottish Attainment Challenge Logic Model in support of the refreshed programme mission.
The Evaluation Strategy for the Attainment Scotland Fund 2022-2026 [2] sets out the broad terms of the Scottish Government's approach to evaluating the ASF during this parliamentary term. The Strategy outlines four key strands of evaluation focus across the multi-year evaluation:
- Process evaluation to consider implementation of the refreshed Attainment Scotland Fund, including Strategic Equity Fund, Pupil Equity Fund and Care Experienced Children and Young People Fund;
- A thematic strand of evaluation which will respond to emerging system priorities and consider 'what works, for whom and in what circumstances'.
- Reporting on the National Improvement Framework (NIF) measures;
- Evaluation of the impact of ASF, through a combination of qualitative and quantitative measures.
This Analytical Plan
This second annual plan, continues a refreshed approach to evaluating progress in closing the poverty related attainment gap through the Attainment Scotland Fund. Priorities in the Analytical Plan include:
- Continuing to evaluate the implementation of the refreshed Scottish Attainment Challenge through research with a wider set of stakeholders.
- Quantitative reporting on core NIF Measures.
- Implement an impact evaluation plan following publication of the Impact Evaluation Feasibility Study undertaken in Year 1.
- Publish three thematic evaluation reports on: readiness to learn; families and communities; children and young person voice. Plans for year two will include a focus on learning and teaching outcomes from the logic model.
Progress of the ASF Evaluation in Year 1 (2022/23)
We have worked with the Evaluation Advisory Panel[3] throughout the first year of the new Evaluation Strategy taking a flexible, adaptive, and responsive approach to building an evidence base. In the first year we have seen some adaptations and changes to the Plan which are noted in Table 1.
The key research activities/outputs from Year 1 were:
- A report on the implementation of the refreshed Scottish Attainment Challenge based on a survey of local authority attainment challenge leads.
- Publication of a report that summarises evidence around improvement in attainment and health and wellbeing, and the gap between pupils from the most and least deprived areas.
- Collaboration with National Programmes on engagement with Children and Young People on their views on readiness to learn.
- Collaboration with Education Scotland on the theme of Families and Communities.
- An impact evaluation feasibility study in collaboration with the Evaluation Advisory Panel.
Evaluation strand |
Planned activities |
Progress |
---|---|---|
Process |
|
|
Thematic |
Thematic evaluation and development activity including:
|
Working in collaboration with Education Scotland who have Families and Communities as the focus of their fourth Triannual Report[4]. Given that we have access to evidence collected from local authorities for Triannual Report 4, we made the decision to utlise this instead of commissioning case study research. A thematic summary report and three separate companion reports (for each thematic area) will be published in February 2024. This will have particular benefits in terms of highlighting the interdepencies between the thematic areas and articulating the cross cutting themes. |
National Improvement Framework (NIF) measures reporting |
|
The National Improvement Framework (NIF) measures reporting published June 2023 |
Impact |
|
Impact Feasibility Study completed |
Evaluation priorities for 2023/24 school year
Our priorities for 2023/24 are to continue to undertake a programme of analytical activity in support of the new Evaluation Strategy, with a focus on taking forward recommendations from the Impact Feasbility Study, reporting on the core NIF measures, continuing aspects of the process evaluation and ongoing exploration of existing and newly identified thematic areas.
Process: The evaluation will continue to focus on the implementation of the refreshed Scottish Attainment Challenge and new funding arrangements, developing insights on issues such as funding, guidance, Stretch Aims, planning and outcomes. In year 2, the focus of research activities will be schools and a wider set of stakeholders, which widens the focus from the local authority level in Year 1.
Impact: An Impact Evaluation Feasibility Study was undertaken in collaboration with the Evaluation Advisory Panel in Year 1. The study provides an in-depth consideration of the most promising potential approaches to assessing impact across Years 2 - 5. A summary of the process and options considered are contained at Appendix C. In year two, we will progress the development and initial implementation of a plan for impact assessment, in line with impact feasibility study recommendations.
NIF Attainment and Health and Wellbeing Measures: In order to maintain trend data and comparability, the core NIF measures continue to form the basis of quantitative reporting in Spring 2024 on the attainment gap. We will explore the feasbility of an expanded analysis of the NIF Attainment and Health and Wellbeing Measures.
Thematic: Analytical activity in Year 1 sought to explore 'what works, for whom and in what circumstances' on a number of priority themes including readiness to learn, families and communities, and children and young peoples engagement in decision-making/voice. We will publish a summary report and three separate companion reports on the thematic areas in February 2024. Whilst we continue to explore the thematic focus going forward with the Evaluation Advisory Panel, plans for year two will include developing a focus on learning and teaching outcomes from the logic model, and a continued focus on readiness to learn and children and children and young peoples voice in decision making. It is proposed that families and communities thematic focus will be complete in year two with the publication of the thematic report.
Our specific areas of activity are:
Evaluation strand |
Objective |
Activities |
---|---|---|
Process |
To provide learning on what worked well/what could be improved in the process of implementing ASF overall and at the funding stream level (Strategic Equity Fund, Pupil Equity Fund and Care Experienced Children and Young People Fund). |
|
Thematic |
To provide learning and increase the evidence base on 'what works and what could be improved, for whom, and in what circumstances' at the thematic level:
|
Thematic evaluation and development activity including:
|
National Improvement Framework (NIF) measures reporting |
|
|
Impact |
|
|
Section 2: Overview of the Analytical Plan
In order to address each of the four key evaluation strands over the duration of the Evaluation Strategy 2022-26, we have developed a high level overview of the analytical plan. This is outlined in Table 3 below. The detail of analytical activity during each year will be determined in line with the adaptive and flexible approach proposed in the Evaluation Strategy. The proposed activity for year 2, 2023/24 is set out in Appendix A.
Evaluation Strand |
Year 1 |
Year 2 |
Year 3 |
Year 4 |
Year 5 |
---|---|---|---|---|---|
Process |
Implementation study |
Implementation study |
Embedded processes |
- |
Summative processes and implementation |
Thematic |
2.1 Readiness to Learn – scoping/ literature review |
2.1 Readiness to Learn – Publication of Thematic Report |
Continuation /identification of emerging themes and priorities for thematic focus |
||
2.2 Families and Communities |
2.2 Families and Communities – Publication of thematic Report |
||||
2.3 Voice |
2.3 Voice Publication of Thematic Report |
Continuation/identification of emerging themes and priorities for thematic focus |
|||
2.4 Learning and Teaching – Explore with EAP |
Explore Learning and Teaching new thematic area |
||||
NIF Reporting |
Annual reporting |
Annual reporting |
Annual reporting |
Annual reporting |
Annual reporting |
Impact |
Impact Feasibility Study |
Development and initial implementation of plan for impact assessment, in line with impact feasibility study |
Contact
Email: joanna.shedden@gov.scot
There is a problem
Thanks for your feedback