Attainment Scotland Fund - 2022 to 2026: evaluation strategy

Section 3

Aims of evaluation

The evaluation of the Attainment Scotland Fund aims to provide learning about the overall implementation of the refreshed Attainment Scotland Fund and the extent to which progress has been made towards meeting intended outcomes articulated in the Scottish Attainment Challenge Logic Model in support of the Scottish Attainment Challenge mission:

'To use education to improve outcomes for children and young people impacted by poverty with a focus on tackling the poverty-related attainment gap to deliver on the Scottish Government's vision of equity and excellence in education'.

The evaluation has the following objectives:

1. Assess the impact of the overall fund in improving outcomes in educational attainment and achievement and health and wellbeing and closing the attainment gap between the most and least socio-economically disadvantaged children and young people.

2. Building on progress evidenced in the ASF Evaluation 2015/16 – 2021/22, assess the extent of progress towards meeting the short, medium and long term outcomes articulated in the Scottish Attainment Challenge Logic Model, including: data and evidence, culture and ethos, embedded practices, and readiness to learn.

3. (a) Provide learning and increase the evidence base of what works and what could be improved for whom, and in what circumstances, to improve educational attainment and achievement, and health and wellbeing of pupils impacted by poverty within the Scottish policy context as well as contributing to the wider evidence base;

and

(b) To provide further learning on what works and what could be improved around the specific funding streams (Strategic Equity Fund, Pupil Equity Fund and Care Experienced Children and Young People Fund) and at the thematic level of intervention.

4. Provide learning on what worked well and what could be improved in the process of implementing the ASF overall, as well as a specific focus at the funding stream level in terms of the Strategic Equity Fund, Pupil Equity Fund and Care Experienced Children and Young People Fund.

5. Provide learning on which factors helped and hindered the fund progressing towards its outcomes, with a specific focus on the impact of COVID-19 and other contextual factors such as the cost of living crisis. Additionally, to provide learning on connections between ASF and other relevant policies at national, local authority and school level.

These objectives are underpinned by a set of evaluation questions covering a range of themes. Evaluation questions are directly related to the revised Logic Model, suggesting both continued and new areas of focus for the evaluation, as shown in Annex B.

The main evaluation questions are:

1. Governance

What worked well and what could be improved in the national and local governance and support with implementation of the refreshed Scottish Attainment Challenge?

2. Funding

What funding was allocated through the Scottish Attainment Challenge Refresh ASF to schools and local authorities, to what extent was it used within funds requirements and/or supplemented with other funding sources? What were stakeholders views on the implementation of the new funding structure introduced with the Scottish Attainment Challenge refresh?

3. Implementation

How did local authorities implement the Strategic Equity Fund alongside Pupil Equity Fund and Care Experienced Children and Young People Fund?

4. Approaches

How do the approaches for equity support pupils (and parents/carers) from the most socio-economically disadvantaged backgrounds?

5. Monitoring and evaluation

How are schools and local authorities monitoring, refining and evaluating their approaches to address the poverty-related attainment gap?

6. Collaboration

To what extent has the Fund resulted in established and/or strategic collaborations, and what has been the impact?

7. Data and evidence

To what extent have schools and authorities embedded use of data, analysis and evidence to drive improvements as part of the fund?

8. Engagement with and support for families and communities

To what extent has the fund embedded engagement with and support for families and communities?

9. Perceptions of local impact

To what extent did schools, local authorities and other stakeholders, including pupils, families and communities feel there had been progress towards achieving outcomes?

10. Unintended consequences

Did the changes to the funding structure with the refreshed SAC have any unintended consequences?

11. Approaches

To what extent was there awareness of and use of evidence based approaches and to what extent were approaches embedded in the system, refined and adapted based on effective interventions in the local context?

12. Engagement in decision-making (Voice)

To what extent were children and young people and their families and communities engaged in decision-making, what was the impact of this engagement, and was there evidence of engagement becoming embedded in the learner journey?

13. Readiness to learn

To what extent was there improvement in children and young people's readiness to learn through focusing on engagement, attendance, confidence and wellbeing?

14. Culture and ethos

To what extent was culture and ethos based on educational equity embedded in the education system that promotes high aspirations and recognises broader achievement for all children and young people and their families.

15. Closing the gap/educational outcomes

To what extent did the fund contribute to a closing of the attainment gap between the most and least socio-economically disadvantaged children and young people, in line with the refreshed Scottish Attainment Challenge Mission?

16. Achievement

To what extent did the fund contribute to an education system which encourages, reflects and values the breadth of achievements that contribute to improved outcomes for children and young people.

17. Embedded practices and culture

To what extent did the fund contribute to an embedded culture of equity at different levels of the education system?

18. Education system

To what extent did the fund contribute to an education system which actively addresses poverty, removing barriers through inclusive ethos, practice and approaches for children and young people, parents and carers and practitioners?