Attainment Scotland Fund evaluation - Headteacher Survey: 2021 report

4. Impact

4.1. This section summarises views on the impact of ASF-supported approaches to closing the poverty-related attainment gap. This includes the factors that contribute to or limit success, and whether impacts are likely to be sustainable.

Progress in closing the poverty-related attainment gap

4.2. A large majority (87%) of survey respondents reported seeing an improvement in closing the poverty-related gap in attainment and/or health and wellbeing during 2020/21 as a result of ASF supported approaches. This included 17% that had seen ‘a lot’ of improvement to date. These findings were similar to previous surveys (90% had seen improvement in 2019/20, and 91% in 2018/19), and were broadly consistent across key respondent groups – although as discussed at Section 8, findings are not wholly consistent with published attainment data.

4.3. Although there has been no significant change in the proportion of schools who had seen improvement in closing the gap during 2020/21, there has been an increase in those expecting improvement in the future. The great majority of schools (94%) expected to see improvement in closing the gap over the next few years, a 6-point increase since the 2020 survey (although this followed a 10-point reduction between 2019 and 2020).

4.4. Survey responses also indicated some correlation between schools having already seen improvement, and expectations of further improvement; 78% of those who had seen ‘a lot’ of improvement to date expected to see ‘a lot’ more, compared with just 28% of those who had only seen ‘a little’ improvement to date.

4.5. Headteachers were asked to provide written comment in support of their response around perceived improvement in closing the poverty-related attainment gap in their school. The main points raised by respondents are summarised at Table 7.

4.6. For schools that have seen ‘a lot’ of improvement, comments most commonly related to schools’ capacity to implement approaches relevant to local needs – also identified as a key factor influencing impact for schools (see Table 9). Comments also referred to teaching and staffing resources (including schools which identified staffing input as a key driver of improvement, and where this informed ongoing approaches) and to the value of a focus on health and wellbeing, including further staffing input from external agencies including specific reference to mental health support.

4.7. For schools that have not seen any improvement in closing the poverty-related attainment gap, comments most commonly referred to the adverse impact of the COVID-19 pandemic. These schools referred to limited in-person contact with pupils as a result of school building closures and increased pupil absences. Some were of the view that this may have resulted in a worsening of the poverty-related attainment gap. Schools also referred to other factors, including pressure on staff time as a result of COVID-19 related absence, and to a view that the level of PEF allocation had limited scope to secure improvement in the attainment gap.

4.8. The great majority of schools (95%) felt that COVID-19 and school building closures had at least some impact on their progress in closing the poverty-related attainment gap, identical to 2020 survey findings. This included 54% who felt that COVID-19 and school building closures had a ‘significant impact’ on their progress. Those with lower PEF allocations, those in rural areas and PEF-only schools were most likely to feel that their progress had been significantly affected by COVID-19 and school building closures.

4.9. Headteachers were asked to provide written comment in support of their response around the impact of COVID-19 and school building closures on closing the poverty-related attainment gap in their school. The main points raised by respondents are summarised at Table 8.

Factors influencing impact

4.10. In addition to variation across respondent groups (such as funding stream, PEF allocation and urban/rural geography), survey analysis also considered correlation between perceived progress in closing the poverty-related attainment gap and other aspects of headteachers’ experiences. This analysis considered a range of factors including schools’ approach to closing the poverty-related attainment gap, headteachers’ understanding and awareness in shaping that approach, embedding equity, use of evidence, collaborative working, and views on the effectiveness of PEF.

4.11. This analysis indicates that a number of respondent groups are more likely to have seen progress in closing the poverty-related attainment gap (see Table 9). In particular, survey results indicate that key factors in closing the poverty-related attainment gap include tailoring use of ASF support to local needs, changes of culture or ethos (such as embedding the approach to equity), effective use of data and evidence (for example to inform approaches and measure impact), and engagement with families and communities.

4.12. The range of factors influencing impact is broadly is similar to that reported in previous surveys. Factors identified as having a significant impact on perceived progress in closing the gap are:

• use of PEF to meet local needs (a new question for 2021, not previously identified as significant);^[7]
• the approach to equity being embedded;
• use of data to inform approaches (identified as significant for the first time this year);
• engagement with families and communities (increasing in ranking from the 7^th to the 4^th most significant factor);
• ASF helping to develop staff data skills; and
• Increased collaborative working.

Sustainability

4.13. More than half (54%) of survey respondents expected that the ASF supported improvement they had seen to date will be sustainable. This represents a 21-point increase from the 2020 survey, although it should be noted that question wording was changed for the present survey.^[8] Survey findings suggest that views on the sustainability of improvement to date were broadly consistent across key respondent groups.

4.14. Views were more positive on the extent to which the focus on equity will be sustainable; 65% felt that this will be the case, a seven-point increase from the 2020 survey, and 17-point increase from 2019 survey. Survey findings also show some variation across key respondent groups; schools with a lower PEF allocation were less likely to feel that the focus on equity will be sustainable.

4.15. Schools had the opportunity to provide written comment in support of their view that progress to date and/or the focus on equity in their school will be sustainable. The main points raised by respondents are summarised at Table 10.

4.16. Those who felt that progress to date and/or the focus on equity will be sustainable most commonly suggested that staff skills and capacity developed with ASF support will be sustainable, even if initiatives themselves cannot continue in their current form with reduced staffing levels. The importance of staff skills and training was also highlighted specifically in relation to developing capacity to use evidence to inform approaches. Schools also referred to ensuring the sustainability of the focus on equity by embedding use of evidence in monitoring progress as part of the school culture.

4.17. Several schools referred to use of ASF support to develop whole school approaches to equity, for example embedding the approach in school policies, training materials and wider culture. Some specifically highlighted these whole school approaches as a means of ensuring future staff turnover does not undermine sustainability.

Staff have been trained in various interventions so this should be sustainable for as long as they are in this school. The additional adult support will be harder to sustain when no recovery staff. (PEF-only school in rural area)

We have not isolated teachers for training but instead have developed whole school approaches. Clear policies which detail our approach, and training materials have been made to ensure that any turnover in staff will not result in approaches being lost. Embedded processes for monitoring and tracking can be sustained as a culture of openness to support and challenge has been developed. (Challenge Authority school in urban area)

High staff turnover in a small school can make investment in staff training a non-sustainable use of PEF. To overcome this, the approach and training will be embedded in our school curriculum design/literacy policy, it will become part of the culture of the school and through parental involvement, families will be aware so it will be the responsibility of future staff members and the team around them to ensure the investment and improvement is sustainable. (PEF-only school in rural area)

4.18. Survey responses also considered the importance of having embedded approaches to achieving equity across the school, including through improved pedagogy. These respondents referred to sustainability of practice in relation to literacy and numeracy, but also the sustainability of improvements in school culture and ethos.

As approaches become embedded into school practice then they become more sustainable. Targeted interventions will still be needed as young people are identified that require individual support. (PEF-only school in urban area)

The most sustainable approach is to improve the quality of pedagogy and approaches to relationships and ethos within schools. If we can get this right and keep the focus on genuinely providing equity, then we may be able to sustain our progress. (Challenge Authority school in urban area)

4.19. Schools who did not expect progress to date and/or the focus on equity in their school to be sustainable also had the opportunity to provide written comment in support of this view. The main points raised by respondents are summarised at Table 11.

4.20. The view that progress and/or the focus on equity will not be sustainable was most commonly based on concerns that staffing levels cannot be sustained without access to funding. These schools noted that securing additional staff time and skills had been central to their use of ASF support, and to delivery of approaches to tackle the poverty-related attainment gap.

All our PEF has been spent on staffing, this has allowed us to deliver additional support for those pupils who are most in need. When this additionality is withdrawn and we lose this staffing there will be no way we can keep this commitment in place and it is likely we will need to move to a more generic universal support model. (Challenge Authority school in urban area)

4.21. The importance of staffing levels were also cited in relation to the sustainability of initiatives and approaches. Several schools suggested that existing initiatives could not be sustained without the staffing and other resources secured by ASF support.

The improvements have been made due to additional staffing. Withdrawal of this staffing would mean less opportunities for our learners to be able to access programmes. People make the difference. Very little in the way of pedagogical approaches can give the same results for our children in closing the gap. (PEF-only school in urban area)