Attainment Scotland Fund Evaluation - Headteacher Survey: 2020 report

5. Impact

5.1. This section summarises views on the impact of ASF supported approaches to closing the poverty-related attainment gap. This includes the factors that contribute to or limit success, and whether impacts are likely to be sustainable.

Progress in closing the poverty-related attainment gap

5.2. A large majority (90%) of schools reported seeing an improvement in closing the poverty-related gap in attainment and/or health and wellbeing as a result of ASF supported approaches. This included 20% that had seen ‘a lot’ of improvement to date. Survey data indicates that PEF-only schools were less likely to have seen ‘a lot’ of improvement to date.

5.3. A similar number of schools (88%) expected to see improvement in closing the poverty-related attainment gap over the next few years, although this represents a 10-point reduction since the 2019 survey. Survey responses also indicated some correlation between schools having already seen improvement, and expectations of further improvement; 67% of those who had seen ‘a lot’ of improvement to date expected to see ‘a lot’ more, compared with 11% of those who had only seen ‘a little’ improvement to date.

5.4. Headteachers were asked to provide written comment in support of their response around having seen improvement in closing the poverty-related attainment gap in their school. The main points raised by respondents are summarised below.

Table 8: Seen ‘a lot’ of improvement in closing the poverty-related attainment gap (n=89)

Theme	% of comments
Teaching and staffing resources	25%
Focus on health and wellbeing, including mental health and nurture	21%
Ability to implement approaches relevant to school	20%
Have made progress despite lockdown disruption to learning, use of remote learning etc to mitigate impact	12%
Higher quality learning and teaching	11%
Use of evidence/data and approach to evaluation	10%
Training and skills development	10%
Engagement with parents, carers and families	8%
Shared commitment to achieving equity in education	3%
Attainment Advisor, local authority support	2%

5.5. For schools that have seen ‘a lot’ of improvement, comments most commonly related to teaching and staffing resources. This included schools which had identified staffing input as a key driver of improvement, and where this informed ongoing approaches. Schools also referred to staffing input from other workers including counselling and other support, and this reflected a wider focus on health and wellbeing. Comments here included specific reference to mental health (such as use of nurture-based approaches) but also approaches focused on other aspects of pupil wellbeing such as nutrition, outdoor access and physical fitness.

5.6. Headteachers who had not seen any real improvement in closing the poverty-related attainment gap were also asked to provide written comment. The main points raised are summarised below.

Table 9: Have not seen improvement in closing poverty-related attainment gap (n=28)

Theme	% of comments
Impact of pandemic, lack of face-to-face contact, pressure on resources	46%
Headteacher new in post, too early to say for this year	32%
Pressure on staff time, and workload	18%
Limited PEF allocation	7%
Pressures on families, remote learning	4%

5.7. For schools that have not seen any improvement in closing the poverty-related attainment gap, comments most commonly referred to the adverse impact of the COVID-19 pandemic. These schools referred to school building closures resulting in extended periods without face-to-face contact with pupils. Some were of the view that this may have resulted in a worsening of the poverty-related attainment gap, particularly where schools had to suspend approaches based on close group working with targeted pupils. Some referred to new approaches having been developed in response to school building closures, but noted that these had taken time to fully implement, and suggested that a longer period will be required to assess impact.

5.8. The great majority of schools (95%) felt that COVID-19 and school building closures had at least some impact on their progress in closing the poverty-related attainment gap. This included 61% who felt that COVID-19 and school building closures had a ‘significant impact’ on their progress. Secondary schools and those with middle to higher PEF allocations were most likely to feel that their progress had been significantly affected by COVID-19 and school building closures.

5.9. Survey findings also indicate some correlation between perceived impact of COVID-19, and perceived improvement in closing the poverty-related attainment gap. For example, those who felt that COVID-19 has had a significant impact on their school were less likely to have seen ‘a lot’ of improvement to date.

Factors influencing impact

5.10. In addition to variation across respondent groups (such as funding stream, PEF allocation and urban/rural geography), survey analysis also considered correlation between perceived progress in closing the poverty-related attainment gap and other aspects of headteachers’ experiences. This considered a wide range of factors including schools’ approach to closing the poverty-related attainment gap, headteachers’ understanding and awareness in shaping that approach, embedding equity, use of evidence, collaborative working, and views on availability of support for PEF.

5.11. This analysis indicates that a number of respondent groups are more likely to have seen progress in closing the poverty-related attainment gap (see the table below). In particular, survey results indicate that key factors in closing the poverty-related attainment gap include changes of culture or ethos (such as embedding the approach to equity or improved collaborative working), better understanding of barriers faced by pupils and families, skills and knowledge in use of data and evidence, and engagement with families and communities. This is similar to the findings of the 2019 survey, which highlighted changes in culture and ethos and improved understanding of barriers faced by pupils.

Table 10: Respondent groups most likely to have seen progress in closing the poverty-related attainment gap

Respondent group

• Feel that approach to achieving equity has been embedded within school community
• Feel they understand the challenges and barriers faced by pupils and parents affected by poverty
• Feel ASF has helped to develop staff data and evidence skills
• Have seen an increase in collaborative working
• Feel their measuring of progress and impact of approaches is ‘very good’ or ‘good’
• Feel their use of data and evidence to measure impact is ‘very good’ or ‘good’
• Engagement with families and communities has been part of the school approach
• Approach to equity has developed from the previous school year

Sustainability of impact

5.12. Around a third (34%) of headteachers expected that the ASF supported improvement they had seen to date will be sustainable beyond the years of funding. This represents a seven-point reduction from the 2019 survey, although there has been a change in the question structure for the present survey which makes direct comparison difficult. Survey findings also show some variation across key respondent groups, with those in urban areas less likely to feel that their progress to date will be sustainable.

5.13. Views were more positive on the extent to which the focus on equity will be sustainable beyond the years of funding; 58% felt that this will be the case, a 17-point increase on the 2019 survey. Survey findings show some variation across key respondent groups, with primary schools less likely to feel that the focus on equity will be sustainable beyond funding.

5.14. Schools had the opportunity to provide written comment in support of their view that progress to date and/or the focus on equity in their school will be sustainable beyond funding. The main points raised by respondents are summarised below.

Table 11: Those who feel progress/focus will be sustainable (n=222)

Theme	% of comments
Staff training, skills development	33%
Developed capacity to use data/evidence to inform approaches	19%
Ongoing access to resources	18%
Embedded practice, pedagogy development	16%
Collaboration within school, with partners and parents	1%
Longer-term health and wellbeing impacts	14%

5.15. Those who felt that progress to date and/or the focus on equity will be sustainable most commonly referred to a view that staff skills and capacity developed through ASF-supported approaches will be sustainable beyond funding, even if the approaches themselves do not continue in their current form. The importance of staff skills and capacity was also referenced specifically in relation to use of evidence to inform the ongoing approach to achieving equity.

We have developed a culture of evidence-informed practice and a greater understanding of the barriers some of our young people face. Departments are far more adept at assessing data against SIMD, ASN, LAAC, etc. Development of staff knowledge and skills will ensure that the initiatives and projects will continue to have a positive impact on our young people for some time to come. (Challenge Authority school in urban area)

5.16. Written comments also reflected on the importance of the sustainability of culture and ethos, including through sharing of good practice. Responses referred to the sustainability of practice around literacy and numeracy, but also sustainability of culture and ethos including for example the focus on health/wellbeing and nurture.

Our health and wellbeing programme will be sustainable as we have a whole school policy and all teachers currently in our school have benefitted from training in this area. Literacy and numeracy improvements will be sustained as we can build on the developments made in these areas and use resources bought to support teaching and learning. (Challenge Authority school in urban area)

Developing a nurturing approach across the whole school, staff members who have had training in specific areas to share that good practice so that it can happen in all classes. (Challenge Authority school in urban area)

5.17. A number of schools also referred to evidence of the positive impact of the change in culture and ethos, for example in terms of maintaining the quality of relationships and levels of pupil engagement throughout school building closures.

All staff are committed to enhancing and developing relationships and nurturing practices, regardless of funding. Children and parents will continue to be supported by us when they need to be. Engaging with our parents and families has been challenging during COVID but survey results suggest that we are managing to maintain the very positive relationships and engagement levels before COVID. (Schools Programme school in urban area)

5.18. Schools who did not expect progress to date and/or the focus on equity in their school to be sustainable beyond funding also had the opportunity to provide written comment in support of this view. The main points raised by respondents are summarised below.

Table 12: Those who feel progress/focus will not be sustainable (n=61)

Theme	% of comments
Loss of staffing and skills	57%
Loss of initiatives and approaches/interventions	31%
Difficult to predict at present, potential future impact of COVID-19 pandemic and school building closures on attainment	8%
Reduction in wider budgets/resourcing	8%
Loss of external support and access to services	7%
Pupils’ health and wellbeing needs	3%

5.19. The view that progress and/or the focus on equity will not be sustainable was most commonly related to concerns regarding the loss of staff and skills without access to funding. These schools noted that additional staff time secured with ASF support has been crucial to delivery of approaches to tackle the poverty-related attainment gap.

Staffing may prove to be an issue for some initiatives where this has been fully funded by PEF – some interventions will not be able to be sustained. (Challenge Authority school in urban area)

5.20. It was suggested that staff input – and the initiatives or approaches being delivered – could not be maintained without funding. This included reference to additional teaching input, and other staffing and skills such as provision of support and assessment of impact.

Without the additional funding we will be unable to offer the bespoke approaches required to overcome the significant harm caused to young people by living in poverty. Building a child's self-confidence, belief in their worth and value, and overcoming the development delay caused by lack of nurture requires staff input. This will not be sustained if we are unable to procure resources through third sector partners, employ additional staffing and purchase resources to support pupils and their families. (Challenge Authority school in urban area)