Attainment Scotland Fund evaluation - Headteacher Survey: 2021 report

1. Introduction

1.1. This report presents findings from the 2021 survey of headteachers of schools in receipt of support from the Attainment Scotland Fund (ASF). The survey was commissioned by Scottish Government’s Learning Analysis Unit to inform the wider evaluation of ASF.

1.2. This section provides an overview of the background to the survey, and summarises the methodology and fieldwork. Further detail on the survey methodology – together with full tabular results and a copy of the survey questionnaire - is provided in the associated Technical Report at https://www.gov.scot/isbn/9781804355510.

Background

1.3. Launched in 2015, the Scottish Attainment Challenge’s (SAC) mission is to use education to improve outcomes for children and young people impacted by poverty. This includes a focus on closing the poverty-related attainment gap between children and young people from the least and most disadvantaged communities. The SAC has developed and expanded since 2015, and currently incorporates the following main strands:

• Pupil Equity Funding (PEF) provides funds directly to schools for headteachers to use at their discretion on initiatives that they consider will help close the poverty related attainment gap. Over 97% of schools in Scotland have been allocated funding, based on the estimated numbers of pupils in Primary 1 to S3 registered for free school meals.
• The Challenge Authority (CA) and Schools Programmes(SP) funds provide additional resource to nine local authorities, and 73 schools outwith those local authorities with the highest levels of deprivation. Each Challenge Authority/Schools Programme school receives funding and support to deliver improvement plans focused on literacy, numeracy and health and wellbeing to tackle the poverty related attainment gap.

1.4. Evidence collated over the first 5 years of the SAC provides an opportunity to refresh the programme, and address the challenges identified through feedback and engagement with stakeholders. Supported by £1 billion over the parliamentary term, from 2022/23 the SAC will be refreshed with a focus on enabling authorities and schools to support education recovery, reduce variation and accelerate progress in tackling the poverty-related attainment gap. This will include:

• a broader recognition of children and young people’s achievements and attainment;
• continued empowerment of school leaders through PEF;
• a clearer and funded strategic role for all local authorities;
• funding for PEF and local authorities confirmed over 4 years to enable long term planning;
• continued support for care experienced children and young people; and
• a clear framework to support recovery and accelerate progress, led by Education Scotland but with clear responsibilities for all parts of the education system to ensure a clearer line of sight right through the system on the impact of local approaches.

1.5. The current SAC programme is evaluated through the Attainment Scotland Fund (ASF) evaluation. As the ASF evaluation has evolved, the focus of the evidence being collected through the Headteacher Survey has moved towards a greater focus on progress and impact of approaches being delivered. This evidence is helping to inform the evaluation, which in turn has shaped the SAC refresh.^[1]

Study objectives and methodology

1.6. This is the sixth survey of headteachers of schools in receipt of ASF support. Surveys in 2016 and 2017 included headteachers across the Challenge Authorities and Schools Programmes, with PEF-only schools included for the first time in the 2018 survey. The survey sample was reduced for the 2020 survey (to include all CA and SP schools, and 50% of PEF-only schools) in recognition of the pressures on schools as a result of school building closures in force at the time. The present survey reinstated a 100% sample approach, with the survey issued to all schools in receipt of Challenge Authority, Schools Programme and/or Pupil Equity Funding. As a result, the survey sample for the present survey was around 50% larger than the 2020 survey sample.

1.7. The survey sought headteacher views and experiences relating to the 2020/21 academic year. Survey content was adapted from that used in the 2020 survey. Key indicators were maintained where survey findings are used to track trends over time, alongside changes to some survey questions to reflect changes in the context to the survey, in terms of the level of government restrictions resulting from COVID-19 in force during the 2020/21 school year. Survey design kept a tight focus on key questions which could inform the ongoing ASF evaluation, to limit survey completion time and the associated burden on schools.

1.8. The profile of schools across the three ASF streams and urban/rural geography^[2] is summarised below.

1.9. Consistent with previous surveys, survey invites were issued directly to schools, supported by promotion via Education Scotland and local authorities. The survey was issued in October 2021 and the fieldwork period ran to mid-November 2021. The survey response and approach to survey weighting is summarised over the following pages.

Survey response

1.10. A total of 597 responses were received by survey close, equivalent to an overall response rate of 25%. The level of survey response is similar to the 27% achieved by the 2020 survey, but remains substantially lower than response to the 2019 survey (47%). This is likely to reflect in part the continuing pressures on schools associated with the COVID-19 pandemic; the present survey was conducted in October and November 2021 with high COVID-19 case rates across Scotland.

1.11. In this context, we very much appreciate those schools able to participate. Moreover, the increased sample size used for the present survey means that the volume of responses is nearly 50% higher than the 2020 survey.

1.12. Table 2 summarises the profile of survey respondents, and compares this with all schools in receipt of ASF support. In terms of the profile of respondents, the largest groups are PEF-only schools, primary schools, and schools in urban areas. This differs somewhat from the profile of all ASF-supported schools on a number of indicators (as noted later in this section, survey weighting has been used to correct for representation of ASF streams and urban/rural geography):

• PEF-only schools are over-represented and Challenge Authority schools under-represented due to a lower response rate for Challenge Authority schools.
• Schools with lower PEF allocations are under-represented, and those with higher allocations are over-represented.
• Schools in rural areas are under-represented, and schools in urban areas over-represented. Again, survey weighting has been used to correct for this.

Analysis and reporting

1.13. Survey responses have been weighted by ASF stream and urban/rural location to adjust for response bias^[4]; all results presented in the remainder of this report are weighted. Base numbers for each survey question vary due to question non-response – i.e. the results exclude non-respondents to the question unless stated otherwise.

1.14. The volume of survey responses permitted more detailed analysis across key respondent groups. Survey analysis has used hypothesis tests with a 5% significance level to identify significant differences across these groups, and from previous survey findings.^[5] These included:

• ASF stream;
• PEF allocation;
• School sector; and
• Urban/rural location.

1.15. Where variation across these groups is noted in the body of the survey report, this is based on a statistically significant difference between groups.

1.16. Survey data showed some inconsistency between responses and data on ASF support provided to schools, for example, 20 Challenge Authority respondents indicated that their school received only Pupil Equity Funding. Consistent with previous surveys, the categorisation of respondents used in our analysis has been based on Scottish Government records rather than self-reporting.

1.17. The survey included a number of questions giving headteachers the opportunity to respond in their own words. This feedback has been coded into broad themes, and the results presented in the survey report. This includes the percentage of respondents coded into each theme – note that these percentages are based on those answering the question, and respondents can be assigned to multiple themes. Presentation of written feedback also includes direct quotes - this material has been edited for brevity and to ensure anonymity.