Attainment Scotland Fund evaluation: third interim report - year 4

Chapter 3 Activities and Outputs: Approaches

3.1 This chapter explores the development of approaches taken by schools and local authorities with regard to addressing the poverty-related attainment gap.

3.2 The first interim report (Years 1 and 2) and the second interim report (Year 3) included a focus on the types of interventions implemented as a result of the Fund and how these were planned for and targeted to reduce the poverty-related attainment gap.

3.3 The evaluation was refocused at the end of Year 3 to consider how schools and local authorities identified, selected and implemented their approach for addressing the poverty-related attainment gap, resulting in a broader exploration of approaches rather than focusing on the intervention level. An associated evaluation question sought to explore the extent to which the selected approach aimed to support pupils and parents from the most deprived backgrounds.

3.4 Evidence is primarily drawn from the Challenge Authority and Schools Programme mid-year progress reports, Local Authority Survey 2019 and Headteacher Survey 2019.

How did schools and local authorities identify, select and implement approaches for addressing the poverty-related attainment gap?

3.5 Overall, evidence sources pointed to the wide variety of approaches implemented around literacy, numeracy, and health and wellbeing. Strategic approaches have been developed by Challenge Authorities tailored to fit local needs and circumstances. There was evidence of an increased focus on the development of local authority-wide approaches, such as whole school nurture approaches related to health and wellbeing. Other authority-wide approaches were also evident, such as the Cost of the School Day^[5] project which has developed in several Challenge Authorities.

3.6 Further evidence on the development of local authority approaches is provided through the Local Authority Survey 2019. Local authorities provided their views on the extent to which the approach for addressing the poverty-related attainment gap had changed within their local authority over the period of funding. Of 27 local authority responses to this question, 20 viewed their approach as having changed either significantly or to some extent. A further seven viewed their approach as having changed to a limited extent. Specifically, with regard to the nine Challenge Authorities, two indicated their approach had changed significantly, three indicated the approach had changed to some extent, and four indicated limited change.

3.7 Local Authority Survey responses also indicated evidence of a shifting focus and streamlining of approaches towards those approaches where there was evidence of effectiveness and impact. Challenge Authority progress reports similarly suggested evidence of change and continuity in approaches, highlighting the maturation of existing approaches in some instances and innovation in others. Innovation included pilot approaches/interventions in development as well as new approaches being rolled out on the basis of positive pilot evaluation. Challenge Authority progress reports clearly linked refinement of approaches to assessments of effectiveness. Where authorities indicated limited or no change in approach, this was primarily due to continuation of an existing approach.

3.8 Challenge Authority reports also pointed to decision-making processes in terms of selecting approaches and interventions. For example, one Challenge Authority had developed a local authority-wide 'menu' of potential interventions which was made available to headteachers throughout the local authority to support decisions at the school level in terms of selecting those approaches most appropriate to the local context.

3.9 It should be noted that Challenge Authorities, which were established in 2015 at the start of the ASF, have had longer to consider, develop and amend their approaches, as well as longer to put evaluation in place. However, non-Challenge Authorities, including some PEF-only local authorities, are also developing co-ordinated, local authority level approaches.

3.10 Enhancements or adaptations in approaches were based on the increasing use of a broad range of data, a focus on measuring impact, and a focus on building sustainability. This is illustrated in the following comment:

'… a shift in focus from use of a wide range of interventions to narrowing the range of approaches ..[..]..informed by our experience and evidence gathering over the past three years' (Schools Programme local authority)

3.11 At the school level, the Headteacher Survey 2019 explored the development of approaches to closing the poverty-related attainment gap and addressing equity in education.

3.12 The survey findings point to high levels of understanding of the challenges and barriers faced by pupils affected by poverty amongst headteachers who responded to the survey. Ninety-six per cent of headteachers who responded to the survey indicated that they understood the challenges and barriers faced by pupils affected by poverty. This was consistent across most respondent groups, although those in rural areas were less likely to feel they understood these challenges. Similarly high levels of awareness of the range of approaches that can help to close the poverty-related attainment gap were evidenced through the survey, with 96% indicating good awareness of the range of approaches. There were similar variation in views across urban and rural areas, with headteachers in rural areas also less likely to have a good level of awareness.

3.13 Confidence amongst headteachers in selecting approaches to close the poverty-related attainment gap that would be most effective in their school setting were also found to be high. Ninety-three per cent of respondents to the Headteacher Survey indicated they felt confident to a great or moderate extent. This underlines the importance of considering local context i.e. selecting approaches suited to their own school setting.

3.14 Change and consistency in approaches within schools to closing the poverty-related attainment gap were explored through the Headteacher Survey 2019. Headteachers were invited to indicate whether approaches within their school had continued at the same level as the previous year, had been scaled up in 2018/19, or had been newly introduced in 2018/19. Analysis of responses indicates that:

• Over half (58%) had scaled up their approaches within schools in 2018/19 from the previous year.
• Almost one third (31%) had continued their approach at the same level as the previous year, with 11% indicating a newly introduced approach in 2018/19.
• Similarly, approximately two thirds (67%) of headteacher respondents to the survey indicated there had been some change in their approach during the past year, including eight per cent whose approach had changed significantly.

Change and consistency in approaches varied according to funding stream, with PEF-only schools, particularly those with lower levels of PEF funding, less likely than Challenge Authority and Schools Programme schools to have changed their approach during the past twelve months. This may be due, in part, to the shorter length of time that PEF has been in operation in comparison with Challenge Authority and Schools Programme funding streams.

3.15 Factors associated with continuing or changing approaches were also explored in the Headteacher Survey. Analysis suggests that evaluation and measuring of impact was most commonly associated with schools changing their approach to closing the poverty-related attainment gap, ranging from the introduction of relatively minor changes through to the termination of specific approaches or interventions which were not perceived as delivering the anticipated impacts. Evaluation and use of evidence was similarly the most common reason for schools continuing with an existing approach, following the demonstration of positive impacts. There was evidence of improved skills in use of data and evidence associated with confidence within schools to change or retain approaches based on data and evidence.

3.16 Further evidence on the development of approaches at the school level was provided via Schools Programme progress reports. Nearly all Schools Programme mid-year progress reports reported on approaches and interventions around literacy, numeracy, and health and wellbeing within a number of different organising themes such as learning and teaching, leadership, and families and communities. As with other evidence sources, there was evidence of continuation and consolidation of existing approaches and interventions, maturation of approaches and interventions, refinement of approaches and also of innovation and new approaches being introduced. For example, some Schools Programme progress reports highlighted the introduction of new activities/interventions in response to needs identified through previous/existing ASF-funded activity, exemplifying the potential for responsiveness within the Schools Programme at the school level to meet local needs.

To what extent did approaches selected aim to support pupils and parents from the most deprived backgrounds?

3.17 The evaluation sought to explore the extent to which selected approaches aimed to support pupils and parents from the most deprived backgrounds with evidence sought from a range of sources.

3.18 Whilst some approaches focused on universal provision, others were more focused in support of pupils from the most deprived backgrounds. The Headteacher Survey 2019 provides some evidence in support of this, with the finding that targeted support for individual pupils was the most common focus. Around three out of four headteachers who responded to the survey indicated that they had a 'strong focus' on targeted support.

3.19 A focus on parental and family engagement has formed a key part of the approach developed in some schools. This has included, for example, approaches aimed at supporting pupil attendance and engagement, and improving pupil aspirations. Headteacher Survey 2019 findings suggest the need for a clear commitment to parental engagement, and recognition of the time required to build relationships with families, not least given negative associations with the school environment which may be held by some parents. However, there was also evidence of the benefits of parental engagement, such as parents being more willing to engage with school activities, and viewing the school as a source of help and support.

3.20 There were a number of specific approaches to achieving and maintaining parental engagement highlighted in headteacher comments, including use of extra-curricular and physical/sports activities, and ensuring free access to activities. The importance of enabling parents to engage with schools in ways parents felt comfortable with was also highlighted. However, there was also the perception raised by some headteachers of the need for greater clarity in terms of how parents can positively support learning without it being overly burdensome.

3.21 Follow-up qualitative feedback gathered as part of the Headteacher Survey highlighted the considerable extent to which achieving equity had been central to schools' approaches to closing the poverty-related attainment gap. However, it also pointed to complexity in understandings of equity in the context of diverse school populations. The following quote illustrates the role played by ASF in encouraging discussions related to equity:

"ASF support has opened the equity conversation out among the wider parent body, through dialogue with school staff and pupils. It has built staff awareness of the pupils most at risk, and what we can do long-term to make a positive impact."

3.22 At the local authority level, approaches to targeting were reported on within Challenge Authority progress reports. Scottish Index of Multiple Deprivation (SIMD) was used most widely as a mechanism to ensure support focused on pupils and parents from the most deprived backgrounds. A number of local authority-wide specific tracking systems have also been developed to target support.