Attainment Scotland Fund evaluation: interim report (years 1 and 2)

6. Choosing Interventions

6.1. This chapter explores the process of choosing interventions and the types of interventions that were planned as a result of the Attainment Scotland Fund.

6.2. The first section of the chapter describes the process of choosing interventions and whether these were organised around the focus areas of the Attainment Scotland Fund. That is; Literacy, Numeracy and Health and Wellbeing.

6.3. The second section of the chapter describes the nature of these interventions. For example, it explores whether the interventions focused on professional learning, parental involvement, leadership or data skills.

6.4. The final section explores whether interventions were new, combined with existing interventions or a scale-up of existing interventions.

6.5. All three sections draw on data from the qualitative research, the annual headteacher survey, Challenge Authority reports and school reports.

Chapter Highlights – Choosing Interventions

• Local authorities and schools used a wide range of methods to choose interventions. However, common factors that played a role were data and evidence, support from Attainment Advisors and consideration of the target audiences.
• Approaches varied, and so did the level of autonomy given to schools to choose their own interventions. On the whole, teachers felt included in the process.
• Most interventions focused on Numeracy, Literacy and Health and Wellbeing, family support and engagement, teacher skills development and equipment and resources.
• During the first two years of the programme, there appeared to be a greater focus on Literacy and Health and Wellbeing interventions than Numeracy.

To what extent did interventions focus on Literacy, Numeracy, Health and Wellbeing or other topics?

Selecting Interventions – Key factors

6.6. Overall, the following factors played a significant role in selecting interventions:

Figure 6.1: Significant factors in selecting interventions

6.7. The qualitative research asked teachers and local authorities to describe their approach to selecting interventions. It revealed that data and evidence played a key part in the process of selecting interventions and that local authorities largely took the lead in this process. Chapter 12 discusses in more detail the extent to which data was used to support the identification of appropriate interventions.

6.8. There was variability in the approach taken to selecting interventions. Some schools were given autonomy and flexibility by their local authorities to select a given intervention. Other schools were provided with a suite of potential interventions to choose from. The approach varied by local authority.

6.9. Given this variability, teachers' perception of how much autonomy they had in selecting interventions also varied. Whilst perceptions differed, most teachers interviewed felt involved in the process of selecting interventions.

6.10. A minority of headteachers (headteacher survey, Year 2) spontaneously mentioned flexibility as a factor which helps the fund to succeed. They claimed it allowed them to be more creative and it gave them freedom to experiment and arrive at the best possible solution.

6.11. At a local authority level, interventions were chosen to reflect a number of broad priorities, based on, for example, attainment gap data, previously successful approaches and engagement with headteachers and other partners or services.

6.12. Teachers in the Schools Programme appeared to have more autonomy to select interventions. At a school level, data and evidence played an important role informing which interventions to implement. Interviews revealed that teachers drew on evidence; existing local priorities; views from staff or parents; and learning from previous interventions.

6.13. The qualitative research revealed that most teachers in the Schools Programme welcomed the autonomy they had to selecting interventions with only one indicating they would have liked more support.

6.14. When asked about what works well in the national and local governance, arrangements and support, some headteachers focused on the right balance between autonomy and guidance as an indicator of successful leadership (headteacher survey, Year 2). Overall, those who mentioned a distinguishing quality of the leadership style of their local authority mentioned flexibility as a key positive feature.

6.15. Conversely, a minority of headteachers (headteacher survey, Year 2) mentioned a need for more flexibility when asked about the improvements in governance they would like to see in the future. They noted that the inability to select the interventions which are most appropriate for the school delays their progress and decreases their effectiveness. Lack of flexibility was also mentioned as one of the barriers to the success of the fund, but seemed to apply to the issues around staff hiring more than insufficient autonomy when selecting initiatives.

6.16. Information from the Challenge Authority reports gave some insight into how and why specific interventions were selected. These reports revealed that in addition to using SIMD data, authorities chose interventions based on specific Numeracy, Literacy or Health and Wellbeing measures. The prominent role of data was also evident, with local authorities confirming the use of self-evaluation to choose interventions.

6.17. Most Attainment Advisors interviewed in the qualitative research indicated that they supported schools and teachers, particularly in primary schools, to identify and monitor interventions. However, a few highlighted that they were not in post at the time of selecting interventions – so their support in that respect was limited.

6.18. The target audience was also considered when deciding which intervention to select; pupils who live in areas of higher deprivation, all pupils, or pupils who may require additional support due to, for example, physical disability, domestic abuse and family conflict, or care status. Chapter 7 provides further detail.

Overview of interventions

6.19. Overall, a wide range of interventions took place as part of the fund. Mostly, these focused around Literacy, Numeracy and Health and Wellbeing. However, interventions also covered family support and engagement, teacher skills development and equipment and resources.

Figure 6.2: Key focus areas of interventions

6.20. There were some marked differences between the primary and the secondary programme, which could be a reflection of the length of time each of these programmes had existed. By the end of the second year, interventions in the primary programme were for the most part, well organised and appeared embedded in the system. Reports from the secondary programme tended to be more general. Though it should be noted, that by the end of Year 2, the secondary programme had been running for one year only.

6.21. From the progress reports submitted, it appeared that interventions around Literacy were more widespread than those focusing on Numeracy. In fact, some authorities did not appear to have any interventions to support Numeracy at all. This issue was emphasised in the secondary programme. The qualitative work suggested that interventions around Numeracy had started later.

6.22. Evidence was unclear as to where the barrier lies in closing the Numeracy attainment gap; if it was around a strategic barrier, teacher confidence/knowledge, lack of suitable resources or something else.

6.23. The remainder of this chapter describes the interventions that focused on Literacy, Numeracy and Health and Wellbeing. The next section of this chapter explores interventions that focused on other or additional areas.

Literacy Interventions

6.24. Most teachers interviewed in the qualitative research felt very positive about Literacy being one of the key focuses of the fund. Teachers believed that the fund had helped the school to embed consistent approaches to Literacy, and spend more time on Literacy within the curriculum.

6.25. Literacy interventions reported in the qualitative research ranged from specific Literacy champions or initiatives, one to one support and partnerships with speech and language therapists.

6.26. Information on the type of Literacy interventions was also collected as part of the headteacher survey (Year 2). Overall, 30% of respondents listed Literacy initiatives.

6.27. Literacy also featured strongly in the Schools Programme. Information provided showed that 235 Literacy interventions and approaches were undertaken during 2016/17. A wide range of expected impacts and measures were proposed by schools to support the implementation of these approaches and establish impact.

Numeracy Interventions

6.28. While Numeracy featured prominently during the first two years of the fund, in the progress reports it appeared relatively less strongly than Literacy or Health and Wellbeing interventions.

6.29. Teachers participating in the qualitative research reported that a focus on Numeracy within the fund had helped to change teaching practice.

6.30. Numeracy interventions reported in the qualitative research included dedicated Numeracy champions, the use of new resources, approaches and additional targeted support to pupils.

6.31. Information collected on the headteacher survey (Year 2) showed that 27% of respondents listed Numeracy initiatives.

6.32. In the Schools Programme in particular, there appeared to be less of a focus on Numeracy when compared to other priorities. This was the case during the first two years of the programme. Specifically, during 2016/17, 190 Numeracy interventions and approaches were identified across 55 of the 74 schools. As was the case for Literacy interventions, a wide range of impacts and measures were proposed by schools to support the implementation of these approaches and establish impact.

Health and Wellbeing

6.33. Overall, Health and Wellbeing interventions were widespread across all Challenge Authorities and schools during the first two years of the programme.

6.34. Many teachers in the qualitative research felt that good Health and Wellbeing was critical in that that it provided the foundation for learning and improved attainment. This included addressing social and emotional needs, to ensure that children were able to attend school, enjoy school and be ready to learn.

6.35. Health and wellbeing interventions described by teachers in the qualitative research were varied and focused on a variety of areas including for example: nurture, transitions, targeted support, outdoor learning and mindfulness.

6.36. Information on the type of Health and Wellbeing interventions was also collected as part of the headteacher survey (Year 2). Overall, 24% of respondents listed Health and Wellbeing initiatives.

6.37. In the Schools Programme self-reporting for 2016/17, 67% of schools included third sector partnership as part of their Health and Wellbeing interventions and approaches.

Other Interventions

6.38. Just under half (48%) of headteachers (Year 2) referred to initiatives that focused on key improvement areas highlighted as part of the fund requirements. That is: parental engagement, use of data, staff development, resources and third sector partnerships.

Schools Programme – Specific evidence on interventions

6.39. Evaluative reports for 2016/17 and proposals for 2017/18 were submitted by schools from all 12 local authorities involved in the Schools Programme in May 2017.

6.40. There was a decrease in the number of interventions and approaches identified by schools in Literacy, Numeracy and Health and Wellbeing. However, there was an increase in the number of combined Literacy, Numeracy and Health and Wellbeing interventions and approaches identified.

6.41. As was the case across Challenge Authorities, learning and teaching remained the most prevalent lever for change in the Schools Programme

6.42. 'Families and Communities' was an area that showed growth over time. In the 2017/18 Schools Programme plans, 86% of schools (63 out of 73) included families and communities approaches and interventions. This represents a 10% increase from the previous year's plans.

6.43. In the Schools Programme there was a decline in planned third sector partnerships for 2017/18.

What type of interventions were organised?

6.44. The headteacher survey asked respondents to comment on the extent to which their interventions focused on:

• Leadership skills
• Collaboration within the school or across schools
• Data skills or use of self-evaluation and/or improvement planning
• Teaching skills or practice
• Resources or tools for teaching or learning
• The learning environment
• Family learning
• Parental or community engagement with the school

6.45. Overall, the responses suggested that the interventions organised focused on each of these areas at least to some extent.

6.46. In both Years 1 and 2, most initiatives focused on teaching skills or practice. Specifically, in Year 2, 76% of headteachers reported that there was a strong emphasis on teaching skills or practice. Headteachers' responses to the questions relating to sustainability of the progress made as a result of the fund suggest that the focus on teaching skills could have resulted from their conviction (shared by 79% of headteachers) that staff up-skilling was one of the benefits of the programme which is most likely to last beyond the years of the fund.

"We have focused on changing practice through professional learning thus making on-going refinement and improvement of approaches cost neutral' (Challenge Authority, Headteacher Survey – Year 2)

6.47. Data skills was also a prominent area of focus, in particular during Year 2. Whilst headteachers in Year 1 indicated that there were relatively fewer interventions focusing on data skills, 50% of headteachers in Year 2 reported there to be a strong emphasis on data skills and a further 36% reported there to be some emphasis.

6.48. Responses to the survey in both years indicated that relatively fewer initiatives focused on family learning or parental or community engagement with the school. The evidence for why that might be the case was not clear, but 30% of headteachers (Year 2) said that the work at school could be undermined by disengaged parents, citing this as a perceived barrier to the success of the fund.

6.49. Further detail on the focus of the interventions as reported by headteachers (Year 2), is displayed in the figure below.

Figure 6.3: Focus of the interventions – headteacher survey Year 2

6.50. The qualitative research revealed that in addition to describing interventions according to the three main priorities ( i.e. Literacy, Numeracy and Health and Wellbeing), teachers and local authority officers also described interventions which focused on:

• Family support and engagement
• Skills development
• Equipment and resources

To what extent were interventions new, a scale-up of existing interventions or a continuation?

6.51. As expected, during Year 1 most initiatives were reported to be new to the school after the launch of the Attainment Scotland Fund. Overall, headteachers stated that 379 initiatives were newly introduced in their school following the launch of the fund, 189 were a scale up initiative that existed prior to the fund and 43 initiatives continued at the same level.

"The additional resource allows the opportunity to increase scale of interventions and accelerate their implementation" (Schools Programme, Headteacher Survey – Year 2)

6.52. In Year 2, most initiatives continued to be either new to the school or a scale up of an existing initiative. 39% of headteachers said that most initiatives were newly introduced during Year 2 and 19% stated that most initiatives were a scale up from the previous year. Only 6% of respondents reported that most initiatives continued at the same level as in previous school year (i.e 2015/16). Further detail in the figure below.

Figure 6.4: Proportion of initiatives new or a scale up (headteacher survey)

6.53. The qualitative research showed that in many cases the interventions being implemented were extensions of previously successful approaches. Teachers commented that the funding had allowed them to strengthen existing approaches or implement and measure the impact of the approach in a more focused way.

6.54. Similarly, local authority officers reported that the fund had been used to scale up the implementation of an intervention that was previously successful or related to local priorities.

6.55. Progress reports submitted by Challenge Authorities suggested that many interventions reached a small number of pupils or schools, with some activities focusing on one to one support. The reports did not include strategies to scale up the programme.