Closing the poverty-related attainment gap: progress report 2016 to 2021

4. Systematic use of evidence to target interventions and maximise impact

Challenge approaches have been developed in a number of different ways, with local authorities supporting schools to target interventions, select the most appropriate approaches, and to evaluate and develop approaches. This chapter provides an overview of the evidence of how these approaches have been developed.

Summary

• Schools select interventions based on a wide range of information, such as local authority support, AA advice and guidance, consideration of data, research on interventions, and input from staff, parents and pupils.
• Local authorities have indicated evidence of a shifting focus and streamlining of approaches towards those approaches where there was evidence of effectiveness and impact.
• Analysis of the different years of the ASF indicate mixed approaches to closing the attainment gap, with a focus on the most deprived alongside other ‘universal approaches’. Most schools have taken a mixed approach, with 85% of all respondents indicating that they have used ASF to support ‘universal’ approaches.
• Targeted support for individual pupils is an emphasis in the approach of nearly all schools (98%) and a ‘high emphasis’ in 3 out of 4 schools. The next most common themes in approaches were teachers skills or practice (92% of schools), resources or tools for teaching (89%), self-improvement/improvement planning (88%), dedicated staff time(86%) and data skills or use (85%).
• Local authorities are supporting effective self-evaluation that is leading to improved tracking and evaluation of impact, and refinement of approaches.

4.1 Approaches to selecting interventions

Schools select interventions based on a wide range of factors, such as local authority support, AA advice and guidance, consideration of data, research on interventions, and input from staff, parents and pupils.

There are many factors that influence how schools select appropriate approaches, as these are often tailored to each school’s needs. Schools and headteachers have support from local authorities to discuss which approaches work best, and the great majority of headteachers feel confident in choosing ones that would help close the attainment gap.

In the 2020 Headteacher survey, factors associated with the development of approaches included:

• Collaborative working;
• Evidence of approaches becoming more embedded;
• Improvements in use of guidance and planning;
• Increased focus on the use of data, including greater rigour in use of data, greater access to data and greater data literacy;
• Staffing increases;
• Increasing opportunities for professional learning;
• Easily accessible support and clear communication channels;
• Improved understanding of the poverty-related attainment gap and of the experience of poverty and its impact on children and families;
• Continued development of mechanisms to support strategic planning and governance;
• Aspirational target setting; and
• Increasingly adaptive and responsive approaches.

In the first two years of the ASF, there were wide-ranging and varied approaches to choosing interventions across authorities. Some schools were given autonomy and flexibility by their local authorities to select a given intervention, while other schools were provided with a suite of potential interventions to choose from. Factors that played a role were data and evidence, support from AAs and input from parents/carers and pupils.

Qualitative research asked teachers and local authorities to describe their approach to selecting interventions. It revealed that data and evidenceplayed a key part in the process of selecting interventions and that local authorities largely took the lead in this process.

AAs have played a key role in supporting the development of approaches. Most AAs interviewed in the qualitative research indicated that they supported schools and teachers, particularly in primary schools, to identify and monitor interventions.

Our collaborative work with our AA has been a significant factor in improvements over the last 12 months. [Their] work with schools to help teams understand the importance of data; to support schools to see the links between raising attainment and an appropriate and relevant curriculum; [their] support to the project lead in developing a more coherent attainment challenge plan all contributed to a productive and successful year. [Challenge Authority Progress Report]

At a local authority level, interventions were chosen to reflect a number of broad priorities, based on, for example, attainment gap data, previously successful approaches and engagement with headteachers and other partners or services.

In the ASF qualitative research case studies undertaken in Year 3 of the ASF, local authority officers indicated that they set broad priorities for selecting approaches through:

• reviewing data to identify attainment gaps;
• engaging headteachers to explore views;
• building on previous successful approaches and experience of programmes;
• gathering evidence and research about successful approaches; and
• building on good relationships with other partners and services.

In some cases, local authority officers encouraged headteachers to select their own interventions, and research approaches themselves (with support). These local authority areas felt that it was important that teachers took ownership of the approaches, and drove the approach. In others, schools were offered access to interventions which were set at a local authority wide level. This was to ensure that the approach was joined up and clearly managed and governed. Some areas also indicated that teachers did not always have time to review research and evidence.

Case study findings from the 2018 Headteacher survey highlighted the value of a coordinated approach to ASF interventions, and the substantial planning time required. This was reflected in several schools using ASF to fund staff whose roles are dedicated to coordinate supported interventions. Some schools (typically primary schools and smaller secondary schools) had used the ASF to introduce a single role to coordinate activities, while others had introduced dedicated leads for specific areas. These schools identified benefits in dedicated staff members having the time required to support delivery, to ensure a more coherent approach across the school, and to support development of whole-school approaches.

Participation of Children and Young People in deciding approaches

Evidence from qualitative case studies show that input from teaching staff, parents/carers and children was identified as an important element in ensuring interventions were based on an accurate understanding of children’s needs and the local community. Schools also noted that pupils and parents often identified a different set of priorities to those identified by staff, demonstrating the value of ensuring a broad range of perspectives. Examples of pupil and parent input included consultation exercises to develop priorities, use of participatory budgeting with pupils, and ongoing engagement through pupil representative groups.

Almost a third of AA professional reports collated in late 2020 noted particular initiatives targeted specifically at promoting participation of children and young people affected by poverty. This included developing programmes to raise the understanding of the impact of poverty with children and young people.

A few reports described how learners are being consulted about ways to respond to emerging needs in the context of the pandemic and some described participation in programmes relating to children’s rights including the UNICEF Rights Respecting Schools programme.

In one authority, funding from the Attainment Scotland Fund is supporting and increasing opportunities for young people to take leadership roles, for example, the Pupil Led Enquiry programme. The Senior Leadership Team has identified the need to ensure that pupil voice is further developed in order for it to make an impact on the delivery of learning and teaching. The authority should continue with its plans to build on innovative programmes which are increasing pupil voice and aspiration and integrate the learning and approaches more fully into the curriculum. [Challenge Authority Inspection]

One local authority implemented the Social Justice Ambassadors Programme which uses the voice of young people to tackle current social issues, many of which are related to poverty. It is an educational programme which supports young people to become social researchers exploring these issues and how they impact on their own community and the world around them. The approach was successfully piloted and culminated in an event where young people presented 19 motions regarding community issues to lead staff from across the LA including Elected Members, third sector and Community Partnerships. They worked together to explore the young people’s findings through an interactive café conversation. The current research focus is centred on the impact of COVID-19, poverty and recovery. Collaborations with school staff and young people are being used to explore PEF spend and in one high school, they are discussing using the Young Scot card as a means to allow young people to purchase resources they require.

4.2 Targeting of approaches

Most schools have implemented both targeted and universal interventions, with 85% of all respondents indicating that they have used ASF to support ‘universal’ approaches.

Analysis of the different years of the ASF indicate mixed approaches to closing the attainment gap, with a focus on the most deprived alongside other ‘universal approaches’. Survey responses from the 2020 Headteacher Survey indicated that a large majority of schools have included a focus on the pupils or parents experiencing deprivation or disadvantage as part of their approach to achieving equity; 83% include a focus on those experiencing socio-economic deprivation and 77% include a focus on other types of disadvantage. However, most schools have taken a mixed approach, with 85% of all respondents indicating that they have used ASF to support ‘universal’ approaches. These findings are consistent across most key respondent groups, although schools in rural areas are less likely to include a specific focus on those affected by disadvantage.

Over half (58%) of respondents in the 2018 survey reported targeting at least some of their interventions in ‘other’ ways. A total of 153 respondents provided a description of these other ways. This included using attainment, attendance, exclusion or risk of exclusion data. Headteachers also looked to individual characteristics when targeting their interventions, including: additional support needs; care experienced; adverse childhood experiences and having English as an additional language.

In addition, the profile of the school influenced the nature of targeted approaches. For example, schools with a large proportion of their school roll from the most deprived areas needed to take into consideration other criteria in order to prioritise resources. For schools with a small proportion of their school roll registered for free school meals, consideration of a wider range of needs helped ensure an inclusive approach.

4.3 Focus of approaches

Most school approaches had a broad focus – referring to multiple themes such as targeted support for individual pupils, improving teachers skills or practice, improving data skills.

As well as addressing key areas for children and young people – such as literacy, numeracy and health and wellbeing – school approaches also sought to develop wider school improvement themes, such as teaching skills and use of data and evidence. Responses from the 2020 Headteacher Survey indicated that the approaches selected by schools generally had a relatively broad focus in this respect, with the great majority referring to multiple themes. The responses to the 2020 survey are shown in Figure 6.1. Targeted support for individual pupils is an emphasis in the approach of nearly all schools (98%) and a ‘high emphasis’ in 3 out of 4 schools. The next most common themes in approaches were teachers skills or practice (92% of schools), resources or tools for teaching (89%), self-improvement / improvement planning (88%), dedicated staff time (86%) and data skills or use (85%). The responses show that approaches included a broad range of themes.

4.4 Changes in approach over time

Local authorities have indicated evidence of a shifting focus and streamlining of approaches towards those approaches where there was evidence of effectiveness and impact.

The ASF evaluation considered the extent to which plans changed over time. Headteachers responding to the online survey were asked to indicate to what extent their interventions (supported by any of the funding streams) were new, a scale up from previous year or continuing at the same level. Across both Year 2 and 3, respondents most commonly indicated that the interventions were newly introduced.

In Year 3 of the Headteacher Survey, there was an increase in those reporting that most interventions were newly introduced and a reduction in those reporting that most interventions were a scale up of an intervention implemented in previous year. This reflects the inclusion of PEF-only schools; 61% reported that most interventions were newly introduced compared to 30% of Schools Programme schools and 47% of Challenge Authority schools.

Further evidence on the development of local authority approaches is provided through the Local Authority Survey 2019. Local authorities provided their views on the extent to which the approach for addressing the poverty-related attainment gap had changed within their local authority over the period of funding. Of 27 local authority responses to this question, 20 viewed their approach as having changed either significantly or to some extent. A further seven viewed their approach as having changed to a limited extent. Specifically, with regard to the nine Challenge Authorities, two indicated their approach had changed significantly, three indicated the approach had changed to some extent, and four indicated limited change.

Local Authority Survey responses indicated evidence of a shifting focus and streamlining of approaches towards those approaches where there was evidence of effectiveness and impact. Challenge Authority progress reports similarly suggested evidence of change and continuity in approaches, highlighting the maturation of existing approaches in some instances and innovation in others. Innovation included pilot approaches/interventions in development as well as new approaches being rolled out on the basis of positive pilot evaluation. Challenge Authority progress reports clearly linked refinement of approaches to assessments of effectiveness. Where authorities indicated limited or no change in approach, this was primarily due to continuation of an existing approach.

Approaches to closing the poverty-related attainment gap are being refined based on improvement and use of evidence. Enhancements or adaptations in approaches to literacy, numeracy, and health and wellbeing introduced to close the poverty-related attainment gap were based on the increasing use of a broad range of data, a focus on measuring impact, and a focus on building sustainability.

Enhancements or adaptations in approaches were based on the increasing use of a broad range of data, a focus on measuring impact, and a focus on building sustainability. This is illustrated in the following comment:

'… a shift in focus from use of a wide range of interventions to narrowing the range of approaches ..[..]..informed by our experience and evidence gathering over the past three years'(Schools Programme local authority)

4.5 Evaluation of approaches

Local authorities are supporting effective self-evaluation that is leading to improved tracking and evaluation of impact, and refinement of approaches.

The majority of the nine challenge authorities were assessed through inspections as having robust and highly-effective self-evaluation processes in place. Where practice is strongest, self-evaluation is securing significant improvement at all levels in the system. There are clear links between the strategic vision for improved outcomes and the day-to-day practice of staff. In a minority of challenge authorities, whilst self-evaluation has improved, it is not yet leading to improved outcomes for learners. More time and a greater depth of consistency is needed to embed their approaches. All challenge authorities use a range of approaches to ensure stakeholders are actively consulted through self-evaluation.

Case study feedback indicates that ASF support has helped to improve capacity for tracking and evaluation of impact, with schools gathering a substantial volume of evidence around intervention. Some schools noted that their local authority had made a positive contribution to their efforts to evaluate ASF activity.

Most schools reported adapting their interventions in response to emerging monitoring data. Schools also reported feeling more confident about trialling new approaches on the basis that findings from their ongoing evaluation activities could be used to refine the approach over time.

A Challenge Authority’s inspection found that self-evaluation permeates every aspect of education at authority, school and individual practitioner level. There is a clear, shared expectation that everyone involved will evaluate their practice and continuously improve it. This involves regularly gathering and analysing the views of learners, parents and other stakeholders who are involved in education. The authority’s extremely effective approach to self-evaluation informs policy and practice and is focused on ensuring that learning activities are appropriate. [Challenge Authority Inspection]