Education - Achieving Excellence and Equity: National Improvement Framework and improvement plan 2023

Measuring the attainment gap

It is important to be able to measure the impact of the system as a whole on progress towards closing the poverty-related attainment gap. That is why we use a range of measures that reflect the breadth of issues that can impact on attainment. This avoids, as far as possible, a situation where a single measure generates perverse behaviours by becoming the single focus of activity in schools.

Consultation on enhanced data for improvement

In 2021, both the OECD and Audit Scotland published reports which made recommendations relating to data collection and the need to ensure it reflects the ambitions of Curriculum for Excellence.

Audit Scotland recommended that the Scottish Government:

• "work with stakeholders to develop and publish consistent and robust national data that reflects the ambitions of the national curriculum, national policy priorities such as health and wellbeing and confidence, and key priorities for Covid-19 recovery and improvement
  
• "update the National Improvement Framework (NIF) to reflect data on these agreed outcomes and consider how to ensure that there is greater prominence on these broader outcome measures in public reporting and messaging, for example by inclusion in the NIF key indicators."

The OECD report said:

• "It is also important to recognise that the broad aims of CfE and the four purposes require considering the influence of the wider context in their accomplishment. Scotland should define indicators or a "matrix of success" aligned to the vision and four capacities to help understand students' progress across all four capacities.
• "In addition to the National Improvement Framework's measures of literacy and numeracy, other metrics informing progress on the four capacities are necessary, especially around health and well-being, enjoyment of learning and other key competencies."

As a result, the Scottish Government issued a consultation on:

• how to ensure that the basket of key measures to assess progress towards closing the poverty related attainment gap reflects the wider ambitions of the curriculum, and
• the value of wider data for improvement purposes, both qualitative and quantitative, and the range of data needed by schools, education authorities and at the national level in order to fulfil their different requirements.

The consultation closed on 18 July 2022, and the Scottish Government received 75 responses from a range of interests. Most respondents were supportive of the key principles and felt they were clear in their ambition to reflect the capacities of CfE and the need to reflect a progressive shift in what is valued in education. Respondents also felt that there should be a greater focus on wider achievement and on the whole learner journey from early years onwards (3-18).

The consultation asked whether there should be additional key measures relating to

• attendance and exclusion
• positive destinations
• health and wellbeing
• wider achievement

Overall, the majority of respondents agreed that most of these should be added to the list of key measures. However, many of the respondents raised concerns about adding exclusion to the basket of key measures because of a perceived risk of creating perverse incentives.

Given the concern raised around the implications for including exclusions as a key measure, we decided to leave it as a sub-measure. There was more support for looking at attendance as a key measure, given the importance of attendance for attainment, so it has been included as a new key measure for 2023.

The majority of respondents felt that positive destinations should be included as a key measure alongside the participation measure. There are two measures of positive destinations – initial (3 months after leaving school) and follow-up (9 months after leaving school). We have decided to use 3 months after leaving school, as this provides a closer link between school accountability and school leaver destination. We recognise that this provides only a snapshot of the activity being undertaken by school leavers on a given day. This is why it will sit alongside the existing participation measure, which reports on the wider activity of the 16-19 cohort, including those still at school, in order to provide a more rounded indication of sustained activity.

The majority of respondents agreed that confidence, resilience and engagement needed to be included in some way in the key measures, but many felt that that more consideration needed to be given to how these things are measured.

The findings from the HWB census are not due for publication until February 2023, so they were not included in the 2023 NIF. However, given the support for including measures to reflect health and wellbeing, we intend to include the data in the NIF in future years, when there should be a greater degree of confidence and consistency in the data.

Most respondents agreed that the key measures should reflect a broad range of achievement and not be limited to school attainment. One clear view which came out of the consultation was that there should be a single data set used for improvement purposes going forward. Schools and local authorities tend to use Insight for improvement and benchmarking purposes, whereas progress against the key measures in the NIF uses data from the publication Scottish Statistics on Attainment and Initial Leaver Destinations (SSAILD). This difference has also been raised as part of the joint SG and local government work on reducing variability in performance outcomes, which is being taken forward in response to a recommendation from Audit Scotland.

Scottish Government statisticians are considering the feasibility of aligning the National Statistics SSAILD report and the Insight tool, including the addition of a wider range of providers and courses to SSAILD (on top of National Qualifications). This will require a programme of comprehensive user engagement and development work, as well as technical aspects which will need worked through to ensure the methodology is sufficiently robust for a National Statistics publication.

In order to create a consistent series of measures, we have decided to wait until the work on aligning the National Statistics with Insight is complete, and then adding it as the new measure of attainment. This should be complete ahead of the next NIF in December 2023.

Stretch aims

The Scottish Attainment Challenge Framework for Recovery and Accelerating Progress introduced a requirement for local authorities to set ambitious, achievable stretch aims for progress in overall attainment and towards closing the poverty related attainment gap in the 2022/23 academic year. Stretch aims are to be embedded in local authority education service improvement plans.

Pre-pandemic, the poverty related attainment gap was closing, but the negative impact of the pandemic cannot be ignored, as was demonstrated by much of the data in last year's National Improvement Framework. Therefore, the consultation on using data for improvement also set out the need to recalibrate the original national stretch aims (set out in the 2018 National Improvement Framework) in order to recognise the impact the pandemic had, not only on children and young people's attainment, but also on their health and wellbeing, and understand the scale of the challenge to recover and improve from there. We also need to take into account the new stretch aims set locally in line with the Framework for Recovery and Accelerating Progress.

The core local stretch aims are a subset of the 13 key measures in the National Improvement Framework, plus a locally identified health and wellbeing aim:

• Achievement of Curriculum for Excellence Levels (ACEL) lliteracy for P1, P4 and P7 combined
• ACEL numeracy for P1, P4 and P7 combined)
• school leavers, 1 or more pass at SCQF 5 or better
• school leavers, 1 or more pass at SCQF 6 or better
• the proportion of 16-19 olds participating in education, employment or training based on the Annual Participation Measure produced by Skills Development Scotland
• a locally identified aim for health and wellbeing, to be measured using local datasets.

In addition to these core aims, local authorities have set additional aims which recognise that progress is identified in a broader range of ways at local level than those set out nationally. These aims are of equal importance to the core aims.

In order to ensure a consistent approach across all the key measures, we have decided to use the aggregated local stretch aims for the 5 key measures set out above, and set national stretch aims for the remaining 8 key measures until 2022/23. This reflects the move to a bottom-up approach to setting stretch aims, which is a result of effective partnership working between national and local government, and reflects the shared ambition to get back to, or exceed, pre-pandemic performance. These stretch aims will be published on the National Improvement Framework webpage.

11 original key measures

Of the 11 key measures which have been tracked in previous iterations of the NIF, data is not available to update for one of the measures (HWB: SDQ of 13 and 15 year olds). Of the remaining 10 measures, there has broadly been a narrowing of the gap in four of the measures, a widening of the gap in four of the measures and little/no change in two of the measures.

27-30 month review (children showing no concerns across all domains)

The COVID-19 pandemic has affected delivery of child health reviews. In April 2020, it was advised that child health reviews should be adapted to be undertaken over the telephone, or using 'NHS Near Me' video conferencing where possible, with in-person reviews only undertaken where it was felt necessary by the Health Visitor, or as resources allowed. From June 2020 it was advised that in-person reviews should re-commence in a phased manner linked with the relevant overall Scottish Government COVID strategic frameworks. The data on coverage and data quality in this report represents reviews for the full year of eligible children from April 2020 to March 2021.

The information indicates that there is likely to have been substantial variation in how reviews were provided across this period. Provisional data on the proportion of eligible children reviewed by month, and on the completeness of development data recorded, show there were differences within the year, with children who became eligible in April 2020 particularly affected by lower timely and overall coverage. There is also survey evidence from Scotland that the families of young children experienced a range of challenges over this period, many of which may influence children's development. Children with additional support needs and their families have been particularly affected by the pandemic period. The information presented here is unlikely to fully reflect any impact of the pandemic period on children's development. This is because it covers reviews for children who became eligible from April 2020, at which point only a short period of 'lockdown' restrictions would have been experienced. Full data on children who became eligible for review in 2021/22 will be available from Public Health Scotland in 2023.

In 2020/21, these latest statistics do appear to show that there has been little change in the percentage of children aged 27-30 months who had no concerns identified across all eight domains, from 68.5% in 2019/20 to 68.4% in 2020/21. The gap between children living in the most and least deprived areas of Scotland decreased from 13.8 percentage points in 2019/20 to 12.2 percentage points in 2020/21. Speech, language and communication remains the most frequent developmental domain in which concerns are noted at this review.

HWB: Children total difficulties score (age 4-12)

The percentage of children with borderline or abnormal total difficulties has remained broadly constant since 2015-18, and the gap between children in the most deprived and least deprived areas has remained largely unchanged since 2012-15, at 16 percentage points.

HWB: Children total difficulties score (age 13&15)

This measure was sourced from the Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS). SALSUS has not been recommissioned since 2018, so there is no update. There is the intention that this measure will be updated in future by the new Health and Wellbeing Census introduced in 2021/22. Therefore, currently, the latest information available shows that the gap between children in the most deprived and least deprived areas has remained the same between 2015 and 2018, at 8 percentage points.

Primary - Literacy (P1, P4, P7 combined)

Primary - Numeracy (P1, P4, P7 combined)

In 2021/22, the data shows that pupils from the least deprived areas performed better than pupils from the most deprived areas. However, the gap between the proportion of primary pupils from the most and least deprived areas who achieved their expected level in literacy narrowed compared with 2020/21 and is now similar to those seen before the Covid pandemic. Similarly for numeracy the gap between the proportion of primary pupils from the most and least deprived areas who achieved their expected level has narrowed since 2020/21, and similar to those seen in 2016/17.

Secondary - Literacy (S3, 3^rd level or better)

Secondary - Numeracy (S3, 3rd level or better)

In 2019/20, the data were not collected for any pupils due to difficulties in collecting data whilst secondary schools were closed due to the COVID-19 pandemic. Furthermore, data for secondary school pupils were not collected in 2020/21 due to other pressures on these schools including implementation of the SQA National Qualifications Alternative Certification Model which was used to award National 5s, Highers and Advanced Highers in 2021.

In 2021/22 the gap between the proportion of S3 pupils from the most and least deprived areas who achieved their expected level in literacy widened (to 16.3 percentage points). While the percentage of S3 pupils achieving the expected level in literacy fell for pupils from both the most and least deprived areas, the decrease was larger for those from the most deprived areas.

The gap between the proportion of S3 pupils from the most and least deprived areas who achieved their expected level in numeracy widened (to 15 percentage points), similar to that seen in 2016/17.

SCQF Levels 4, 5 and 6 (1 or more on leaving school)

The leavers' attainment data may include attainment gained throughout all stages of a pupil's education at school. The cancellation of exams and external assessment of coursework in 2020, and the use of the Alternative Certification Model in 2021, will have affected the attainment of many 2020/21 school leavers. It is also likely that the pandemic will have continued to affect the destination choices made by, and opportunities available to, some school leavers in 2020/21. The impacts of these different approaches to certification upon school leaver attainment means that care should be taken when making comparisons over time. The attainment data provides an accurate reflection of the attainment with which school leavers in Scotland left school; for this reason attainment data for 2020/21, 2019/20 and previous years are presented together.

The data based on school leaver attainment between 2015/16 and 2020/21 shows a small increase in the gap for pupils leaving school with 1 or more qualifications at SCQF Level 4 from 6.1 percentage points in 2015/16 to 6.3 percentage points in 2020/21. This is due to a small decrease in the percentage of pupils leaving school from the most deprived areas with at least one qualification at this level and a small increase for those from the least deprived areas.

Over the same time period, the gap for pupils leaving school with 1 or more qualifications at SCQF Level 5 has decreased. Whilst there has been an increase in the percentage of pupils from both the most and least deprived areas leaving school with 1 or more qualification at SCQF Level 5, the increase has been greater for those from the most deprived areas.

For pupils who left school with 1 or more qualification at SCQF Level 6, the gap narrowed between 2015/16 and 2020/21, from 38.5 percentage points to 34.4 percentage points. The increase in the percentage of pupils from the most deprived areas achieving 1 or more qualifications at SCQF Level 6 increased at a faster rate than those from the least deprived areas.

Participation measure

The effects of Coronavirus (COVID-19) pandemic and associated lockdown measures and re-opening of the economy may have influenced the choices made by young adults across the period covered by the 2022 statistics. In addition, for most of the period covered by the 2022 statistics data from DWP about Universal Credit claimants was received. This enabled identification of claimants who were both out of work (unemployed seeking and unemployed not seeking) and in work (employed and self-employed). However, for previous years, this data was not received from DWP. The impacts of the pandemic and the absence of Universal Credit data in prior years should be kept in mind when making comparisons between 2022 and 2021, and when comparing 2022 data with earlier years.

The participation measure shows the gap between the proportion of 16-19 year olds in the most and least deprived areas participating in education, training, and employment has decreased from 12.9 percentage points in 2016 to 9.3 percentage points in 2022. This is driven by an increase in the proportion of 16-19 year olds from the most deprived areas participating in education, training and employment from 83.3% in 2016/16 to 87.4% in 2022. Over the same period the percentage of 16-19 year olds from the least deprived areas participating in education, training or employment has largely remained the same, ranging from 96.2% in 2016 to 96.7% in 2022.

Two new key measures

Of the two new key measures added as a result of the consultation on additional data for improvement (as set out above), there has been a widening of the gap in one, and a narrowing of the gap in the other.

School Attendance

The COVID-19 pandemic has affected children's ability to attend school. As such, school attendance rates have fallen slightly compared with 2018/19. While the attendance rates decreased for pupils from both the most and least deprived areas, the decrease was larger for those from the most deprived areas. In 2020/21, the gap in attendance rates between children living in the most and least deprived areas of Scotland was 6.3 percentage points. This compares with 4.9 percentage points in 2018/19.

Initial Positive destination

This measure provides information on the outcomes for young people approximately three months after the end of the academic year (the 1st Monday in October). Positive destination includes higher education, further education, training, employment, voluntary work, Personal Skills Development and Activity Agreements (up to 2017/18).

The 2019/20 and 2020/21 figures on school leavers' initial destinations will reflect the impact of the coronavirus pandemic (COVID-19) on choices made by, and opportunities available to, leavers in these years. Particular care should therefore be taken when interpreting changes between 2019/20, 2020/21, and other years. The pandemic will have been at least partly responsible for the relatively low proportion of 2019/20 school leavers in a positive destination compared to surrounding years and the size of the latest increase in 2020/21 should be interpreted with this in mind.

The gap in the proportion of school leavers in a positive destination has mainly been narrowing each year since 2015/16 and is now at 4.8 percentage points compared with 7.9 percentage points in 2015/16. While the percentage of school leavers in an initial positive destination has generally been increasing for those from both the most and least deprived areas, the increase has been greater for those from the most deprived areas.

Summary tables of the key measures are set out below, while the remaining sub-measures can be seen in the NIF Interactive Evidence Report.