Attainment Scotland Fund evaluation: fourth interim report - year 5

Chapter 5 Long-term outcomes: Contribution to Improvement and Reduction of the Poverty-related Attainment Gap, and Sustainability

5.1 This chapter explores evidence around improvement in attainment and health and wellbeing, and the gap between pupils from the most and least deprived areas. Evidence of impact draws on both analysis of quantitative data on attainment and wellbeing, based on agreed measures for monitoring progress on closing the poverty-related attainment gap, and on reported impacts from survey evidence.

5.2 The first section provides analysis of quantitative data on attainment and wellbeing based on the measures for monitoring progress on closing the poverty-related attainment gap as set out in the National Improvement Framework. For 2019/20, data on Attendance and Exclusions is not available as it is published on a biennial basis. ACEL 2019/20 data was not collected during the period of school building closures.

5.3 The second section explores evidence of impact provided by local authorities and schools, based primarily on survey data drawn from the Headteacher Survey 2020 and the Local Authority Survey 2020. This is followed by consideration of evidence around sustainability.

Evidence of impact: attainment and wellbeing

5.4 Evidence of impact draws on analysis of quantitative data on attainment and wellbeing based on the agreed measures for monitoring progress towards closing the poverty-related attainment gap set out in the National Improvement Framework. This sets out a basket of key measures and sub-measures to assess progress. The measures with available data for this (and previous) reporting periods are shown in Table 5.1 below.

Table 5.1: National Improvement Framework Measures

			Pre ASF 2014/15	Year 1 (2015/16)	Year 2 (2016/17)	Year 3 (2017/18)	Year 4 (2018/19)	Year 5 (2019/20)
Attainment	Achievement of Curriculum for Excellence Levels	P1, P4 and P7 S3			√	√	√
	School leaver attainment in National Qualifications – SCQF Level 5 and 6 or better*	School leavers		√	√	√	√	√
	Annual Participation Measure	16-19 year olds			√	√	√	√
Health & Wellbeing	Attendance rates	Primary, Secondary	√		√		√
	Exclusion rates	Primary, Secondary	√		√		√
	Total difficulties score	Primary, Secondary		√	√	√	√
	Mental Wellbeing Score (SALSUS)	Primary, Secondary		√	√			√

* Level 4 is not included within the attainment related measures – see Evaluation Strategy for Year 3 and 4 Table 2.1.

5.5 All of the measures are available at both Scotland and local authority level. Patterns of attainment in Challenge Authorities, who have been involved with ASF since 2015^[5], and non-Challenge Authorities are outlined in this analysis.

5.6 To address the research questions, each of the measures outlined in the above table are considered in terms of:

• Overall attainment
• Attainment for pupils from most and least deprived SIMD quintiles
• Percentage point gap between the most and least deprived

5.7 In addition, data for the following groups will be shown.

• Each of the nine Challenge Authorities and Challenge Authority total
• Non-Challenge Authorities
• Scotland Level

5.8 A range of additional factors influence the extent to which different measures provide us with an accurate and up to date assessment of progress:

• There is a delay in terms of the progress of the ASF and the evaluation. The evaluation is retrospective, in that data reported has been collected for the previous year;
• Some of the data is only available over a small time period, and therefore it is difficult to start ascertaining broader trends. In addition, data is not gathered at each year stage. For example, an authority/school could have been targeting P2 which would not be seen in ACEL for at least two years as this is gathered at P1, P4 and P7;
• The differential implementation of ASF at local authority/school level (based on the funding streams) also introduces a further level of complexity, and affects the extent to which meaningful trends can be inferred from the data. For example, implementation in the secondary sector did not begin until Year 2 of the ASF.
• The introduction of PEF in 2018/19 brings further complexity.

School leaver attainment: percentage of school leavers achieving awards by Scottish Credit and Qualifications Framework (SCQF) level

5.9 The attainment of school leavers in Scotland is based on the Summary Statistics for Attainment and Initial Leaver Destinations, No. 3: 2021 Edition published 23 February 2021. This data includes attainment in National Qualifications achieved throughout all stages of a pupil’s schooling.

5.10 When considering the school leaver attainment publication and the data it presents it is important to be aware of the circumstances around National Qualification certification in 2020. The COVID-19 pandemic led to the cancellation of 2020 National 5 (SCQF Level 5), Higher (SCQF Level 6) and Advanced Higher (SCQF Level 7) exams and the decision for the Scottish Qualifications Authority (SQA) not to collect assess coursework. Instead the grades in these qualifications in 2019/20 were based on teacher estimates^[6]. This will have affected the attainment of many 2019/20 school leavers presented in this data. It is for this reason that a dashed line break in the series has been placed between 2018/19 and 2019/20 to indicate that care must be taken when comparing 2019/20 attainment to that of earlier years. It is not possible to fully determine the extent to which the coronavirus pandemic and, more specifically, the certification methods used in 2020 have affected the attainment levels of the 2019/20 school leaver cohort.

5.11 Whilst the School Leaver Attainment publication presents the data and notes the change year on year it is important to understand that any change between the attainment levels of the 2019/20 cohort and those of previous years should not be seen as an indication that performance has improved or worsened without further evidence.

5.12 This section will consider the proportion of school leavers attaining one or more pass at SCQF Level 5 to 7 in Challenge Authorities (combined), non-Challenge Authorities (combined) and Scotland overall, from 2016/17 to 2019/20.

5.13 As seen in Table 5.2 the proportion of school leavers attaining one pass or more at SCQF Level 5 or better for Challenge Authorities (combined) decreased slightly initially (from 84.8 per cent in 2016/17, to 84.2 per cent in 2017/18 and to 83.7 per cent in 2018/19) before returning back to 2017/18 levels (84.2 per cent) in 2019/20. This mirrored the trend at a national level. Non-Challenge Authorities saw a slight increase from 86.6 in 2016/17 to 86.7 percent in 2017/18 and decreased to 85.6 percent in 2018/19 increasing to 86.4 percent in 2019/20.

Table 5.2: Percentage of school leavers achieving 1 or more passes at SCQF Level 5 or better, 2016/17 to 2019/20

	2016/17	2017/18	2018/19	2019/20
Clackmannanshire	79.4	76.1	78.8	77.1
Dundee	82.7	76.8	78.8	80.1
East Ayrshire	83.4	82.4	80.0	83.0
Glasgow	83.2	83.8	83.2	82.0
Inverclyde	88.9	89.1	89.5	89.3
North Ayrshire	84.7	86.2	83.3	89.4
North Lanarkshire	85.5	85.2	84.4	85.2
Renfrewshire	88.3	88.2	88.2	89.7
West Dunbartonshire	87.9	83.1	85.6	85.6
Challenge Authorities	84.8	84.2	83.7	84.2
Non-Challenge Authorities	86.6	86.7	85.6	86.4
Scotland	86.1	85.9	85.1	85.7

Note: A dotted line break has been placed between the attainment figures for 2018/19 and 2019/20 school leavers reflecting the impact of the change to the assessment approach in 2020 on 2019/20 school leaver attainment.

5.14 Table 5.3 shows the proportion of school leavers attaining one or more pass at SCQF Level 6 or better has seen a similar trend across Challenge Authorities (combined), and Scotland overall, from 2016/17 to 2019/20. This trend has featured an increase from 2016/17 to 2017/18, followed by a decrease in 2018/19 and then an increase of more than 3 percentage points in 2019/20. In Challenge Authorities, the proportion increased from 59.1 per cent in 2016/17 to 59.8 per cent in 2017/18, and decreased to 58.7 per cent in 2018/19 before rising to 61.9 in 2019/20. In non-Challenge Authorities there was an increase from 62.1 per cent to 63.1 per cent in 2016/17 to 2017/18, a decrease to 61.3 per cent in 2018/19 and an increase to 64.7 per cent in 2019/20.

Table 5.3: Percentage of school leavers achieving 1 or more passes at SCQF Level 6 or better, 2016/17 to 2019/20

	2016/17	2017/18	2018/19	2019/20
Clackmannanshire	53.5	49.7	54.4	53.2
Dundee	58.1	48.9	53.6	58.0
East Ayrshire	58.4	60.5	58.1	59.2
Glasgow	55.9	59.6	58.5	60.6
Inverclyde	64.8	68.0	67.4	70.3
North Ayrshire	57.6	59.9	56.6	61.4
North Lanarkshire	59.3	60.7	59.0	63.5
Renfrewshire	65.6	65.7	63.1	66.5
West Dunbartonshire	64.2	57.6	58.1	59.9
Challenge Authorities	59.1	59.8	58.7	61.9
Non-Challenge Authorities	62.1	63.1	61.3	64.7
Scotland	61.2	62.2	60.5	63.9

Note: A dotted line break has been placed between the attainment figures for 2018/19 and 2019/20 school leavers reflecting the impact of the change to the assessment approach in 2020 on 2019/20 school leaver attainment.

5.15 As seen in Table 5.4, from 2017/18 to 2019/20, the proportion of school leavers attaining or more one pass at SCQF Level 7 or better for Challenge Authorities (combined), non-Challenge authorities (combined) and Scotland increased slightly in 2017/18, decreased marginally in 2018/19 before increasing by around 4 percentage points in 2019/20. In Challenge Authorities, the proportion slightly increased from 15.4 per cent in 2016/17 to 15.9 per cent in 2017/18, decreased to 14.8 per cent in 2018/19 and rose to 18.7 per cent in 2019/20. In non-Challenge Authorities there was an increase from 21.0 per cent to 22.0 per cent in 2016/17 to 2017/18, a decrease to 20.9 per cent in 2018/19 then an increase to 25.1 per cent in 2019/20. At Scotland level there was an increase from 19.3 per cent to 20.2 per cent in 2016/17 to 2017/18, a decrease to 19.1 per cent in 2018/19 and an increase in 2019/20 to 23/3 per cent.

Table 5.4: Percentage of school leavers achieving 1 or more passes at SCQF Level 7, 2016/17 to 2019/20

	2016/17	2017/18	2018/19	2019/20
Clackmannanshire	13.9	14.9	19.1	19.1
Dundee	15.9	12.8	14.4	20.6
East Ayrshire	15.0	16.9	15.4	15.7
Glasgow	13.7	15.0	13.7	17.4
Inverclyde	19.1	19.1	17.9	22.4
North Ayrshire	17.1	17.5	15.5	21.7
North Lanarkshire	13.9	15.2	12.3	17.3
Renfrewshire	19.2	19.3	20.6	21.6
West Dunbartonshire	16.9	14.8	14.3	19.2
Challenge Authorities	15.4	15.9	14.8	18.7
Non-Challenge Authorities	21.0	22.0	20.9	25.1
Scotland	19.3	20.2	19.1	23.2

Note: A dotted line break has been placed between the attainment figures for 2018/19 and 2019/20 school leavers reflecting the impact of the change to the assessment approach in 2020 on 2019/20 school leaver attainment.

5.16 At SCQF Level 5 or better (Table 5.5), the percentage point gap between the proportion of school leavers from the most and least deprived areas attaining one or more pass has reduced for Challenge Authorities and at Scotland level between 2017/18 to 2018/19.

5.17 The gap at SCQF Level 5 widened for Challenge Authorities (combined) from 2016/17 to 2017/18 (18.3 to 18.8 percentage points) before decreasing again slightly in 2018/19 (18.5 percentage points) and widening to 19.4 percentage points in 2019/20. A similar pattern is seen at Scotland level.

5.18 In non-Challenge authorities (combined), the gap at SCQF Level 5 has increased slightly year on year between 2016/17 to 2019/20 from 21.7 percentage points in 2016/17, to 22.7 percentage points in 2017/18, 22.9 percentage points in 2018/19 and 23.2 percentage points in 2019/20.

5.19 At SCQF Level 6 or better (Table 5.6), the gap between the proportion of school leavers from the most and least deprived areas that have attained one pass or more narrowed across Challenge Authorities (combined) every year between 2016/17 and 2019/20. The gap declined steadily from 2016/17 (37.9 percentage points), 2017/18 (36.3 percentage points), 2018/19 (35.2 percentage points) to 2019/20 (35.1 percentage points). At Scotland level there was a similar pattern between 2016/17 and 2018/18: the gap narrowed slightly each year from 2016/17 to 2017/18 but widened slightly in 2019/20.

5.20 The gap at SCQF Level 6 widened in non-Challenge Authorities (combined) from 2016/17 to 2017/18 (40.3 to 41.1 percentage points), and subsequently decreased in 2018/19 (39.6 percentage points) and again in 2019/20 (39.5 percentage points).

5.21 At SCQF Level 7, across Challenge Authorities, non-Challenge Authorities and Scotland in 2019/20 there has been an increase of over 4 percentage points in the gap from 2018/19 figures.

5.22 The gap at SCQF Level 7 or better (Table 5.7) narrowed for Challenge Authorities (combined) from 2016/17 to 2017/18 (20.9 to 20.4 percentage points), and decreased again in 2018/19 (18.8 percentage points), rising to 23.4 percentage points in 2019/20. A similar pattern was seen at Scotland level. In non-Challenge Authorities (combined), the gap reduced slightly from 26.8 percentage points in 2016/17, to 26.6 percent in 2017/18 and further to 24.3 percentage points in 2018/19 before increasing to 28.6 percentage points in 2019/20.

5.23 The alternative certification methodology for 2020 National Qualifications should be borne in mind given the attainment of many 2019/20 school leavers presented in this data. Care must be taken when comparing 2019/20 attainment patterns to that of earlier years.

Table 5.5: Challenge Authorities - Percentage of pupils achieving SCQF Level 5, by local authority and deprivation, 2016/17, 2017/18, 2018/19, 2019/20

	Most deprived (bottom 20% SIMD)				Least deprived (top 20% SIMD)				Percentage point gap
	2016/17	2017/18	2018/19	2019/20	2016/17	2017/18	2018/19	2019/20	20/1617	20/1718	20/1819	2019/20
Clackmannanshire	63.6	64.0	64.2	60.1	96.6	95.0	94.5	*	33.0	31.0	30.4	*
Dundee	71.7	65.0	68.0	67.4	94.8	93.0	93.5	92.1	23.2	28.0	25.4	24.7
East Ayrshire	72.1	69.9	64.7	71.8	96.6	94.5	96.2	95.7	24.5	24.6	31.6	23.8
Glasgow	79.6	79.3	79.4	78.0	93.8	94.9	94.8	95.3	14.2	15.5	15.4	17.2
Inverclyde	83.6	83.4	82.3	83.6	94.9	95.4	96.3	*	11.4	12.0	14.1	*
North Ayrshire	76.1	79.8	74.6	77.0	96.1	97.7	95.5	95.3	20.0	17.9	20.9	18.3
North Lanarkshire	76.5	74.7	75.2	73.7	96.1	95.9	93.4	95.3	19.6	21.2	18.2	21.6
Renfrewshire	75.2	79.5	79.5	79.4	96.0	96.8	95.5	97.8	20.9	17.3	16.0	18.4
West Dunbartonshire	80.7	77.4	80.4	78.3	98.1	95.6	94.2	*	17.4	18.2	13.8	*
Challenge Authorities	77.3	76.7	76.2	76.1	95.6	95.5	94.7	95.5	18.3	18.8	18.5	19.4
Non-Challenge Authorities	72.8	72.6	71.6	72.0	94.6	95.3	94.5	95.2	21.7	22.7	22.9	23.2
Scotland	75.5	75.0	74.4	74.5	94.8	95.4	94.6	95.3	19.3	20.3	20.2	20.8

Note: A dotted line break has been placed between the attainment figures for 2018/19 and 2019/20 school leavers reflecting the impact of the change to the assessment approach in 2020 on 2019/20 school leaver attainment.

Table 5.6: Challenge Authorities - Percentage of pupils achieving SCQF Level 6, by local authority and deprivation, 2016/17, 2017/18, 2018/19, 2019/20

	Most deprived (bottom 20% SIMD)				Least deprived (top 20% SIMD)				Percentage point gap
	2016/17	2017/18	2018/19	2019/20	2016/17	2017/18	2018/19	2019/20	20/1617	20/1718	20/1819	2019/20
Clackmannanshire	36.4	33.6	36.6	36.2	77.6	77.5	76.4	79.6	41.2	43.9	39.8	43.2
Dundee	42.3	33.3	39.0	39.3	82.0	75.5	75.7	81.8	39.7	42.2	36.7	42.4
East Ayrshire	38.2	45.3	39.5	41.6	84.2	82.8	80.6	82.0	46.0	37.5	41.2	40.4
Glasgow	48.6	51.3	51.7	52.7	82.2	87.4	83.3	84.5	33.6	36.1	31.6	31.9
Inverclyde	47.9	53.8	53.6	58.8	83.5	89.7	85.4	87.5	35.7	35.8	31.8	28.7
North Ayrshire	40.3	46.0	43.2	46.7	83.8	86.3	81.8	79.8	43.5	40.2	38.6	33.1
North Lanarkshire	43.5	45.0	42.7	48.7	82.8	83.1	83.2	84.6	39.3	38.1	40.5	36.0
Renfrewshire	41.0	47.9	43.6	46.6	84.1	84.4	82.1	87.7	43.1	36.4	38.5	41.1
West Dunbartonshire	53.2	46.9	46.8	48.3	88.5	80.0	88.5	84.6	35.3	33.1	41.6	36.3
Challenge Authorities	45.2	47.2	46.5	49.1	83.1	83.5	81.7	84.2	37.9	36.3	35.2	35.1
Non-Challenge Authorities	39.6	40.3	39.1	42.8	79.9	81.4	78.7	82.3	40.3	41.1	39.6	39.5
Scotland	43.0	44.4	43.5	46.6	80.6	81.8	79.3	82.7	37.6	37.4	35.8	36.1

Note: A dotted line break has been placed between the attainment figures for 2018/19 and 2019/20 school leavers reflecting the impact of the change to the assessment approach in 2020 on 2019/20 school leaver attainment.

Table 5.7: Challenge Authorities - Percentage of pupils achieving SCQF Level 7, by local authority and deprivation, 2016/17, 2017/18, 2018/19/2019/20

	Most deprived (bottom 20% SIMD)				Least deprived (top 20% SIMD)				Percentage point gap
	2016/17	2017/18	2018/19	2019/20	2016/17	2017/18	2018/19	2019/20	20/1617	20/1718	20/1819	2019/20
Clackmannanshire	9.3	6.4	9.0	10.1	22.4	22.5	30.9	35.2	13.1	16.1	22.0	25.0
Dundee	7.7	7.5	8.4	10.7	27.0	18.0	19.6	36.4	19.3	10.5	11.2	25.8
East Ayrshire	6.4	8.6	7.3	8.7	25.4	29.7	27.4	27.3	19.0	21.1	20.1	18.7
Glasgow	9.3	10.4	9.0	12.1	34.2	34.8	29.6	39.1	24.9	24.4	20.6	26.9
Inverclyde	8.8	9.6	7.2	14.8	34.2	37.9	34.1	40.9	25.4	28.3	27.0	26.1
North Ayrshire	9.9	9.0	7.8	13.6	31.2	35.9	27.3	31.8	21.3	26.9	19.5	18.2
North Lanarkshire	8.0	8.1	8.1	10.6	26.6	24.8	21.1	35.1	18.6	16.8	13.0	24.5
Renfrewshire	9.6	9.9	10.4	11.1	31.8	33.7	33.2	34.7	22.3	23.8	22.9	23.6
West Dunbartonshire	11.9	10.6	7.9	13.6	38.5	31.1	34.6	35.9	26.6	20.5	26.7	22.3
Challenge Authorities	9.0	9.4	8.5	11.8	29.8	29.8	27.4	35.2	20.9	20.4	18.8	23.4
Non-Challenge Authorities	7.7	8.8	8.8	11.4	34.5	35.3	33.1	40.0	26.8	26.6	24.3	28.6
Scotland	8.6	9.1	8.7	11.7	33.8	34.7	32.4	39.5	25.3	25.5	23.7	27.8

Note: A dotted line break has been placed between the attainment figures for 2018/19 and 2019/20 school leavers reflecting the impact of the change to the assessment approach in 2020 on 2019/20 school leaver attainment.

Annual Participation Measure

5.24 The Annual Participation Measure (APM) is an agreed measure for considering progress in closing the poverty-related attainment gap, set out in the National Improvement Framework. Since August 2017 the APM has been the source of the National Performance Framework Indicator, ‘Percentage of young adults (16-19 year olds) participating in education, training or employment’. It is produced by Skills Development Scotland (SDS) using a shared data set to report on the economic and employment activity of the wider 16-19 year old cohort, including those at school. A range of partners contribute to the shared dataset, including Local Authorities (schools), colleges, the Department of Work and Pensions (DWP) and SAAS.

5.25 When considering the APM as a measure in the ASF Evaluation it is important to highlight that the ASF funding started in 2015, therefore a lag occurs between intervention at school level and impact on APM (mostly post school). The APM is also measured from 1^st April – 31^st March, as opposed to the academic year considered in the ASF evaluation. As such the figures are not considered across a directly comparable time period.

5.26 This section reports on the participation rate for Scotland overall and at local authority level. For school pupils, local authority relates to the location of the school they are attending. For those who have left school local authority continues to relate to location of their school up to 12 months after leaving, and thereafter the individual is reported against the local authority linked to their postcode.

5.27 All local authority level data disaggregated by SIMD is available online and therefore the focus in this report is specifically on the Challenge Authorities, who have been involved with the ASF for the longest period of time.

5.28 Table 5.8 below shows the proportion of 16-19 year olds participating in education, training or employment was 92.1% in 2020 which represents an increase of 0.5 percentage points compared to 91.6% in 2019 and is the highest rate since the inception of the APM. The 2019 figure showed a decrease of 0.2 percentage points compared to the 2018 figure (91.8%). Previous figures were in 2017 (91.1%) and 2016 (90.4%).

5.29 The participation gap between those who lived in the 20% most deprived areas and those in the 20% least deprived areas continues to show narrowing with a gap of 9.9 percentage points in 2020 (compared to 12.9 percentage points in 2016,11.5 percentage points in 2017, 10.8 percentage points in 2018 and 10.5 percentage points in 2019).

5.30 Looking specifically at the participation rate in the Challenge Authorities, eight of the nine authorities recorded an increase between 2017 and 2020, with the exception of Clackmannanshire whose participation rate has fluctuated between years, peaking at 90.1 per cent at 2019.

Table 5.8: APM - Challenge Authorities Percentage of 16-19 year olds participating, by local authority, 2017 - 2020

	Participation rate				Percentage point change between 2017 and 2020
	2017	2018	2019	2020
Clackmannanshire	89.7	89.3	90.1	89.3	-0.4
Dundee	87.6	88.7	88.2	90	2.4
East Ayrshire	88.1	89.3	88.7	90	1.9
Glasgow	88.2	88.8	89.2	89.5	1.3
Inverclyde	91.9	91.6	91.8	92.9	1
North Ayrshire	90.3	91.1	90.5	90.6	0.3
North Lanarkshire	90.2	90.5	90.1	90.7	0.5
Renfrewshire	91.4	91.6	91.7	92.4	1
West Dunbartonshire	88.3	90.1	90.3	90.2	1.9
Challenge Authorities	89.3	89.9	89.9	90.6	1.3
Non-Challenge Authorities	91.9	92.8	92.4	93.5	1.6
Scotland	91.1	91.8	91.6	92.1	0.5

Source: Annual Participation Measure, Skills Development Scotland

5.31 At Scotland level, as shown in Table 5.9, between Year 4 and 5 of the ASF, there was an overall reduction in the participation gap between those living in the most deprived areas compared to those living in the least deprived areas (9.9 percentage points in 2020, 10.5 percentage points in 2019, 10.8 percentage points in 2018, 11.5 percentage points in 2017). The participation rate increased for both those living in the most and least deprived areas; but by a greater extent for those living in the most deprived areas (0.8 percentage points). With regards Challenge Authorities Table 5.9 shows the following:

• In Year 5 of the ASF, four Challenge Authorities had a smaller or similar participation gap compared to Scotland
• Between Year 4 and 5 of the ASF, the participation gap narrowed in seven Challenge Authorities
• In Year 5 of the ASF, the participation rate for those living in the 20% most deprived areas was higher or similar in four Challenge Authorities, compared to Scotland
• Between Year 4 and 5 of the ASF, the participation rate for those living in the 20% most deprived areas increased or was maintained in six Challenge Authorities

Table 5.9: APM - Challenge Authorities percentage of 16-19 year olds participating, by local authority and deprivation, 2017, 2018, 2019, 2020

	Most deprived (bottom 20% SIMD)				Least deprived (top 20% SIMD)				Gap Percentage points
	2017	2018	2019	2020	2017	2018	2019	2020	2017	2018	2019	2020
Clackmannanshire	82.0	81.6	84.5	83.9	95.6	93.9	96.6	96.3	13.6	12.3	12.1	12.4
Dundee	82.4	82.9	83.4	85.4	96.2	95.6	96	96.2	13.8	12.7	12.6	10.8
East Ayrshire	82.7	83	83.1	84.8	94.2	97.2	96.2	96.3	11.5	14.1	13.1	11.5
Glasgow	85.5	86.1	86.3	86.8	96.3	96.1	95.5	95.6	10.9	10.0	9.2	8.8
Inverclyde	88.2	87.4	88.3	89.9	96.4	96.3	96.3	96.3	8.2	8.9	8.1*	6.4
North Ayrshire	85.9	87.6	87.8	86.9	97.5	96.4	94.7	96.7	11.6	8.8	6.9	9.8
North Lanarkshire	85.0	85.6	84.3	85.9	96.2	95.8	96.7	96.5	11.2	10.2	12.4	10.6
Renfrewshire	85.5	86.6	85.1	87.3	96.5	96.9	97.3	97.6	11.0	10.3	12.1*	10.3
West Dunbartonshire	84.8	85.7	85.7	85.4	95.5	96.9	98.9	95.3	10.7	11.2	13.2	9.9
Scotland	84.8	85.7	85.8	86.6	96.3	96.5	96.3	96.5	11.5	10.8	10.5	9.9

Source: Annual Participation Measure, Skills Development Scotland (SDS). Note that 2016-19 APM uses the SIMD 2016 whereas the 2020 APM uses the SIMD 2020.

*SDS published figures used which are calculated with absolute percentages. In the 2020 publication SDS have altered how the PP differences within the resources are calculated. They now use percentages calculated rounded to one decimal place rather than absolute percentages, this means that in the 2020 publication the pp difference will match the figures presented in charts and text.

Health and wellbeing

5.32 Data on Attendance and Exclusions is not available for consideration in the ASF Year 5 Report as it is published on a biennial basis. Information on health and wellbeing is available from the Scottish Health Survey and the Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS). The findings are based on two variables: scores on the Strengths and Difficulties Questionnaire (SDQ) which measures emotional and behaviour problems, and the Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS). Mental wellbeing is measures using WEMWBS questionnaire and is used as a sub measure to report progress around Health and Wellbeing in the ASF Evaluation.

Total Difficulties Score

5.33 The social, emotional and behavioural development of children has been measured via the Strengths and Difficulties Questionnaire (SDQ). The SDQ is a brief behavioural screening questionnaire designed for use with the 3-16 age group.

5.34 The SDQ comprises 25 questions covering themes such as consideration, hyperactivity, malaise, mood, sociability, obedience, anxiety and unhappiness. It is used to measure five aspects of development: emotional symptoms; conduct problems; hyperactivity/ inattention; peer relationship problems; and pro-social behaviour.

5.35 A score was calculated for each of the five aspects, as well as an overall ‘total difficulties’ score which was generated by summing the scores from all the domains, except pro-social behaviour. The total difficulties score ranged from 0 to 40 with a higher score indicating greater evidence of difficulties. There are established thresholds indicating ‘normal’ (score of 13 or less), ‘borderline’ (14-16) or ‘abnormal’ scores (17 or above).

5.36 Regardless of age, children in the most deprived areas were more likely to have a borderline or abnormal total difficulties score. This is summarised in Table 5.10 and further detail is given in the paragraphs that follow.

Table 5.10: Total Difficulties Score – By Deprivation

	Year	Most disadvantaged (bottom 20% SIMD)	Least disadvantaged (top 20% SIMD)	Gap
		%	%	Percentage points
Total difficulties score (aged 4-12)	2012/15	22	6	16
	2014/17	22	10	12
	2015/18	25	10	15
	2016/19	25	9	16
Total difficulties score (aged 13 & 15)	2015	34	26	8
	2018	42	34	8

Children aged 4-12 years old

5.37 The social, emotional and behavioural development of children aged 4-12 has been measured in the Scottish Health Survey via the SDQ. In the Scottish Health Survey, the SDQ was completed by a parent on behalf of all children aged 4-12.

5.38 Children in the most deprived areas were more likely to have a borderline or abnormal total difficulties score (25 per cent) than those in the least deprived (9 per cent) in 2016/19. Whilst the gap of 16 percentage points initially decreased (12 percentage points in 2014/17 and 15 percentage points in 2015/18), it returned to 16 percentage points in 2016/19.

Children aged 13 and 15 years old

5.39 The social, emotional and behavioural development of children aged 13 and 15 was measured using the same approach, that is the Strengths and Difficulties Questionnaire (SDQ). The data collection used was the Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS), which allows for greater sample size amongst the year groups of interest. Pupils complete the survey themselves.

5.40 Children in the most deprived areas were more likely to have a borderline or abnormal total difficulties score (42 per cent) than those in the least deprived (34 per cent) in 2018. The gap was also 8 percentage points in 2015 (the proportion of children aged 13 and 15 with a borderline or abnormal total difficulties score in the most and least deprived areas both increased and by the same amount.

Health and wellbeing sub-measures

Mental wellbeing score

5.41 Mental wellbeing is measured using the Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS) questionnaire and is used as a sub measure to report progress around Health and Wellbeing.

5.42 While the SDQ measures emotional and behavioural problems, WEMWBS measures mental wellbeing – for example how good a pupil is feeling or how well they think they are coping in their life. In the WEMWBS scale, the lowest score possible (indicating poor mental wellbeing) is 14 and the highest is 70 (indicating good mental wellbeing), so a higher average score for any particular group indicates higher mental wellbeing.

5.43 Overall, mental wellbeing among 13 to 15 year olds decreased with age for all children. Mental wellbeing recorded higher levels for 13 to 15 year old boys than for girls. The figure below presents data by year group and gender.

5.44 Mental wellbeing showed a correlation with areas of deprivation. Overall, pupils in the least deprived areas had a higher WEMWBS mean score indicating better mental wellbeing than those in the most deprived areas. Table 5.11 shows the mental wellbeing score by those most and least deprived and displays the gap between the two.

Table 5.11: Mental Wellbeing mean score – By Deprivation, 2018

	All children	Most disadvantaged (bottom 20% SIMD)	Least disadvantaged (top 20% SIMD)	Gap
Mental Wellbeing Score (13 year old boys)	50.0	48.4	51.2	2.8
Mental Wellbeing Score (13 year old girls)	46.3	45.0	47.0	2.0
Mental Wellbeing Score (15 year old boys)	48.3	48.0	48.7	0.7
Mental Wellbeing Score (15 year old girls)	43.3	41.9	43.9	2.0

Reported evidence of impact

Perceptions of improvement

5.45 Reported evidence of impact emerged from a variety of evidence sources.

5.46 Challenge Authority progress reports featured reports of local impact, particularly with regard to health and wellbeing.

5.47 There were acknowledgements of the ongoing difficulties with measuring progress on health and wellbeing at the local level and that this will continue to be affected through the pausing of the HWB census rollout.

5.48 However, there was also an emerging narrative within Challenge Authority progress reports in terms of the impact of approaches, interventions and programmes on ‘soft indicators’ of health and wellbeing, with evidence cited of, for example, improved readiness to learn, decreased disruption in the classroom, and improved social and emotional competence. Improvements were also indicated in terms of improved practitioner confidence, knowledge and practice in supporting health and wellbeing, as a result both of professional learning and strategic focus on health and wellbeing in policy and guidance. Pupil voice was also noted as a further form of impact evidence in terms of improvements in health and wellbeing. A number of Challenge Authorities described data gathered on such measures as social and emotional competence, reduced disruptive behaviours, improved empathy and problem solving skills.

5.49 At the school level, positive perceptions of success in meeting long term outcomes amongst headteachers continued as a theme for Year 5. The large majority (90%) of schools responding to the Headteacher Survey 2020 reported seeing an improvement in closing the poverty-related attainment gap as a result of ASF supported approaches, including 20% who perceived ‘a lot’ of improvement to date. There was some variation amongst respondent groups however, with schools in receipt of PEF only less likely to have seen ‘a lot’ of improvement to date than schools in receipt of Challenge Authority or Schools Programme funding in addition to PEF. The Headteacher Survey 2019 finding on this measure was very similar (91% of headteachers indicated they had seen such an improvement).

5.50 A similar proportion of headteachers (88%) reported that they expected to see improvements in closing the poverty-related attainment gap over the next few years. The evidence suggests future expectations have declined to some extent, with a ten percentage point decline on this measure from the Headteacher Survey 2019 (from 98%). This finding may be related to some extent to schools who have already made good progress not expecting to see further progress (due to the gap being virtually closed from their perspective). It may also reflect a change in expectations due to the impact of the COVID-19 pandemic and resulting school building closures. However, there was some correlation between improvement to date and future expectations, with 67% of respondents who had seen ‘a lot’ of improvement to date having expectations of future improvements, whereas 11% of respondents who had only seen ‘a little’ improvement to date had expectations of future improvements.

Influencing Factors

5.51 Evidence on the factors that local authorities and schools thought influenced success in progress towards closing the poverty-related attainment gap were gathered from the Headteacher Survey 2020 and the Local Authority Survey 2020. Increased capacity, culture/ethos, collaborative working, increasing skills and knowledge in using data and evidence, and increased levels of understanding of poverty were identified as factors supporting progress from the perspective of local authority respondents. However, it was also recognised by several respondents that further progress was required - for example in increasing data and evidence skills and use, and understanding of poverty - across all levels of the system.

5.52 Factors hindering progress were also raised. Whilst the specific influence of COVID-19 is addressed in the following section, a number of other factors were suggested by some local authority respondents, including staffing issues (such as staff turnover), issues associated with geography/rurality, and short-term funding cycles. There was also a recognition of the level and extent of poverty influencing progress.

5.53 At the school level, findings from the Headteacher Survey 2020 continue to suggest factors associated with schools’ experiences and positive perceptions of progress in closing the poverty related attainment gap including:

• Change of culture or ethos – evidence of embedded approach to equity; and increased collaborative working;
• Improved understanding of barriers faced by pupils and families affected by socio-demographic disadvantage;
• Improved skills and knowledge in use of data and evidence, and application of data and evidence; and
• Developing approaches in terms of approach to equity overall, and in terms of developing approaches to engaging with families and communities.

COVID-19 as an influencing factor

5.54 Evidence from the Headteacher Survey 2020 suggests headteachers perceived progress in closing the poverty-related attainment gap had been impacted by school building closures as a result of COVID-19. Almost two-thirds (61%) of respondents perceived that school building closures resulting from COVID-19 had significantly impacted on their progress, with nearly all (95%) viewing an impact to some extent. There was some variation across key respondent groups, with headteachers from secondary schools and those with middle to higher PEF allocations more likely to feel significant impact.

5.55 There was also some evidence of correlation between perceptions of impact of COVID-19 on progress and perceptions of improvement to date in terms of closing the poverty-related attainment gap, with those who perceived a significant impact of COVID-19 on their progress also less likely to have seen a lot of improvement to date.

5.56 In terms of local authority perceptions of the impact of COVID-19 on progress towards closing the poverty-related attainment gap, all respondents to the Local Authority Survey 2020 were of the view that COVID-19 had impacted on progress towards closing the poverty-related attainment gap, albeit to varying degrees (five indicated this to a great extent, eight to some extent, and two to a limited extent). A number pointed to emerging local evidence of the impact of COVID-19 widening the gap between most and least affluent pupils at the local authority level.

5.57 A number of areas of particular concern were highlighted by Local Authority Survey 2020 respondents. This included the potential for rates of recovery to be slower for pupils affected by poverty, despite a recognition that the school building closures affected all pupils. Concern regarding ongoing impact on pupils at key transition points, potentially for a long time duration, was also highlighted, with one local authority respondent indicating initial data suggested this was indeed a factor.

Additionality

5.58 In terms of the extent to which ASF was seen as additional, there continued to be little direct evidence gathered in terms of perceptions of additionality apart from the inclusion of a specific question in the Headteacher Survey. At the school level, headteacher perceptions remain positive regarding additionality as a result of PEF resource. The great majority (89%) of headteachers viewed PEF as having provided additional resource to address the poverty-related attainment gap, broadly reflecting the 2019 survey findings.

Sustainability

5.59 Sustained and embedded practices related to addressing the impact of the poverty-related attainment gap remains one of the high level long term outcomes for SAC.

5.60 The section below considers a number of aspects of sustainability in the context of the ASF. This includes both stakeholder perceptions of the extent to which improvements achieved as a result of ASF are likely to be sustainable beyond the period of funding, and of the extent to which the focus on closing the poverty-related attainment gap is likely to be sustainable beyond the period of funding.

5.61 Considerations of sustainability in the context of COVID-19 were also sought, specifically in terms of whether sustainability remained a factor under consideration during the school building closures, or whether this was replaced by more immediate COVID-19 concerns.

To what extent has sustainability featured in planning and developing approaches to addressing the poverty-related attainment gap?

5.62 Actions to support, encourage and/or plan for sustainability were explored in Local Authority Survey 2020 responses. Reflecting findings of the Local Authority Survey 2019, there continued to be broad statements of ongoing commitment to sustainability and the relevance of culture change and ethos, partnership working, embedding of approaches and capacity building to sustainability. Specific actions were also described, including:

• schools considering the sustainability of interventions when planning and implementing PEF;
• support from Attainment Advisors to specifically consider sustainability within plans;
• professional development opportunities focusing on sustainability.

Sustainability of improvements

5.63 At the school level, headteachers’ reported perceptions of the extent to which improvements achieved as a result of ASF are likely to be sustainable had decreased in 2020 when compared to 2019 results. Approximately one third (34%) of headteachers who responded to the 2020 survey thought improvements would be sustainable, a decline of seven percentage points. (It is however noted that changes to the question structure introduced in the 2020 survey mean that it is not possible to directly compare results between 2019 and 2020).

Sustainability of focus

5.64 At the local authority level, there continue to be broadly positive expectations that the focus on closing the poverty-related attainment gap will be sustainable. The majority of respondents to the Local Authority Survey 2020 (12 of 15) indicated they viewed the focus would be sustainable to a great or to some extent, whilst a further two local authority respondents viewed the focus would be sustainable to only a limited extent.

5.65 Continuation of funding was the primary factor associated with ensuring a sustainability of focus on equity from the perspective of local authorities. Local authorities also made reference made to:

• ongoing professional learning;
• continued focus on understanding the challenges associated with poverty for children and their families (and associated understanding of the need for a focus on equity);
• focus on data and evidence;
• creating a culture of embedding sustainable approaches; and
• focus on collaboration.

5.66 The importance of strategic planning in promoting a sustainable focus was particularly evident from a number of local authority responses. For example, reference was made by one Schools Programme local authority to a requirement for a closing the poverty-related attainment gap strategy within all School Improvement Plans. Another Schools Programme local authority described the inclusion of SAC/ASF priorities within quality and school improvement plans. Structures and mechanisms to support strategic planning were described, including local authority level governance structures and strategies.

5.67 It was broadly recognised that COVID-19 has had, and will continue to, impact on sustainability of focus to some extent. Local authority perspectives reflected on the reprioritisation which took place during the March to June 2020 period of school building closures, including a greater focus on addressing the immediate challenges of responding to COVID-19 and an increased focus on health and wellbeing. However, there was also a broad affirmation of sustained focus on closing the poverty-related attainment gap, as illustrated by the following quote from Local Authority Survey 2020 respondents.

‘As part of the programme management of ASF, sustainability has been an area which has been addressed since the programme started. While Covid-19 affected our ability to deliver aspects of ASF, our focus continues to be on the sustainability of our approach.’ (Challenge Authority respondent)

5.68 The recognition of capacity at the school level to continue focusing on sustainability was however also highlighted, as illustrated below:

‘The focus of sustainability and impact remains as ever a high priority at local authority level. Our concern is the capacity of the headteachers to lead this at present as their focus has turned very much to day to day operational matters and not the sustainability of PEF plans.’ (Schools Programme respondent)

5.69 Headteacher perspectives of the extent to which the focus on equity will be sustainable beyond the years of funding was considerably more positive than their views on sustainability of improvements. Over half (58%) of headteachers who responded to the 2020 survey were of the view that the focus on closing the poverty-related attainment gap will be sustainable, representing a 17 percentage point increase on the 2019 survey. There was some variation across respondent groups on this measure, with headteachers of primary schools less likely to perceive the focus on equity to be sustainable beyond the funding period.