Attainment Scotland Fund evaluation: interim report (year 3)

5. Progress towards Long-term Impact

5.1. This section explores progress towards improvement in attainment and health and wellbeing and a reduction in the gap between pupils from the most and least deprived areas. 

5.2. The first section explores evidence provided by local authorities and schools on the extent to which different types of interventions improved attainment and health and wellbeing, particularly for pupils from the most deprived areas. 

5.3. The second section provides analysis of attainment and health and wellbeing data based on the agreed measures for monitoring progress towards closing the poverty-related attainment gap set out in the National Improvement Framework. 

5.4. The final section explores the sustainability of both interventions and impact of the ASF. 

Evidence on the impact of interventions 

5.5. This section describes reported evidence of progress towards closing the poverty-related attainment gap by schools and local authorities. It also considers the factors that were felt to contribute or hinder any progress. These findings are based on feedback from schools and local authorities and should be read within this context. 

Reported evidence of Impact

5.6. In the headteacher survey, 88% indicated that they had seen an improvement in closing the poverty-related gap in attainment and health & wellbeing. This was a 10 point increase from 2017, when 78% of headteachers said they had seen an improvement (see Figure 5.1).

5.7. Furthermore, 95% indicated that they expected to see an improvement in the next 5 years as a result of interventions supported by the ASF. This included 56% who expected to see ‘a lot’ of improvement and 39% who expected to see ‘a little’ improvement. This pattern of responses was largely consistent across previous waves of the survey. However, it is notable that those receiving the highest 25% of PEF allocations were more likely to report expecting to see ‘a lot’ of improvement than those receiving the lowest 25% of PEF allocations (72% vs 42%). 

Figure 5.1: Perceived improvement in closing the poverty-related attainment gap, headteacher survey

5.8. Other evidence on the perceived impact of the ASF has been drawn from the Challenge Authority progress reports and school case studies. 

5.9. Challenge Authorities were asked to report the findings from analysis of evidence they were using to measure progress towards long-term outcomes. 

5.10. Authorities reported a range of changes in pupils, teachers or families engaged in specific interventions. For example, one authority noted improved wellbeing for pupils attending counselling. Another example was authorities noting that staff were more confident in their teaching practice as the result of training. 

5.11. Others provided evidence of impact more generally within their authority. Five authorities drew on Achievement of Curriculum for Excellence Level (ACEL) data to provide evidence of improvement in numeracy and literacy within their authority. Other sources of evidence collected by authorities included: attendance and exclusion statistics, standardised assessment data, information contained in evidence reports (e.g. Standards and Quality reports, inspection reports and case studies). 

5.12. Whilst Challenge Authorities noted improvements overall in attainment and achievement, it was not always clear the extent to which this related to closing the poverty-related attainment gap. In addition, some statements of impact were not supported by the provision of clear evidence. Statistical evidence in relation to patterns of attainment within and across authorities is provided later in the chapter. 

5.13. Schools participating in the follow-up case studies were asked to note the impact of their interventions on improved outcomes for pupils as well as specific improvements in closing the poverty-related attainment gap. 

5.14. Many of the schools reported that the evidence base around what works to improve attainment and wellbeing in relation to the poverty-related gap is still developing. 

5.15. As a result of their improvement work, case study schools had observed improvements in pupils’ confidence, engagement and emotional wellbeing. These improvements were sometimes accompanied by reported changes in pupils’ literacy and numeracy outcomes. However, schools were of the view that improvements in wellbeing was critical for more long-term improvements in attainment. 

Influencing factors 

5.16. Evidence on the factors that local authorities and schools thought helped interventions to succeed (or not) in closing the poverty-related attainment gap was gathered from the headteacher survey, follow up school case studies and Challenge Authority progress reports.  

5.17. One of the most commonly mentioned factors contributing positively to achieved impacts was an increase in collaborative working and sharing of practice. This was discussed in detail in chapter 4. 

5.18. There was also evidence, from both a local authority and schools perspective, that increased opportunities for staff to develop their skills and undertake continued professional development had supported the success of interventions. 

5.19. The evaluation of Years 1 & 2 noted that schools and local authorities found parental engagement challenging. There continued to be evidence that this was an area of difficulty for schools and authorities. However, schools noted that where it had worked, parental engagement had been key to their wider improvements. 

5.20. There were also factors that seemed more specific to either a local authority or schools perspective. Nearly 250 of the headteachers participating in the online survey gave responses in relation to the factors that helped interventions to succeed in closing the poverty-related attainment gap. 

5.21. At the school level, 70%^[2] of these headteachers participating in the online survey indicated that the increase in the number of teachers and other staff was an important factor in contributing to the success of the interventions. More dedicated staff time was also mentioned by schools participating in the follow up case studies. 

5.22. For Challenge Authorities, more effective use of data seemed to have been particularly important to driving forward improvements. 

5.23. Other factors reported by headteachers in the online survey included:

• Teacher training and skills development
• Resources and funding
• Family engagement
• Shared set of objectives, collective focus
• Use of data and evidence
• Effective targeting of support 

5.24. The most common factor noted by both Challenge Authorities and schools that had the potential to hinder the success of the ASF was staffing issues. 

5.25. At the school level, 51% of the 247 headteachers responding to this question noted that staffing, staff time and workload was a potential barrier to success. Similarly, evidence from the case studies suggested that both recruitment and staff cover for training had been particularly difficult. 

5.26. Challenge Authorities also noted recruitment difficulties, particularly for specialised posts, and issues with regard to managing both staff cover and turnover. 

5.27. Several of the schools involved in the case studies reflected that the potential impact of the fund was somewhat limited within the context of a wider reduction in funding for services or resources previously available. A few schools questioned the extent to which the ASF was ‘additional’. 

5.28. For Challenge Authorities, there was some evidence of difficulties in ensuring a consistent approach where all staff and schools are committed to the authority’s approach.

5.29. Other less frequently mentioned factors hindering the success of interventions reported by headteachers included:

• The nature and level of pupils’ needs
• Wider community issues, including pupils’ home situation
• Organisational issues, including reporting and paperwork requirements
• Difficulties ensuring effective targeting
• Teacher training and skills development 

Evidence of impact: attainment and wellbeing 

5.30. The measures used to assess improvement in literacy, numeracy and health and wellbeing and the poverty-related attainment gap are in line with those reported in the 2018 National Improvement Framework Evidence Dashboard. Measures with available data for this reporting period are shown in Table 5.1. 

5.31. All of the measures are available at Scotland and local authority level. For the purposes of this report, we focus on patterns of attainment in Challenge Authorities, who have been involved with the ASF since 2015. Where appropriate, we report on differences between Challenge Authorities and non-Challenge Authorities.

5.32. We will continue to consider what additional analysis might be appropriate to explore patterns of attainment and how the poverty-related attainment gap varies according to different characteristics. 

Table 5.1: Measures of progress towards long-term outcomes

	Measure	Age group(s)	Pre ASF 2014/15	Years reported
				Year 1 (2015/16)	Year 2 (2016/17)	Year 3 (2017/18)
Attainment	Achievement of Curriculum for Excellence Levels	P1, P4 and P7 S3			a	✓
	SQA Qualifications – SCQF Level 5 and 6 or better	School leavers		✓	✓
	Participation Measure	16-19 year olds			✓	✓
Health & Wellbeing	Attendance rates	Primary Secondary	✓		✓
	Exclusion rates	Primary  Secondary	✓		✓

^a Data is reported in the previously published interim evaluation of Years 1 and 2

Primary school attainment 

5.33. This section describes the attainment of P1, P4 and P7 pupils in Literacy and Numeracy, using Achievement of Curriculum for Excellence Levels (ACEL). 

5.34. ACEL data is provided for Year 3 (2017/18) of the ASF and focusses on progress within Challenge Authorities. The data continues to be under development and caution should be applied when making comparisons between local authorities and years.

Primary attainment and the poverty-related gap – Literacy 

5.35. Literacy attainment is defined in this section by combining scores across the 3 curriculum organisers (Reading, Writing, Listening & Talking). A combined score for pupils at Primary 1, 4 and 7 is reported^[3]. This is in line with the finalised basket of key measures set out in the National Improvement Framework. 

5.36. Table 5.2 shows the percentage of primary pupils achieving expected levels in literacy across Scotland in each of the Challenge Authorities.  

5.37. In Year 3 of the ASF, four of the nine Challenge Authorities reported a higher percentage of their primary pupils achieving expected levels in literacy compared to Scotland overall. 

5.38. At an overall level, Challenge Authorities reported a lower percentage of primary pupils achieving expected levels in literacy compared to Scotland. 

Table 5.2: Percentage of Primary Pupils achieving expected levels in Literacy, by Challenge Authority, 2017/18

Local Authority
Clackmannanshire	72.1
Dundee	65.5
East Ayrshire	58.6
Glasgow	68.8
Inverclyde	73.5
North Ayrshire	72.4
North Lanarkshire	69.0
Renfrewshire	76.4
West Dunbartonshire	66.8
Challenge Authorities	69.1
Non-Challenge Authorities	72.5
Scotland	71.4

Source: Achievement of Curriculum of Excellence Levels, Scottish Government 

5.39. Table 5.3 shows the percentage of primary pupils from the 20% most and 20% least deprived areas achieving expected levels in literacy across the Challenge Authorities, and considers performance in Challenge and non-Challenge Authorities. 

5.40. In relation to the attainment gap in literacy for primary pupils in 2017/18: 

• 6 Challenge Authorities had a smaller attainment gap compared to Scotland. 
• 5 Challenge Authorities recorded a higher percentage of primary pupils from the most deprived areas achieving expected levels, compared to Scotland as a whole

Table 5.3: Percentage of Primary Pupils achieving expected levels in Literacy, by deprivation and Challenge Authority, 2017/18

	Most deprived (bottom 20% SIMD) %	Least deprived (top 20% SIMD) %	Gap  Percentage points
Clackmannanshire	63.5	79.5	15.9
Dundee	59.3	80.1	20.8
East Ayrshire	47.8	77.6	29.8
Glasgow	65.0	85.1	20.1
Inverclyde	62.8	89.3	26.5
North Ayrshire	65.7	83.3	17.6
North Lanarkshire	60.0	83.0	23.0
Renfrewshire	68.1	85.7	17.6
West Dunbartonshire	61.2	82.6	21.4
Challenge Authorities	62.6	83.3	20.7
Non-Challenge Authorities	60.9	83.5	22.5
Scotland	62	83.4	21.4

Source: Achievement of Curriculum of Excellence Levels, Scottish Government 

5.41. Table 5.3 also shows how Challenge and non-Challenge Authorities performed as a group, compared to Scotland overall. 

5.42. Overall, in Year 3, the attainment gap in literacy for primary pupils was smaller in Challenge Authorities compared to the average at both national level and in non-Challenge Authorities. This is consistent with findings from Year 2, published in the previous interim report. 

5.43. In Year 3, a higher percentage of primary pupils from the most deprived areas achieved expected levels in literacy in Challenge Authorities than in non-Challenge Authorities. This is also consistent with the pattern in Year 2. 

5.44. There was little difference in the performance of pupils living in the least deprived areas in Year 3 (2017/18); around 83% of these pupils achieved expected levels in literacy in Challenge and non-Challenge Authorities and at national level. 

5.45. Given that these statistics are still badged as experimental, it is important to consider differences in approaches to assessment across authorities and over time. 

Primary attainment and the poverty-related gap – Numeracy 

5.46. This section reports on the numeracy attainment of primary pupils and again provides a combined score for P1, P4 and P7 pupils. 

5.47. Table 5.4 shows that in Year 3, three of the nine Challenge Authorities reported a higher percentage of their primary pupils achieving expected levels in numeracy, compared to Scotland overall. 

5.48. At an overall level, Challenge Authorities reported a lower percentage of primary pupils achieving expected levels in numeracy compared to Scotland.

Table 5.4: Percentage of Primary Pupils achieving expected levels in Numeracy, by Challenge Authority, 2017/18

Local Authority
Clackmannanshire	77.0
Dundee	73.8
East Ayrshire	68.0
Glasgow	77.8
Inverclyde	80.0
North Ayrshire	79.5
North Lanarkshire	75.8
Renfrewshire	82.8
West Dunbartonshire	74.0
Challenge Authorities	76.8
Non-Challenge Authorities	79.1
Scotland	78.4

Source: Achievement of Curriculum of Excellence Levels, Scottish Government 

5.49. Table 5.5 shows the percentage of primary pupils from the 20% most and 20% least deprived areas achieving expected levels in numeracy across the Challenge Authorities, and considers performance in Challenge and non-Challenge Authorities.

5.50. In relation to the attainment gap in numeracy for primary pupils in 2017/18: 

• 5 Challenge Authorities had a smaller attainment gap compared to Scotland
• 3 Challenge Authorities recorded a higher percentage of primary pupils from the most deprived areas achieving expected levels, compared to Scotland as a whole

Table 5.5: Percentage of Primary Pupils achieving expected levels in Numeracy, by deprivation and Challenge Authority, 2017/18

	Most disadvantaged (bottom 20% SIMD) %	Least disadvantaged (top 20% SIMD) %	Gap  Percentage points
Clackmannanshire	69.4	82.9	13.5
Dundee	69.1	85.4	16.3
East Ayrshire	59.3	81.7	22.4
Glasgow	75.4	89.3	13.9
Inverclyde	70.8	94.9	24.0
North Ayrshire	74.8	88.8	14.0
North Lanarkshire	68.6	87.9	19.2
Renfrewshire	75.5	90.5	15.0
West Dunbartonshire	68.3	87.2	18.8
Challenge Authorities	72.0	88.1	16.0
Non-Challenge Authorities	70.1	87.9	17.8
Scotland	71.3	87.9	16.6

Source: Achievement of Curriculum of Excellence Levels, Scottish Government 

5.51. Table 5.5 also shows numeracy attainment for primary pupils at an aggregate level in Challenge and non-Challenge Authorities.  

5.52. Similar to literacy, a higher percentage of pupils from the most deprived areas achieved expected levels in numeracy in Challenge Authorities than in non-Challenge Authorities. This is again consistent with the pattern reported in Year 2. 

5.53. The attainment gap in numeracy for primary pupils was smaller in Challenge Authorities than in non-Challenge Authorities. This is consistent with the pattern in Year 2. 

Secondary school attainment 

5.54. This section provides data on the percentage of S3 pupils achieving expected levels in Literacy and Numeracy, using Achievement of Curriculum for Excellence Levels (ACEL).

5.55. Data is provided for Challenge Authorities and considers progress in Year 3 (2017/18). However, as stated earlier, as the data continues to be under development, caution should be applied when making comparisons between local authorities.

Secondary attainment and the poverty-related gap – Literacy 

5.56. In Year 3, three of the nine Challenge Authorities reported a higher percentage of S3 pupils achieving expected levels in literacy, compared to Scotland as a whole. 

5.57. Overall, Challenge Authorities reported a similar percentage of S3 pupils achieving expected levels in literacy compared to Scotland. 

Table 5.6: Percentage of S3 Pupils achieving Third Level or better in Literacy, by Challenge Authority, 2017/18

Local Authority
Clackmannanshire	83.4
Dundee	84.2
East Ayrshire	82.6
Glasgow	86.4
Inverclyde	90.5
North Ayrshire	90.8
North Lanarkshire	86.7
Renfrewshire	93.7
West Dunbartonshire	83.4
Challenge Authorities	87.1
Non-Challenge Authorities	87.4
Scotland	87.3

Source: Achievement of Curriculum of Excellence Levels, Scottish Government 

5.58. Table 5.7 shows the percentage of S3 pupils from the 20% most and 20% least deprived areas achieving expected levels in literacy across the Challenge Authorities, and considers performance in Challenge and non-Challenge Authorities. 

5.59. Disclosure control limits the reporting for relatively small authorities. Of the five Challenge Authorities with available data one had a smaller gap compared to Scotland. 

5.60. Focussing on pupils from the most deprived areas, 5 of the 9 Challenge Authorities recorded a higher percentage of S3 pupils from the most deprived areas achieving expected levels, compared to Scotland as a whole

Table 5.7: Percentage of S3 pupils achieving Third Level or better in Literacy, by deprivation and Challenge Authority, 2017/18

	Most disadvantaged (bottom 20% SIMD) %	Least disadvantaged (top 20% SIMD)%	Gap  Percentage points
Clackmannanshire	76.7	89-100	-
Dundee	77.1	94.9	17.8
East Ayrshire	73.7	89.3	15.7
Glasgow	83.6	96.6	13.1
Inverclyde	87.6	93-100	-
North Ayrshire	90.8	96-100	-
North Lanarkshire	82.0	95.9	13.9
Renfrewshire	90.3	96.7	6.4
West Dunbartonshire	78.4	90-100	-
Challenge Authorities	83.0	95.7	12.8
Non-Challenge Authorities	79.0	94.4	15.4
Scotland	81.4	94.6	13.2

Source: Achievement of Curriculum of Excellence Levels, Scottish Government 

5.61. Table 5.7 also shows the literacy attainment for S3 pupils at an aggregate level in Challenge and non-Challenge Authorities.  

5.62. In line with the findings from Year 2, a higher percentage of S3 pupils from the most deprived areas achieved Third Level or better in literacy in Challenge Authorities than in non-Challenge Authorities. 

5.63. The attainment gap in literacy for S3 pupils was also smaller in Challenge Authorities than in non-Challenge Authorities. This is also consistent with the pattern in Year 2.

Secondary attainment and the poverty-related gap – Numeracy

5.64. Table 5.8 shows four Challenge Authorities reported a higher percentage of S3 pupils achieving expected levels, compared to Scotland overall. 

5.65. Overall, Challenge Authorities reported a lower percentage of S3 pupils achieving expected levels in numeracy compared to Scotland overall. 

Table 5.8: Percentage of S3 Pupils achieving Third Level or better in Numeracy, by Challenge Authority,  2017/18

Local Authority
Clackmannanshire	80.4
Dundee	80.5
East Ayrshire	90.5
Glasgow	84.3
Inverclyde	85.6
North Ayrshire	91.6
North Lanarkshire	90.1
Renfrewshire	92.9
West Dunbartonshire	82.6
Challenge Authorities	87.2
Non-Challenge Authorities	89.8
Scotland	89.0

Source: Achievement of Curriculum of Excellence Levels, Scottish Government 

5.66. Table 5.9 shows the attainment gap in numeracy for S3 pupils, across Challenge Authorities in Year 3 (2017/18) of the ASF.

5.67. In relation to the numeracy attainment of S3 pupils from the most deprived areas, five Challenge Authorities reported a higher percentage of pupils achieving expected levels, compared to Scotland overall. 

5.68. As with literacy results, disclosure control checks limit our ability to consider performance within small Challenge Authorities. Of the four authorities with available data, three had a smaller attainment gap than Scotland overall. 

Table 5.9: Percentage of S3 pupils achieving Third Level or better in Numeracy, by deprivation and Challenge Authority, 2017/18

	Most disadvantaged (bottom 20% SIMD)%	Least disadvantaged (top 20% SIMD)%	Gap Percentage points
Clackmannanshire	76.0	89-100	-
Dundee	75.6	87.1	11.5
East Ayrshire	84.8	96-100	-
Glasgow	80.5	96.7	16.2
Inverclyde	82.4	93-100	-
North Ayrshire	91.7	96-100	-
North Lanarkshire	85.0	96.6	11.5
Renfrewshire	87.1	98.3	11.2
West Dunbartonshire	74.7	90-100	-
Challenge Authorities	82.2	95.8	13.5
Non-Challenge Authorities	81.1	95.7	14.5
Scotland	81.8	95.7	13.9

Source: Achievement of Curriculum of Excellence Levels, Scottish Government 

5.69. Table 5.9 also shows the numeracy attainment for S3 pupils at an aggregate level in Challenge and non-Challenge Authorities. 

5.70. In Year 3, a higher percentage of S3 pupils from most deprived areas were achieving expected levels in Challenge Authorities, compared to those in non-Challenge Authorities. This is consistent with findings for Year 2. 

5.71. The attainment gap in Year 3 was smaller in Challenge Authorities than in non-Challenge Authorities. This is also consistent with findings from Year 2. 

Senior Phase Attainment

5.72. The first interim evaluation report provided detailed analysis of 2015/16 school leaver attainment data. 

5.73. This section reports on the percentage of school leavers achieving awards by SCQF levels in 2017/18 – Year 3 of the ASF. Progress in attainment over time is considered from Year 2 (2016/17) to Year 3 (2017/18) of the ASF. Analysis of the data in this report focuses on progress in Challenge Authorities.   

5.74. Overall, 85.9% of leavers achieved 1+ award at SCQF Level 5 or better and 62.2% at SCQF Level 6 or better in 2017/18. Figure 5.2 and Figure 5.3 show this data at local authority level, and how it compares with attainment in 2016/17.  

5.75. The percentage of school leavers achieving at least 1 award at SCQF Level 5 decreased slightly over time. The picture was mixed across Challenge Authorities with five authorities showing a decline over time. 

5.76. At SCQF Level 6 or better, the percentage of school leavers achieving at least 1 award increased slightly over time. The picture was again mixed across Challenge Authorities with five authorities showing improvement over time. 

5.77. As with previous years, the attainment gap between school leavers from the 20% most and least deprived areas was wider at SCQF Level 6 or better than Level 5 or better (see Figure 5.4)  

5.78. Between 2016/17 and 2017/18, the attainment gap widened at SCQF Level 5 or better and narrowed slightly at Level 6 or better. 

Figure 5.2: Percentage of leavers achieving 1+ award at SCQF Level 5 or better, 2016/17 & 2017/18

Figure 5.3: Percentage of leavers achieving 1+ award at SCQF Level 6 or better, 2016/17 & 2017/18

Source: School Leaver Destinations and Attainment, Scottish Government 

Figure 5.4: Percentage of school leavers attaining 1+ SCQF awards, by deprivation, 2016/17 & 2017/18

Source: School Leaver Destinations and Attainment, Scottish Government 

Across the Challenge Authorities, the attainment gap at SCQF Level 5 or better between Year 2 and 3 of the ASF (see Table 5.10):

• Increased in five local authorities: Dundee, Glasgow, Inverclyde, North Lanarkshire and West Dunbartonshire.
• Decreased in three local authorities: Clackmannanshire, North Ayrshire and Renfrewshire.
• Remained stable over time in East Ayrshire. 

5.79. However, there were variations across Challenge Authorities in the reasons underlying changes in the attainment gap. For two of the three Challenge Authorities that showed a narrowing of the gap, this was because the increase in the proportion of leavers from the most deprived areas attaining one pass or more at SCQF Level 5 was greater than the increase in the proportion of leavers from the least deprived areas attaining this. 

Table 5.10: Percentage of leavers attaining 1+ awards at SCQF Level 5 or better, by Challenge Authority and deprivation, 2016/17 & 2017/18

	Most deprived (bottom 20% SIMD) %		Least deprived (top 20% SIMD) %		Gap Percentage points
	2016/17	2017/18	2016/17	2017/18	2016/17	2017/18
Clackmannanshire	63.6	64.0	96.6	95.0	33.0	31.0
Dundee	71.7	65.0	94.8	93.0	23.2	28.0
East Ayrshire	72.1	69.9	96.6	94.5	24.5	24.6
Glasgow	79.6	79.3	93.8	94.9	14.2	15.5
Inverclyde	83.6	83.4	94.9	95.4	11.4	12.0
North Ayrshire	76.1	79.8	96.1	97.7	20.0	17.9
North Lanarkshire	76.5	74.7	96.1	95.9	19.6	21.2
Renfrewshire	75.2	79.5	96.0	96.8	20.9	17.3
West Dunbartonshire	80.7	77.4	98.1	95.6	17.4	18.2
Scotland	75.5	75.0	94.8	95.4	19.3	20.3

Source: School Leaver Destinations and Attainment, Scottish Government 

5.80. The first interim evaluation reported that in 2015/16, the attainment gap was narrower in Challenge Authorities than it was at Scotland level or within non-Challenge Authorities. We found that this was due to pupils from the 20% most deprived areas performing better in Challenge Authorities. 

5.81. A similar pattern was found in the analysis of 2016/17 and 2017/18 attainment data. 

5.82. At SCQF Level 5 or better, the attainment gap in Challenge Authorities was smaller than the gap at Scotland level and in non-Challenge Authorities. The attainment gap in non-Challenge Authorities was wider when compared to Scotland. This was the case for both Year 2 and 3 of the ASF consistent with the pattern reported for Year 1. 

5.83. In addition, a higher proportion of pupils from the most deprived areas attained at least 1 award at SCQF Level 5 or better in Challenge Authorities, compared to those in non-Challenge Authorities. This was again the case for both Year 2 and 3 of the ASF and consistent with the pattern reported in the interim evaluation report. 

5.84. Between Year 2 and 3 of the ASF, the attainment gap widened in Challenge Authorities. It also widened in non-Challenge Authorities. 

5.85. However, within Challenge Authorities this seemed to be due to a decrease in the attainment of leavers from most deprived areas. For non-Challenge Authorities, the gap widened due to an increase in the attainment of leavers from the least deprived areas. 

Table 5.11: Percentage of leavers attaining 1+ awards at SCQF Level 5 or better – Challenge and non-Challenge Authorities, by deprivation, 2016/17 & 2017/18

	Most deprived (bottom 20% SIMD) %		Least deprived (top 20% SIMD) %		Gap Percentage points
	2016/17	2017/18	2016/17	2017/18	2016/17	2017/18
Challenge Authorities	77.3	76.7	95.6	95.5	18.3	18.8
Non-Challenge Authorities	72.8	72.6	94.6	95.3	21.7	22.7
Scotland	75.5	75.0	94.8	95.4	19.3	20.3

Source: School Leaver Destinations and Attainment, Scottish Government 

5.86. Table 5.12 provides detail on the proportion of leavers achieving 1 or more award at SCQF Level 6 or better. Across the Challenge Authorities, we found that the attainment gap at SCQF Level 6 or better between 2016/17 and 2017/18:

• Increased in three Challenge Authorities: Clackmannanshire, Dundee and Glasgow. 
• Decreased in five Challenge Authorities: East Ayrshire, North Ayrshire, North Lanarkshire, Renfrewshire and West Dunbartonshire.
• Remained stable over time in Inverclyde.

5.87. However, as with the SCQF Level 5 or better data, there were various reasons for these changes. For three of the five Challenge Authorities that showed a narrowing of the gap, the increase in the attainment of leavers from the most deprived areas was greater than the increase in the attainment of leavers from the least deprived areas.  

Table 5.12: Percentage of leavers attaining 1+ awards at SCQF Level 6 or better, by Challenge Authority and deprivation, 2016/17 & 2017/18

	Most deprived (bottom 20% SIMD) %		Least deprived (top 20% SIMD) %		Gap Percentage points
	2016/17	2017/18	2016/17	2017/18	2016/17	2017/18
Clackmannanshire	36.4	33.6	77.6	77.5	41.2	43.9
Dundee	42.3	33.3	82.0	75.5	39.7	42.2
East Ayrshire	38.2	45.3	84.2	82.8	46.0	37.5
Glasgow	48.6	51.3	82.2	87.4	33.6	36.1
Inverclyde	47.9	53.8	83.5	89.7	35.7	35.8
North Ayrshire	40.3	46.0	83.8	86.3	43.5	40.2
North Lanarkshire	43.5	45.0	82.8	83.1	39.3	38.1
Renfrewshire	41.0	47.9	84.1	84.4	43.1	36.4
West Dunbartonshire	53.2	46.9	88.5	80.0	35.3	33.1
Scotland	43.0	44.4	80.6	81.8	37.6	37.4

Source: School Leaver Destinations and Attainment, Scottish Government 

5.88. At SCQF Level 6 or better, the attainment gap in Challenge Authorities was smaller than the gap in non-Challenge Authorities (see Table 5.13). This was the case for both Year 2 and 3 of the ASF.

5.89. A higher proportion of school leavers from the most deprived areas attained 1 or more award at SCQF Level 6 or better in Challenge Authorities than in non-Challenge Authorities. This was the case for both Year 2 and 3 of the ASF and is consistent with the pattern reported for Year 1.

5.90. The first interim evaluation reported that the difference amongst pupils living in the least deprived areas was less pronounced in 2015/16; a similar proportion in Challenge and non-Challenge Authorities attained 1 or more award at SCQF Level 6 or better. However, in both 2016/17 and 2017/18 a higher proportion of school leavers from the least deprived areas attained 1 or more award at SCQF Level 6 or better in Challenge Authorities than in non-Challenge Authorities. 

5.91. Between Year 2 and 3 of the ASF, the attainment gap narrowed in Challenge Authorities due to the increase in attainment of leavers from most deprived areas being greater than the increase in attainment of leavers from least deprived areas. 

5.92. In contrast, the attainment gap widened in non-Challenge Authorities. This was due to the increase in attainment of leavers from least deprived areas being greater than the increase in the attainment of leavers from the most deprived areas. 

Table 5.13: Percentage of leavers attaining 1+ awards at SCQF Level 6 or better – Challenge and non-Challenge Authorities, by deprivation, 2016/17 & 2017/18

	Most deprived (bottom 20% SIMD) %		Least deprived (top 20% SIMD) %		Gap Percentage points
	2016/17	2017/18	2016/17	2017/18	2016/17	2017/18
Challenge Authorities	45.2	47.2	83.1	83.5	37.9	36.3
Non-Challenge Authorities	39.6	40.3	79.9	81.4	40.3	41.1
Scotland	43.0	44.4	80.6	81.8	37.6	37.4

Source: School Leaver Destinations and Attainment, Scottish Government 

Participation Measure

5.93. The Annual Participation Measure (APM) is another key measure for measuring progress in closing the poverty-related attainment gap, set out in the National Improvement Framework. It is managed by Skills Development Scotland and reports on the economic and employment activity of the 16-19 year old cohort. 

5.94. In this section, we report on the participation rate for Scotland and at local authority level. It is not a school-based measure and therefore it is not possible to distil findings at other levels.  

5.95. All local authority level data disaggregated by SIMD is available online and therefore in this report we focus specifically on the Challenge Authorities, who have been involved with the ASF for the longest period of time. 

5.96. Table 5.14 shows the proportion of 16-19 year olds participating in education, training or employment was 91.8% in 2017/18, an increase of 0.7 percentage points compared to 2016/17. 

5.97. Looking specifically at the participation rate in the Challenge Authorities, seven of the nine recorded an increase between Year 2 and Year 3 of the ASF. 

Table 5.14: Percentage of 16-19 year olds participating, by local authority, 2016/17 & 2017/18

	Participation rate		Percentage point change between 2016/17 and 2017/18
	2016/17	2017/18
Clackmannanshire	89.7	89.3	-0.4
Dundee	87.6	88.7	1.1
East Ayrshire	88.1	89.3	1.2
Glasgow	88.2	88.8	0.6
Inverclyde	91.9	85.7	-6.2
North Ayrshire	90.3	91.1	0.8
North Lanarkshire	90.2	90.5	0.3
Renfrewshire	91.4	91.6	0.2
West Dunbartonshire	88.3	90.1	1.8
Scotland	91.1	91.8	0.7

Source: Annual Participation Measure, Skills Development Scotland

5.98. Between Year 2 and 3 of the ASF, there was an overall reduction in the participation gap between those living in the most deprived areas compared to those living in the least deprived areas (11.5 pp in 2016/17 and 10.8 pp in 2017/18). This was due to increases in the participation rate amongst those living in the 20% most deprived areas. 

5.99. Table 5.15 shows the participation gap over Year 2 and 3 of the ASF, by Challenge Authority. It shows:

• In Year 3 of the ASF, 5 Challenge Authorities had a smaller participation gap compared to Scotland.
• Between Year 2 and 3 of the ASF, the participation gap narrowed in 6 Challenge Authorities. 
• In Year 3 of the ASF, the participation rate for those living in the 20% most deprived areas was higher or similar in 6 Challenge Authorities, compared to Scotland.
• Between Year 2 and 3 of the ASF, the participation rate for those living in the 20% most deprived areas increased or was maintained in 7 Challenge Authorities. 

Table 5.15: Percentage of 16-19 year olds participating, by local authority and deprivation, 2016/17 & 2017/18

	Most deprived (bottom 20% SIMD)		Least deprived (top 20% SIMD)		Gap Percentage points
	2016/17	2017/18	2016/17	2017/18	2016/17	2017/18
Clackmannanshire	82	81.6	95.6	93.9	13.6	12.3
Dundee	82.4	82.9	96.2	95.6	13.8	12.7
East Ayrshire	82.7	83	94.2	97.2	11.5	14.1
Glasgow	85.5	86.1	96.3	96.1	10.9	10
Inverclyde	88.2	87.4	96.4	96.3	8.2	8.9
North Ayrshire	85.9	87.6	97.5	96.4	11.6	8.8
North Lanarkshire	85	85.6	96.2	95.8	11.2	10.2
Renfrewshire	85.5	86.6	96.5	96.9	11	10.3
West Dunbartonshire	84.8	85.7	95.5	96.9	10.7	11.2
Scotland	84.8	85.7	96.3	96.5	11.6	10.8

Source: Annual Participation Measure, Skills Development Scotland

Health and Wellbeing

5.100. This section reports on two health and wellbeing related measures: attendance and exclusions rates. The interim report for Years 1 and 2 also reported on the total difficulties score in the Scottish Health Survey and SALSUS, as well as the Mental Wellbeing Score from SALSUS. However, data from additional waves is not yet available. 

5.101. Information on attendance and exclusion from schools is collected on a biennial basis. In the first interim report, we analysed data for 2014/15 – the year prior to the Attainment Scotland Fund. In this section, we consider how these figures changed between 2014/15 and 2016/17 – the second year of the ASF.

Health and Wellbeing – Attendance rates 

5.102. Figure 5.5 shows the attendance levels for primary and secondary schools, by deprivation and year. 

5.103. Pupils from the most deprived areas had lower attendance rates and the effect was greater in secondary schools.  

5.104. The gap in attendance rates increased over time, with the effect again being greater in secondary schools. Whilst the attendance of pupils from the least deprived areas remained fairly stable over time, the attendance rate of the most deprived pupils decreased. 

Figure 5.5: Total Attendance Rates, by deprivation, 2014/15 & 2016/17

Source: Summary Statistics for Schools, Scottish Government

5.105. Table 5.16 shows the attendance rates for primary pupils, disaggregated by deprivation and Challenge Authority. 

5.106. In relation to the gap in attendance rates for primary pupils:

• In Year 2 of the ASF, 8 Challenge Authorities had a smaller gap compared to Scotland
• Between 2014/15 (pre-ASF) and Year 2 of the ASF, the gap narrowed in 3 Challenge Authorities. 

5.107. The primary attendance rate for pupils from the least deprived areas was fairly consistent across Challenge Authorities (~96%). 

5.108. Attendance rate for pupils in the most deprived areas was somewhat more varied. In Year 2, the attendance rate of pupils from the most deprived areas was higher or similar in 6 Challenge Authorities compared to Scotland overall. 

Table 5.16: Primary Attendance Rates – By local authority and deprivation, 2014/15 & 2016/17 

Primary attendance rates	Most deprived (bottom 20% SIMD) %		Least deprived (top 20% SIMD) %		Gap Percentage points
	2014/15	2016/17	2014/15	2016/17	2014/15	2016/17
Clackmannanshire	93.4	93.7	96.1	96.2	2.8	2.5
Dundee City	93.3	92.9	96.5	96.6	3.3	3.7
East Ayrshire	93.8	93.6	96.5	96.6	2.7	3.0
Glasgow City	93.1	92.8	96.8	96.5	3.7	3.7
Inverclyde	93.3	93.2	96.9	96.3	3.6	3.1
North Ayrshire	94.2	93.5	96.3	96.7	2.1	3.2
North Lanarkshire	92.8	92.3	96.7	96.5	3.9	4.2
Renfrewshire	94.2	94.0	97.1	96.8	2.9	2.8
West Dunbartonshire	94.0	93.3	97.1	96.5	3.1	3.2
Scotland	93.3	92.9	96.7	96.7	3.4	3.8

Source: Summary Statistics for Schools, Scottish Government

5.109. Table 5.17 shows the attendance rates for secondary pupils, disaggregated by deprivation and Challenge Authority.

5.110. In relation to the gap in attendance rates for secondary pupils:

• In Year 2 of the ASF, 4 Challenge Authorities had a smaller gap compared to Scotland 
• From the year prior to ASF (2014/15) to Year 2 of the ASF, the gap did not narrow in any Challenge Authorities. 

Table 5.17: Secondary Attendance Rates – By local authority and deprivation, 2014/15 & 2016/17

Secondary attendance rates	Most deprived (bottom 20% SIMD) %		Least deprived (top 20% SIMD) %		Gap Percentage points
	2014/15	2016/17	2014/15	2016/17	2014/15	2016/17
Clackmannanshire	88.2	86.8	94.0	94.4	5.8	7.6
Dundee City	87.3	87.1	93.7	93.9	6.3	6.8
East Ayrshire	87.7	87.0	94.5	93.9	6.8	6.9
Glasgow City	90.1	89.6	95.1	95.0	5.0	5.4
Inverclyde	88.7	87.7	94.5	94.3	5.7	6.6
North Ayrshire	89.1	87.5	93.9	93.0	4.8	5.5
North Lanarkshire	87.8	85.3	94.5	92.7	6.7	7.4
Renfrewshire	87.5	87.1	93.7	93.4	6.2	6.3
West Dunbartonshire	87.3	86.1	93.4	92.6	6.2	6.5
Scotland	88.7	87.7	94.5	94.3	5.8	6.6

Source: Summary Statistics for Schools, Scottish Government

Health and Wellbeing – Exclusion rates 

5.111. Full details on exclusion rates disaggregated by local authority and SIMD are available on the National Improvement Framework interactive dashboard. 

5.112. Overall, exclusion rates were higher for pupils from the most deprived areas compared to those from the least deprived. Secondary schools had a higher exclusion rate than primary schools and the gap in exclusion rates was also higher in secondary schools. 

5.113. Figure 5.6 also shows that over time, the gap in exclusion rates widened for primary pupils and narrowed for secondary pupils. 

Figure 5.6: Total Exclusion Rates per 1000 pupils, by deprivation, 2014/15 & 2016/17

Source: Summary Statistics for Schools, Scottish Government

5.114. Table 5.18 shows the primary exclusion rates for Challenge Authorities, by deprivation and year. Due to disclosure control, it is not possible to offer a full analysis of the gap in relation to exclusion rates. 

5.115. In Year 2 of the ASF, the primary exclusion rate for pupils from the most deprived areas was lower in 6 of the 9 Challenge Authorities, compared to Scotland overall. 

5.116. From the year prior to ASF (2014/15) to Year 2 of the ASF, the primary exclusion rate for pupils from the most deprived areas decreased for 4 Challenge Authorities. 

Table 5.18: Primary Exclusion Rates per 1000 pupils - By local authority and deprivation, 2014/15 & 2016/17

Primary exclusion rates per 1000 pupils	Most deprived (bottom 20% SIMD) %		Least deprived (top 20% SIMD) %		Gap Percentage points
	2014/15	2016/17	2014/15	2016/17	2014/15	2016/17
Clackmannanshire	68.0	49	5.7	*	62.3	-
Dundee City	29.9	20	2.1	*	27.8	-
East Ayrshire	54.2	52	3.8	*	50.4	-
Glasgow City	12.8	18	3.5	4	9.2	14
Inverclyde	3.9	3	0.0	*	3.9	-
North Ayrshire	8.8	9	0.0	*	8.8	-
North Lanarkshire	17.7	20	1.9	3	15.7	17
Renfrewshire	5.3	17	0.4	0	4.9	17
West Dunbartonshire	13.8	23	0	0	13.8	23
Scotland	19.0	22	2.1	3	16.9	19

Source: Summary Statistics for Schools, Scottish Government

5.117. Table 5.19 shows the secondary exclusion rates for Challenge Authorities, by deprivation and year. 

5.118. The gap in secondary exclusion rates narrowed in 5 of the 8 Challenge Authorities between 2014/15 and 2016/17. 

5.119. In Year 2 of the ASF, the secondary exclusion rate for pupils from the most deprived areas was lower in 5 of the 9 Challenge Authorities, compared to Scotland overall. Over time, the secondary exclusion rate for these pupils decreased in 6 Challenge Authorities. 

Table 5.19: Secondary Exclusion Rates per 1000 pupils - By local authority and deprivation, 2014/15 & 2016/17

Secondary exclusion rates per 1000 pupils	Most deprived (bottom 20% SIMD) %		Least deprived (top 20% SIMD) %		Gap Percentage points
	2014/15	2016/17	2014/15	2016/17	2014/15	2016/17
Clackmannanshire	110.3	42	36.7	24	73.5	18
Dundee City	228.1	124	22.8	32	205.3	92
East Ayrshire	131.8	147	26.7	16	105.1	131
Glasgow City	78.3	60	7.6	19	70.7	41
Inverclyde	61.2	55	9.6	21	51.6	34
North Ayrshire	66.4	57	19.9	*	46.5	-
North Lanarkshire	95.9	86	18.0	20	78.0	66
Renfrewshire	56.0	79	9.6	19	46.4	60
West Dunbartonshire	81.0	133	23.3	30	57.7	103
Scotland	95.2	85	15.1	18	80.1	67

Source: Summary Statistics for Schools, Scottish Government

Sustainability

5.120. This section explores issues related to sustainability of both interventions and impact beyond the years of the ASF. 

5.121. Evidence for this section comes from the local authority mini survey conducted in spring/summer 2018, Challenge Authority progress reports, headteacher survey and school case studies. 

5.122. As part of the mini survey, local authorities were asked whether they would expect the different improvements achieved as a result of the ASF to be sustainable. Overall, there was a positive outlook in terms of sustainability across both Challenge and non-Challenge Authorities, with a majority viewing improvements to be sustainable. Only 3 of the 22 reported they did not expect improvements to be sustainable. 

5.123. Key areas identified as reasons to be confident in sustainability were:

• Changes in culture/ethos/focus. Challenge Authorities in particular indicated an increased understanding of the impact of poverty on attainment and a strong commitment to close the poverty-related attainment gap. 
• Clear focus on staff in terms of capacity, leadership, training and development. 
• Changes in practice and improvements to the quality of learning experiences for pupils was believed to be sustainable.

5.124. Sustainability was an important consideration across all Challenge Authorities and they indicated a range of ways in which sustainability issues were being addressed:

• staff capacity and development: upskilling the teacher workforce and using PEF to support further professional development opportunities;
• linking key aspects of Attainment Scotland Fund projects with core local authority provision;
• specific actions at the local authority level. For example, altering their planned programme of work to ensure sustainability. 

5.125. The evidence suggests that exit strategies were being developed by several authorities, although this was not universal. One authority highlighted that whilst they were building sustainability into their programme, there would undoubtedly be a negative impact if funding was withdrawn. 

5.126. Where local authorities were less positive about the potential to sustain improvements beyond the funding timeframe, this was typically related to staffing considerations. For example, one authority noted that, without additional support, local authority budgets would be unable to meet required staffing costs to ensure sustainability. Such concerns about the heavy reliance of the ASF to secure staff was also shared amongst some authorities who believed improvements to be sustainable. 

5.127. Findings from the headteacher survey suggested some evidence of a decrease in the confidence of sustainability. In Year 3 of the survey, 42% of headteachers thought that improvements would be sustainable compared to over half (58%) in Year 2. This was consistent across funding streams. 

5.128. Headteachers who believed in the sustainability of improvements were more likely to be those who expected to see improvements. They also indicated that the increase in staff training/development (70%), the embedding of practice/pedagogies (26%) and overall change in ethos culture (20%) were common reasons to be more confident in sustainability. 

5.129. Evidence from the school case studies also suggested that a belief in sustainability was linked to knowledge that improved practice, skills and expertise was being embedded and that this would have long-lasting impact. 

5.130. As was found in Years 1 and 2, the most common reason for being less confident in the sustainability of progress was due to the potential loss of staffing, resources and skills (reported by 81% of those providing comment on reasons for not expecting improvement to be sustainable). Schools involved in the case studies also raised concerns that without ASF support, the inevitable loss of staffing would impact on the sustainability of the improvements. 

Figure 5.7: Sustainability of improvements, headteacher survey