Scottish Survey of Literacy and Numeracy 2016: literacy

Chapter 3: Reading attainment over time

• Reading performance of P4 and P7 pupils declined slightly between 2012 and 2016 (six and two percentage points respectively). S2 performance in 2016 was similar to 2012.
• Girls outperformed boys in reading at S2 in all three literacy surveys but the picture at primary was mixed; girls outperformed boys at P4 in 2014 and P7 in 2012 and 2016.
• Pupils from the least deprived areas outperformed pupils from the most deprived areas in reading at all stages in all three surveys. The size of the gap in performance has not changed between 2012 and 2016.

A key objective of the SSLN is to monitor national performance over time, in all aspects of literacy, in P4, P7 and S2. This chapter details the methodology used to assess reading attainment over time (2012, 2014, 2016) and provides the trends for key measures from the survey.

3.1 Methodology

Table 3.1 provides the number of pupils participating per stage who completed both a written booklet and an online assessment in each literacy survey. Participation in the reading assessment has increased over the three surveys from 8,450 pupils in 2012 to 10,071 pupils in 2016. The increase in participation was due to a greater number of schools taking part and an increase in the number of pupils successfully submitting the online element of the reading assessment.

Table 3.1: Number of pupils participating in reading in 2012, 2014 and 2016

Stage	2012	2014	2016
P4	2,613	3,155	3,184
P7	2,667	3,223	3,287
S2	3,170	3,542	3,600
Total	8,450	9,920	10,071

The SSLN survey design includes a provision for item release and replacement in order to refresh the survey design or provide examples of the tasks pupils are asked to undertake. They are included, for example, in Education Scotland's Professional Learning Resources. Therefore there was a proportion of assessment booklets that were new in both 2014 and 2016.

In order to check the item release and replacement strategy did not have an adverse effect on the ability to make comparisons, investigative analysis was conducted on 2012 and 2014 data where results were produced on two bases; firstly on all assessment booklets and secondly excluding booklets which had been released or replaced between cycles. A series of in-year and between year comparisons showed that excluding released and replacement booklets did not affect the overall picture of pupil performance and therefore that results across survey years are directly comparable. Item release and replacement between 2014 and 2016 was minimal.

Time series data in this report and the supplementary tables are based on an analysis of all booklets in each survey year. Statistical tests were used to test for significant differences in performance between 2012 and 2014, between 2014 and 2016 and between 2012 and 2016.

3.2 Reading attainment over time

Chart 3.1 shows reading performance by reporting categories and stage, over the three literacy surveys: 2012, 2014 and 2016.

Chart 3.1: Reading performance by reporting category and stage, 2012, 2014 and 2016

The time series trend varies between stages; Table 3.2 shows which changes over time are statistically significant:

Table 3.2: Difference in proportion of pupils performing well or very well in 2012 , 2014 and 2016, by stage

Stage	2012 to 2014	2014 to 2016	2012 to 2016
P4	Lower in 2014	No difference	Lower in 2016
P7	Lower in 2014	No difference	Lower in 2016
S2	Lower in 2014	Higher in 2016	No difference

The proportion of P4 and P7 pupils performing well or very well declined between 2012 and 2014 and remained stable between 2014 and 2016. The decrease between 2012 and 2014 has contributed to the overall fall evident between 2012 and 2016. The extent of the decrease between 2012 and 2016 was greater in P4 (six percentage points) than P7 (two percentage points).

At S2, there was a decrease of four percentage points in the proportion of pupils performing well or very well between 2012 and 2014, which was offset against an increase of two percentage points between 2014 and 2016. As a result, S2 performance in 2016 was statistically similar to performance in 2012.

3.3 Attainment by gender

The pattern of performance by gender over time varied between stages, only some changes in performance were statistically significant. These changes are displayed in Table 3.3.

Table 3.3: Difference in proportion of pupils performing well or very well in 2012, 2014 and 2016, by stage and gender

Girls

Stage	2012 to 2014	2014 to 2016	2012 to 2016
P4	Lower in 2014	No difference	Lower in 2016
P7	Lower in 2014	No difference	No difference
S2	No difference	No difference	No difference

Boys

Stage	2012 to 2014	2014 to 2016	2012 to 2016
P4	Lower in 2014	No difference	Lower in 2016
P7	No difference	No difference	No difference
S2	Lower in 2014	No difference	No difference

For both P4 girls and boys, there was a significant decrease in pupils performing well or very well between 2012 and 2014 but there was no significant change between 2014 and 2016. This reflects the pattern of the overall trend, so the decrease in overall results was attributable to both boys and girls. (Chart 3.2)

At P7, boys' performance remained constant over the three surveys. For P7 girls, despite a small decrease between 2012 and 2014 (three percentage points), performance in 2016 was statistically similar to 2012.

For S2 girls, the proportion of pupils performing well or very well remained stable across all three surveys, ranging between 84 and 86 per cent. The performance of S2 boys decreased between 2012 and 2014 by five percentage points, however overall performance levels in 2012 and 2016 were similar.

Chart 3.2: Proportion of pupils performing well or very well in reading in 2012, 2014 and 2016, by stage and gender

Girls outperformed boys in reading in P4 (2014 only), P7 (2012 and 2016) and S2 (all years). The size of the gender gap has not changed between 2012 and 2016 (or between each of the three surveys) in reading.

3.4 Attainment by deprivation

The pattern of reading performance by deprivation was consistent over the three surveys: pupils from the least deprived areas perform higher than pupils from the most deprived areas.

Changes in performance of pupils within each deprivation category varied between stages, with only some changes in performance being statistically significant. These changes are displayed in Table 3.4.

Table 3.4: Difference in proportion of pupils performing well or very well in 2012, 2014 and 2016, by stage and deprivation

Least deprived category (pupils from the least deprived 30% of datazones)

Stage	2012 to 2014	2014 to 2016	2012 to 2016
P4	Lower in 2014	No difference	Lower in 2016
P7	No difference	No difference	No difference
S2	No difference	No difference	No difference

Middle deprived category (pupils from the middle 40% of datazones)

Stage	2012 to 2014	2014 to 2016	2012 to 2016
P4	Lower in 2014	No difference	Lower in 2016
P7	Lower in 2014	No difference	No difference
S2	No difference	No difference	No difference

Most deprived category (pupils from the most deprived 30% of datazones)

Stage	2012 to 2014	2014 to 2016	2012 to 2016
P4	No difference	No difference	Lower in 2016
P7	No difference	No difference	No difference
S2	Lower in 2014	Higher in 2016	No difference

All measures of performance by stage and deprivation were statistically the same in 2016 as in 2014, apart from the most deprived S2 pupils, where the proportion of pupils performing well or very well increased from 68 per cent to 73 per cent.

Chart 3.3: Proportion of P4 pupils performing well or very well in reading by deprivation category, 2012, 2014 and 2016

Between 2012 and 2016, there was a decrease in P4 pupils performing well or very well for all deprivation categories. As the results were stable between 2014 and 2016, the overall decrease between 2012 and 2016 can be attributable to the decrease between 2012 and 2014 for the least and middle categories.

Chart 3.4: Proportion of P7 pupils performing well or very well in reading in 2012, 2014 and 2016, by stage and deprivation

At P7, the proportions of pupils from the least and most deprived areas performing well or very well have been statistically stable between all three surveys. The only change in performance was in the middle deprivation category where performance fell by three percentage points between 2012 and 2014, but was constant between 2014 and 2016 at 89 per cent (Chart 3.4).

Chart 3.5: Proportion of S2 pupils performing well or very well in reading in 2012, 2014 and 2016, by stage and deprivation

There were no statistical differences between the three surveys for S2 pupils in the least and the middle deprivation categories (Chart 3.5). However, for the most deprived S2 pupils, performance dipped between 2012 and 2014 (eight percentage points) before rising again between 2014 and 2016 (five percentage points). Overall, performance in 2016 (73 per cent of pupils performing well or very well) was statistically similar to that in 2012 (75 per cent).

Table 3.5 shows the performance gap between the least and most deprived pupils in 2016 was statistically similar to 2012, across all stages, even after accounting for changes between intervening years at S2.

Table 3.5: Difference in reading performance by deprivation category in 2012, 2014 and 2016, by stage

Deprivation gap (least deprived minus most deprived), percentage points

Stage	2012	2014	2016
P4	16	13	18
P7	14	12	13
S2	15	22	16

Change in deprivation gap

Stage	2012 to 2014	2014 to 2016	2012 to 2016
P4	No difference	No difference	No difference
P7	No difference	No difference	No difference
S2	Larger in 2014	Smaller in 2016	No difference