Scottish National Standardised Assessments: national report

2 Numeracy

2.1 The scope of the numeracy assessment

2.1.1 Numeracy for P1, P4, P7 and S3

There were separate assessments of numeracy for P1, P4, P7 and S3. The same set of organisers was used for each of the assessments, with the proportion of items in each organiser varying by stage to reflect the change in emphasis on each area within Curriculum for Excellence (CfE).

2.1.2 Alignment with Curriculum for Excellence

In the academic year 2018 to 2019, the final version of the Benchmarks (published in June 2017) is used as the reference point for the assessments.

2.1.3 A note on contexts used in the numeracy SNSA

For numeracy assessments within SNSA, a wide variety of simple contexts was used. Typically, questions were short and discrete, so contexts were relatively straightforward in nature, covering a mix of everyday life and the school environment. In a small number of instances, two questions in an assessment referred to the same chart or graph.

All questions were reviewed by Education Scotland for their appropriateness to the Scottish context, prior to inclusion in the assessments.

2.2 Coverage of Curriculum for Excellence: Benchmarks and organisers

SNSA is just one part of the range of assessments that teachers use in making their evaluations of children's and young people's learning. It has been possible to assess most areas of the numeracy Benchmarks within this standardised assessment, using closed response questions that can be automatically marked. It is important to note that, although it is possible to assess all areas of the numeracy Benchmarks through the assessment, the assessments comprise between 30 and 36 questions per learner, according to stage. It is not possible to assess every aspect of the numeracy Benchmarks for a stage within this number of questions. As a result, there will be variation in specific areas assessed in different assessments. Through consultation with Education Scotland staff with a numeracy background, it was agreed that the numeracy assessments should be based on the following organisers: Number; Fractions, decimal fractions and percentages; Measurement, time and money; and Information handling. Each of the questions selected for inclusion in SNSA numeracy assessments for the academic year 2018 to 2019 is aligned with numeracy elements of the Benchmarks: Numeracy and Mathematics. To note: although mathematics Benchmarks are included within the Curriculum for Excellence document Benchmarks: Numeracy and Mathematics, these are not assessed within SNSA, since the numeracy assessment only assesses the numeracy Benchmarks.

All four organisers were represented in the P1, P4, P7 and S3 numeracy assessments, with different proportions across the stages. In the reports provided to schools, teachers received information about the outcomes of the learner at the organiser level, if the learner was presented with at least five questions from that organiser. Similarly, in this report, outcomes for organisers that were addressed by at least five questions in each learner's assessment are analysed. In the case of numeracy, all four organisers are reported on here for each of the assessments, apart from P1.

2.2.1 Number

The main focus of this organiser is understanding the properties of numbers and the ability to use the four basic number operations. It included questions that asked about estimating and rounding; number and number processes; addition, subtraction, multiplication and division; and place value. In P7 and S3, negative numbers were also assessed. This organiser could be regarded as containing some of the basic building blocks of numeracy: the ability to engage with content in the other organisers is dependent to some extent upon the ability to understand place value and properties of number, as well as being able to understand and use the four basic operations of addition, subtraction, multiplication and division.

Figure 2 shows a typical question reflecting the Number organiser for the P1 numeracy assessment. Note that the 'mouth' icon indicates to the learner that there is a voiced component to the question.^[2] In this case, by clicking on the icon, the child heard the full onscreen question text contained in the dialogue balloon. Please note that the question included in Figure 2 and all subsequent figures throughout this report are not included in any of the current assessments for the 2019 to 2020 academic year, and they will not be included in any future SNSA.

Figure 2: Example of a P1 Number question: assessing ability to solve missing number problems

This question is designed to assess children's ability to solve word problems with missing numbers. As is common in number problems for this age group, the question is scaffolded with the inclusion of an image with countable objects. This enables children to use a counting strategy to solve the problem.

Figure 3 shows a difficult Number question from the S3 numeracy assessment for the academic year 2017 to 2018. It builds on young people's existing knowledge of place value in whole numbers to assess their understanding of place value in decimal numbers.

Figure 3: Example of an S3 Number question: assessing understanding of place value in decimal fractions

In this question, young people were asked to choose the correct numerical representation of a decimal number presented in word form. The answer options have been selected carefully to represent some of the most common errors that young people make in relation to place value in decimal numbers. For example, the second answer option is based on the misunderstanding that digits to the right of the decimal point behave identically to the whole numbers to the left of the decimal point, so this number will be regarded by some young people as zero point five thousand. The first and second options were the most common answers among the learners showing a lower outcome over the whole assessment, with the third option (the correct answer) and fourth option being more commonly chosen by learners showing a higher outcome overall.

2.2.2 Fractions, decimal fractions and percentages

The main focus of this organiser is on the ability to recognise and work with fractions, decimal fractions and percentages. At P1, this involves sharing objects into equal sets and recognising representations of halves. By P4, children were asked to recognise standard fraction notation for common fractions, order fractions by size, and recognise pictorial representations of fractions. In the P7 assessment, children were also assessed on their current capacity to work with decimal fractions and percentages, recognise equivalent values in the different forms, and find fractions and percentages of a quantity. At S3, young people were also assessed on their current capacity to work with ratios.

In the assessments for the higher stages, there was an increase in the number of questions assessing this organiser. This shift in the assessments' weightings reflects learners' increasing familiarity with whole numbers, and the transition to applying their understanding of how to calculate with whole numbers to their growing understanding of Fractions, decimal fractions and percentages.

Figure 4 shows a difficult Fractions, decimal fractions and percentages question focusing on percentages from the P7 numeracy assessment for the academic year 2017 to 2018.

Figure 4: Example of a P7 Fractions, decimal fractions and percentages question: assessing ability to find a percentage of a quantity

This question asks children to calculate a given percentage of an amount. The use of 5% as the percentage in the question prompt enables children to use a variety of approaches to solving the problem. The most likely strategy would be the standard approach of dividing by 100 and then multiplying by five; alternatively, children may make use of known facts to divide by 10 and then halve. The use of vinegar as a context enables children to demonstrate that they are able to apply their knowledge to situations where they may not have previously worked with percentages.

Figure 5 shows a Fractions, decimal fractions and percentages question from the middle of the difficulty range for the S3 numeracy assessment for the academic year 2017 to 2018.

Figure 5: Example of an S3 Fractions, decimal fractions and percentages question: assessing conversion of a percentage into an equivalent fraction

This question is set in a simple context and asks young people to find a fraction that is equivalent to 35%. Since none of the denominators for the answer options are 100, young people demonstrated their understanding of simplifying fractions when successfully answering the question. In this case, they had to recognise that for 35/100, both the numerator and denominator are divisible by five.

2.2.3 Measurement, time and money

This organiser draws together the numeracy Benchmarks that cover measurement, time and money. The measurement strand of SNSA progresses from comparing and ordering, and the use of non-standard units of measure at P1, to the use of standard units, reading scales and estimation, and calculating areas by counting squares or other methods at P4. At P7, children are expected to be estimating using standard units of measures and calculate areas of squares, rectangles and right-angled triangles. By S3, young people were asked to demonstrate that they could calculate other areas and volumes and convert between standard units of measure. Money at P1 and P4 focused on the recognition of coin values and calculating change, with P4 additionally covering accurate use of pounds and pence notation. At P7 and S3, budgeting, calculating profit and loss, and currency conversion also featured. Time covered areas such as reading clock times, interpreting timetables, calculating time durations and speed/distance/time calculations, as appropriate for each stage.

Figure 6 shows a question from the middle of the difficulty range for the P4 numeracy assessment for the academic year 2017 to 2018, reflecting the measurement aspect of the organiser.

Figure 6: Example of a P4 Measurement, time and money question: using standard units of measure

The focus of this question is on identifying an appropriate unit of weight. This might be regarded as a less familiar style of question, since the majority of measurement questions focus on calculating values. Nevertheless, being able to recognise units of mass and then choose the most appropriate unit for a context is an important skill. The most commonly chosen answer was the correct answer; this was true across the whole capacity range for the assessment. However, a reasonably high proportion of children at the lower bands chose units of length rather than mass, suggesting a limited understanding of standard units of measure.

Figure 7 shows a typical P1 numeracy question, reflecting the time aspect of the organiser.

Figure 7: Example of a P1 Measurement, time and money question: assessing ability to read digital and analogue clock times

This question is designed to assess children's ability to recognise and read o'clock times in both digital and analogue formats. In this instance, children were asked to select the analogue clock which shows the same time as the digital clock. Typically, in SNSA and other assessments for this age group, learner's understanding of analogue clocks correlate less well with overall capacity in numeracy than is the case for many other aspects of numeracy. As such, typically, there is less of a difference in outcomes for learners working across the capacity range than for other areas of numeracy. This may be due to experiences outside of school that affect children's understanding of this aspect more than in other areas of numeracy.

Figure 8 shows a question from the middle of the difficulty range for the P1 numeracy assessment for the academic year 2017 to 2018, reflecting the money aspect of the organiser.

Figure 8: Example of a P1 Measurement, time and money question: assessing understanding of coin values

This question is designed to assess recognition of coin values and it also assesses the ability to compare and order values. Retaining and processing both pieces of information in a two-step problem adds an additional level of challenge.

2.2.4 Information handling

The main focus of this organiser is on data and analysis. It involves the use and interpretation of a wide range of increasingly complex tables, charts and graphs. At P1, this begins with sorting and categorising objects according to specified criteria. Forms of data presentation that are assessed include Venn and Carroll diagrams, tally charts, tables, block graphs, bar graphs, line graphs and pie charts; types of charts that are covered vary according to stage. Scales on the charts progress in complexity from those numbered in ones to having scales where not all values are marked. From P4 onwards, elements of probability, as detailed in the 'chance and uncertainty' Benchmarks, are also assessed. This includes the language of chance and also the use of the 0 to 1 probability scale.

Figure 9 shows a question from the middle of the difficulty range for the P4 numeracy assessment for the academic year 2017 to 2018, reflecting the data and analysis aspect of the organiser.

Figure 9: Example of a P4 Information handling question: comparing values on a block graph

This question shows a simple column graph where the scale is numbered in ones. For this stage, data from multiple SNSA questions demonstrate that the ability to successfully make comparisons is challenging. Among lower-outcome learners, the fourth option (7), which represents the number of children who chose green, was by far the most common answer, with the third option (5, the correct answer) typically selected only by learners who showed high outcomes overall. Although the presentation of the data is relatively straightforward, this suggests that the need to both read values from the graph, and then make a comparison, added a significant level of challenge.

2.3 National outcomes for numeracy

2.3.1 Overall outcomes

Charts 1a to 1d show, for each stage (P1, P4, P7 and S3), the overall outcomes of all learners for numeracy. These charts also show the outcomes of all learners categorised by each of the four organisers: Number (NUM); Fractions, decimal fractions and percentages (FDP); Measurement, time and money (MTM); and Information handling (IH). The shaded regions on the charts show outcomes across the six capacity bands that are specific to each stage, in line with SNSA reports for the academic year 2018 to 2019.

The charts for P4, P7 and S3 learners include outcomes across six capacity bands. For each stage, the two middle capacity bands had the largest proportions of learners, followed by the second-largest proportions found in the top two capacity bands.

Chart 1a: Numeracy outcomes for P1

For P1, the largest proportions of learners had outcomes corresponding to the top two capacity bands, while the second largest proportions demonstrated outcomes in the middle two bands. At P1, the majority of children achieved outcomes in the top two bands overall, with most of the others having outcomes in the two middle bands. Only very few children had outcomes in the lower two bands.

The majority of children in P1 showed outcomes in the top two bands with regard to all organisers. Across the organisers, few children had outcomes corresponding to the lower two capacity bands.

Chart 1b: Numeracy outcomes for P4

Overall, at P4, a majority of learners achieved outcomes in the two middle bands, while lower proportions achieved outcomes in the two highest and the two lowest bands.

For the organisers Number and Information handling, about half of learners or somewhat fewer demonstrated outcomes in the middle two bands. For the organiser Fractions, decimal fractions and percentages, relatively large proportions of P4 learners, albeit still fewer than half, achieved outcomes in the two upper bands. For the organiser Measurement, time and money, the majority of P4 learners achieved outcomes in the two middle bands. Across all organisers, a small minority of P4 learners had outcomes in the two lowest capacity bands.

Chart 1c: Numeracy outcomes for P7

Overall, more than half of learners at P7 achieved outcomes in the middle two bands on their numeracy assessment, while smaller proportions achieved outcomes in the lowest two bands.

For the organisers Information handling and Measurement, time and money, a majority of children in P7 had outcomes corresponding to the two middle capacity bands. For the other two organisers (Number and Fractions, decimal fractions and percentages), less than half of the learners achieved outcomes in the two middle bands, while more than a third achieved outcomes in the two highest bands. Across all four individual organisers, relatively few learners had outcomes corresponding to the lowest two capacity bands.

Chart 1d: Numeracy outcomes for S3

At S3, more than half of the learners achieved outcomes corresponding to the two middle bands on their numeracy assessment, while only smaller proportions demonstrated outcomes corresponding to the lowest two bands.

For the organisers Information handling and Fractions, decimal fractions and percentages, half or more of the learners at S3 achieved outcomes corresponding to the two middle bands. For the other two organisers, somewhat higher proportions of learners achieved outcomes corresponding to the highest two bands. Across all organisers, relatively low proportions of learners at S3 showed outcomes in the lower two capacity bands.

2.3.2 Sex

Charts 2a to 2d show numeracy outcomes for boys and girls for each stage. Outcomes are given for numeracy overall and also by organiser.

For P1, overall, about half of boys and girls had outcomes corresponding to the top two capacity bands. For P4, P7 and S3, much lower proportions of boys and girls had outcomes in the upper two bands. For these stages, most boys and girls had assessment outcomes corresponding to the two middle capacity bands. Across all stages, only small percentages of girls and boys had outcomes in the two lowest bands.

Chart 2a: Numeracy outcomes distributed by sex for P1

At P1, there was a slightly higher proportion of boys than girls who achieved outcomes in the two lower bands and the two upper bands. The proportion of girls achieving outcomes in the two middle bands was slightly larger than among boys. This pattern was consistent for the numeracy assessment as a whole and when the individual organisers are considered.

Differences in assessment outcomes between boys and girls were generally small, regardless of whether the whole numeracy assessment or only specific organisers were considered. Both boys and girls mostly had outcomes corresponding to the top four capacity bands.

Chart 2b: Numeracy outcomes distributed by sex for P4

At P4, slightly more boys than girls achieved an outcome in the top two capacity bands, while a higher proportion of girls achieved an outcome in the two middle bands. The outcomes are similar when reviewing each organiser separately.

While the difference in outcomes between boys and girls was small overall and by organiser, for Number, there were slightly higher proportions of boys than girls in the upper two bands.

Chart 2c: Numeracy outcomes distributed by sex for P7

At P7, overall, there were larger proportions of boys than girls with outcomes in the top two bands. These differences were noticeably larger than those for learners at P4. This finding holds also when considering the four organisers separately. The largest differences in outcomes between boys and girls were observed for the organisers Number and Fractions, decimal fractions and percentages, where there were higher percentages of boys in the upper two bands.

Chart 2d: Numeracy outcomes distributed by sex for S3

At S3, overall, the differences between the boys and girls were larger than for P4 or P7, with higher proportions of boys than girls achieving outcomes in the upper two bands. There were also somewhat larger proportions of girls than boys with outcomes in the lower two bands. Differences between boys and girls were largest for Fractions, decimal fractions and percentages. For Measurement, time and money, there were also noticeable differences in the proportions of boys and girls in the two bottom and the two top categories.

2.3.3 Scottish Index of Multiple Deprivation

This section reviews the distribution of learners for all stages across categories that reflect the Scottish Index of Multiple Deprivation (SIMD). To simplify the display of outcomes and aid their interpretation, we have divided the original 20 categories (vigintiles) into three combined categories of socio-economic background. These combined categories are: SIMD 1–4, indicating the bottom socio-economic quintile (that is, the most deprived children and young people, those in vigintiles 1 to 4); SIMD 5–16, indicating the three middle quintiles (vigintiles 5 to 16); and SIMD 17–20, indicating the top socio-economic quintile (that is the least deprived children and young people, those in vigintiles 17 to 20).

Charts 3a to 3d show the proportions of numeracy outcomes by capacity band for learners within these SIMD categories for all stages. Outcomes are presented for numeracy overall and grouped by organiser.

At each stage, the proportion of learners with outcomes in the two upper capacity bands was noticeably larger among learners in the least deprived group (SIMD 17–20), while there were relatively larger proportions of learners with outcomes in the two lowest capacity bands among learners in the most deprived group (SIMD 1–4). This pattern was present across all stages, both for numeracy overall and for outcomes grouped by organiser.

The charts illustrate that the difference in numeracy outcomes between learners from the most deprived and least deprived groups (bottom quintile and the top quintile) was relatively smaller among children in P1, while it was more pronounced among learners at P4 and P7, and largest among those enrolled in S3. These observations apply both to overall outcomes in numeracy and to each of the organisers in this learning area.

At P1, across all SIMD groupings, the majority of children achieved outcomes in the top two capacity bands both overall and by organiser. Across other stages, both for numeracy overall and by organiser, there were higher proportions of learners with outcomes in the two upper capacity bands among learners in the least deprived group (SIMD 17–20). Similarly, for P4, P7 and S3, there were noticeably lower proportions of learners from the least deprived group (SIMD 17–20) in the lowest two bands, compared to other learners.

Chart 3a: Numeracy outcomes distributed by SIMD* for P1

*The Scottish Index of Multiple Deprivation grades the socio-economic condition of the population over 20 categories (vigintiles). SIMD 1 to 4 correspond to the most deprived 20% of the population. SIMD 5 to 16 correspond to the middle 60% of the population. SIMD 17 to 20 correspond to the least deprived 20% of the population.

Chart 3b: Numeracy outcomes distributed by SIMD for P4

Chart 3c: Numeracy outcomes distributed by SIMD for P7

Chart 3d: Numeracy outcomes distributed by SIMD for S3

2.3.4 Ethnic background

This section looks at differences in outcomes between learners from 'White Scottish' and other ethnic backgrounds. Charts 4a to 4d show the percentages of learners in these two groups.

The outcomes show that, generally, there were only relatively small differences across the two comparison groups, both in terms of overall numeracy outcomes and when considering assessment outcomes regarding each of the organisers.

At P1, among children from a 'White Scottish' background, there were slightly higher proportions of children with outcomes in the top two bands than for children from other backgrounds. However, among learners enrolled in the other stages (P4, P7 and S3), both for numeracy overall and by organiser, there were slightly larger proportions with outcomes in the two upper capacity bands among learners from other ethnic backgrounds than among those from a 'White Scottish' background.

Chart 4a: Numeracy outcomes distributed by ethnic background for P1

Chart 4b: Numeracy outcomes distributed by ethnic background for P4

Chart 4c: Numeracy outcomes distributed by ethnic background for P7

Chart 4d: Numeracy outcomes distributed by ethnic background for S3

2.3.5 Free School Meal Entitlement

Charts 5a to 5d show the numeracy outcomes of learners according to entitlement to free school meals (FME). These charts distinguish those with entitlement from all other learners.^[3]

At each stage, there were notably larger proportions with outcomes in the upper two capacity bands among learners not entitled to free school meals than among those with FME. There were correspondingly higher proportions of outcomes in the lower two capacity bands among learners with FME. This pattern was similar across all stages, both for numeracy overall and by organiser.

At P1, about half of the children without FME had outcomes corresponding to the two upper bands. For the other stages, fewer learners without FME had outcomes in the two upper bands, while about half of the learners achieved outcomes corresponding to the two middle bands. Differences between the FME / not FME groups were roughly similar for each organiser at each stage.

Chart 5a: Numeracy outcomes distributed by FME for P1

Chart 5b: Numeracy outcomes distributed by FME for P4

Chart 5c: Numeracy outcomes distributed by FME for P7

Chart 5d: Numeracy outcomes distributed by FME for S3

2.3.6 Additional Support Needs

Charts 6a to 6d show the distributions of learners according to whether or not learners were registered as learners with Additional Support Needs (ASN).

For all stages, the proportions of learners in the two upper bands were notably larger among learners with no ASN. Similarly, there were relatively larger proportions of learners with ASN in the two lower bands.

While in P1, differences between the proportions of learners with and without ASN in the two lower bands were somewhat less pronounced, at P4, P7 and S3, there tended to be about a third of learners with ASN in the two lower bands, compared to a fifth, or fewer, of learners without ASN. These differences were similar for numeracy overall as well as when considering each organiser separately.

Chart 6a: Numeracy outcomes distributed by ASN for P1

Chart 6b: Numeracy outcomes distributed by ASN for P4

Chart 6c: Numeracy outcomes distributed by ASN for P7

Chart 6d: Numeracy outcomes distributed by ASN for S3

2.3.7 Looked After Children at Home and Looked After Children Away from Home

Charts 7a to 7d compare learners who were classified as Looked After Children at Home (LAH) and Looked After Children Away from Home (LAA) with all other learners. The information was taken from the national database, SEEMiS.

Across all stages, among learners classified as LAH and/or LAA, notably smaller proportions achieved outcomes in the two upper capacity bands, compared to other learners without these classifications. Similarly, among learners classified as LAH or LAA, there were higher proportions with outcomes in the two lower bands than among other learners. This pattern was similar across all stages, both for numeracy overall and by organiser.

Chart 7a: Numeracy outcomes distributed by LAH/LAA for P1

Chart 7b: Numeracy outcomes distributed by LAH/LAA for P4

Chart 7c: Numeracy outcomes distributed by LAH/LAA for P7

Chart 7d: Numeracy outcomes distributed by LAH/LAA for S3

2.3.8 English as an Additional Language

Charts 8a to 8d compare learners by language background: English as an Additional Language (EAL) compared with all other learners. The category 'EAL' refers to those learners whose record in SEEMiS showed that they had English as an additional language. The 'Not EAL' category comprises both learners who were recorded as not having EAL and those for whom there was no entry in this field.

When comparing learners for whom English is an Additional Language (EAL) and those for whom English is their first language, we observed only relatively small differences in the number of learners achieving outcomes in the highest two capacity bands, both in terms of numeracy overall and by organiser. These patterns were similar across all stages.

At P1, differences between the two groups were slightly larger than for the other stages (P4, P7 and S3). When comparing proportions in bands for each organiser, we observed similar differences between the two groups, with larger proportions of learners with English as their first language having outcomes in the highest two bands.

Chart 8a: Numeracy outcomes distributed by EAL for P1

Chart 8b: Numeracy outcomes distributed by EAL for P4

Chart 8c: Numeracy outcomes distributed by EAL for P7

Chart 8d: Numeracy outcomes distributed by EAL for S3