Scottish National Standardised Assessments: national report for academic year 2017-2018

2 Numeracy

2.1 The scope of the numeracy assessment

2.1.1 Numeracy for P1, P4, P7 and S3

There were separate assessments of numeracy for P1, P4, P7 and S3. The same set of organisers was used for each of the assessments, with the proportion of items in each organiser varying by stage to reflect the change in emphasis on each area within Curriculum for Excellence (CfE).

2.1.2 Alignment with Curriculum for Excellence

In the academic year 2017 to 2018, the assessments of numeracy were based on elements of CfE as articulated in the Benchmarks: Numeracy and Mathematics, published as a draft in August 2016. It should be noted that for the academic year 2018 to 2019, the final version of the Benchmarks (published in June 2017) is used as the reference point for the assessments.

2.1.3 A note on contexts used in SNSA numeracy assessments

For SNSA numeracy, a wide variety of simple contexts was used across the assessments. Questions typically comprised short, discrete questions, so contexts were relatively straightforward in nature, covering a mix of everyday life and the school environment. In a small number of instances, two questions in an assessment referred to the same charts or graphs.

All questions were reviewed by Education Scotland for their appropriateness to the Scottish context, prior to inclusion in the assessments.

2.2 Coverage of Curriculum for Excellence: benchmarks and organisers

SNSA is just one part of the range of assessments that teachers use in making their evaluations of children’s and young people’s learning. It has been possible to assess most areas of the numeracy benchmarks within this standardised assessment, using questions capable of being scored automatically. It is important to note that, although it is possible to assess all areas of the numeracy benchmarks through the assessment, the number of questions available does not allow for all areas to be covered within a single year, so the coverage will vary each year. Through consultation with Scottish numeracy experts, it was agreed that the numeracy assessments should be based on the following organisers: Number; Fractions, decimal fractions and percentages; Measurement, time and money; and Information handling. Each of the questions selected for inclusion in SNSA numeracy assessments for the academic year 2017 to 2018 is aligned with numeracy elements of the Benchmarks: Numeracy and Mathematics.

All four organisers were represented in the P1, P4, P7 and S3 numeracy assessments, with different proportions across the year groups. In the reports provided to schools, teachers received information about the capacity of the learner at the organiser level, if the learner was presented with at least five questions from that organiser. Similarly, in this report, results for organisers that were addressed by at least five questions in the year group’s full set are analysed. In the case of numeracy, all four organisers are reported on here for each of the assessments.

2.2.1 Number

The main focus of this organiser is understanding the properties of numbers, and the ability to use the four basic number operations. It included questions that asked about estimating and rounding; number and number processes; addition, subtraction, multiplication and division; and place value. In P7 and S3, negative numbers were also assessed. This organiser could be regarded as containing some of the basic building blocks of numeracy: the ability to engage with content in the other organisers is dependent to some extent upon the ability to understand place value and properties of number, as well as being able to understand and use the four basic operations of addition, subtraction, multiplication and division.

Figure 2 shows a typical question reflecting the Number organiser for the P1 numeracy assessment. Note that the ‘mouth’ icon indicates to the learner that there is a voiced component to the question.^[3] In this case, by clicking on the icon the child heard the full onscreen question text contained in the dialogue balloon. Please note that the question included in Figure 2 and all subsequent figures throughout this report are not included in any of the current assessments for the 2018 to 2019 academic year, and they will not be included in any future SNSA.

Figure 2: Example of a P1 Number question: assessing ability to solve missing number problems

This question is designed to assess children’s ability to solve word problems with missing numbers. As is common in number problems for this age group, the question is scaffolded with the inclusion of an image with countable objects. This enables children to use a counting strategy to solve the problem.

Figure 3 shows a high difficulty Number question from the S3 numeracy assessment for the academic year 2017 to 2018. It builds on young people’s existing knowledge of place value in whole numbers to assess their understanding of place value in decimal numbers.

Figure 3: Example of an S3 Number question: assessing understanding of place value in decimal fractions

In this question, young people were asked to choose the correct numerical representation of a decimal number presented in word form. The answer options have been carefully selected to represent some of the most common misconceptions in relation to place value in decimal numbers. For example, the second answer option is based on the misunderstanding that digits to the right of the decimal point behave identically to the whole numbers to the left of the decimal point, so this number will be regarded by some young people as zero point five thousand. The first and second options were the most common answers among the learners showing lower capacity over the whole assessment, with the third option (the correct answer) and fourth option being more commonly chosen by learners showing higher capacity overall.

2.2.2 Fractions, decimal fractions and percentages

The main focus of this organiser is on the ability to recognise and work with fractions, decimal fractions and percentages. At P1, this involves sharing objects into equal sets and recognising representations of halves. By P4, children were asked to recognise standard fraction notation for common fractions, order fractions by size, and recognise pictorial representations of fractions. In the P7 assessment, children were also assessed on their capacity to work with decimal fractions and percentages, recognise equivalent values in the different forms, and find fractions and percentages of a quantity. At S3, young people were also assessed on their capacity to work with ratios.

In the assessments for the higher year groups, there was an increase in the number of questions assessing this organiser. This shift in the assessments’ weightings reflects learners’ increasing familiarity with integers and the transition to applying their understanding of how to calculate with integers to their growing understanding of fractions, decimal fractions and percentages.

Figure 4 shows a high difficulty Fractions, decimal fractions and percentages question focusing on percentages from the P7 numeracy assessment for academic year 2017 to 2018.

Figure 4: Example of a P7 Fractions, decimal fractions and percentages question: assessing ability to find a percentage of a quantity

This question asks children to calculate a given percentage of an amount. The use of 5% as the percentage in the question prompt enables children to use a variety of approaches to solving the problem. The most likely strategy would be the standard approach of dividing by 100 and then multiplying by five; alternatively, children may make use of known facts to divide by 10 and then halve. The use of vinegar as a context enables children to demonstrate that they are able to apply their knowledge to situations where they may not have previously worked with percentages.

Figure 5 shows a medium difficulty Fractions, decimal fractions and percentages question from the S3 numeracy assessment for academic year 2017 to 2018.

Figure 5: Example of an S3 Fractions, decimal fractions and percentages question: assessing conversion of a percentage into an equivalent fraction

This question is set in a simple context and asks young people to find a fraction that is equivalent to 35%. Since none of the denominators for the answer options are 100, young people demonstrated their understanding of simplifying fractions when successfully answering the question. In this case, they had to recognise that for 35/100, both the numerator and denominator are divisible by five.

2.2.3 Measurement, time and money

This organiser draws together the numeracy benchmarks that cover measurement, time and money. The measurement strand of SNSA progresses from comparing and ordering, and the use of non-standard units of measure at P1, to the use of standard units, reading scales and estimation, and calculating areas by counting squares at P4. By S3, young people were asked to demonstrate that they could calculate areas and volumes and convert between standard units of measure. Money at P1 and P4 was about recognition of coin values and calculating change. At P7 and S3, budgeting, calculating profit and loss, and currency conversion also featured. Time covered areas such as reading clock times, interpreting timetables, calculating time durations and speed/distance/time calculations.

Figure 6 shows a medium difficulty question from the P4 numeracy assessment for academic year 2017 to 2018, reflecting the measurement aspect of the organiser.

Figure 6: Example of a P4 Measurement, time and money question: using standard units of measure

The focus of this question is on identifying an appropriate unit of weight. This might be regarded as a less familiar style of question, since the majority of measurement questions focus on calculating values. Nevertheless, being able to recognise units of mass and then choose the most appropriate unit for a context is an important skill. The correct answer was the most commonly chosen answer among children, whether the overall capacity they demonstrated was high, medium or low. However, a reasonably high proportion of children showing lower capacity overall chose units of length rather than mass, suggesting a limited understanding of standard units of measure.

Figure 7 shows a typical P1 numeracy question, reflecting the time aspect of the organiser.

Figure 7: Example of a P1 Measurement, time and money question: assessing ability to read digital and analogue clock times

This question is designed to assess children’s ability to recognise and read o’clock times in both digital and analogue formats. In this instance, children were asked to select the analogue clock which shows the same time as the digital clock. Typically, in SNSA and other assessments for this age group, differences in understanding between children with different levels of capacity are less pronounced for questions involving analogue clocks than for many other curriculum areas. This may be due to experiences outside of school that affect children’s understanding of this aspect more than in other areas of numeracy.

Figure 8 shows a medium difficulty question from the P1 numeracy assessment for academic year 2017 to 2018, reflecting the money aspect of the organiser.

Figure 8: Example of a P1 Measurement, time and money question: assessing understanding of coin values

This question is designed to assess recognition of coin values and it also assesses the ability to compare and order values. Retaining and processing both pieces of information in a two- step problem adds an additional level of challenge.

2.2.4 Information handling

The main focus of this organiser is on data and analysis. It involves the use and interpretation of a wide range of increasingly complex tables, charts and graphs. At P1, this begins with sorting and categorising objects according to specified criteria. Forms of data presentation that are assessed include Venn and Carroll diagrams, tally charts, tables, block graphs, bar graphs, line graphs and pie charts. Scales on the charts progress in complexity from those numbered in ones to having scales where not all values are marked. From P4 onwards, elements of probability, as detailed in the ‘chance and uncertainty’ benchmarks, are also assessed. This includes the language of chance and also the use of the 0 to 1 probability scale.

Figure 9 shows a medium difficulty question from the P4 numeracy assessment for academic year 2017 to 2018, reflecting the data and analysis aspect of the organiser.

Figure 9: Example of a P4 Information handling question: comparing values on a block graph

This question shows a simple column graph where the scale is numbered in ones. For this year group, data from multiple SNSA questions demonstrate that the ability to successfully make comparisons is challenging. Among lower-capacity learners, the fourth option (7), which represents the number of children who chose green, was by far the most common answer, with the third option (5, the correct answer) typically selected only by learners who showed high capacity overall. Although the presentation of the data is relatively straightforward, this suggests that the need to both read values from the graph and then make a comparison added a significant level of challenge.

2.3 National results for numeracy

2.3.1 Overall capacity

Chart 1 shows, for each year group (P1, P4, P7 and S3), the overall capacity of all learners for numeracy as well as the capacity of all learners when only considering assessment content related to each of the four organisers: Number (NUM); Fractions, decimal fractions and percentages (FDP); Measurement, time and money (MTM); and Information handling (IH).

Regions show high, medium and low capacity which are specific to each year group, in line with SNSA reports for the academic year 2017 to 2018.

Chart 1: Numeracy capacity by SNSA year

The chart shows that P1 and P7 learners are distributed across the three capacity regions such that their largest proportion is in the high capacity category, followed by the second- largest proportion in the medium capacity. For P4 and S3, the largest proportion of learners

is in the medium overall capacity category, followed by those who demonstrate high capacity.

At P1, the majority of children showed high capacity overall, with most of the others showing medium capacity. Only very few children showed low capacity. The majority of children in P1 showed high capacity with regard to all organisers. Across the four organisers, few children were found in the low capacity groups.

Overall, at P4, a majority of the learners demonstrated medium capacity, while a lower proportion showed high capacity and a minority showed low capacity. The Information handling organiser had the majority of learners in the high capacity group, while the Fractions, decimal fractions and percentages organiser had the majority of children in the medium capacity group. The Measurement, time and money and Number organisers had a similar distribution across medium and high capacity groups, with slightly larger proportions of learners in the medium capacity group. There was little difference when comparing levels of low capacity for each of the organisers, with only small proportions of children demonstrating low capacity in any of the organisers.

Overall, the majority of learners at P7 demonstrated high capacity on their numeracy assessment, while only relatively small proportions demonstrated low capacity. A majority of children in P7 showed high capacity with regard to each individual organiser. Across the four individual organisers, very few children were found in the low capacity groups.

At S3, the majority of learners demonstrated medium capacity for Measurement, time and money; Number; and Information handling, while for the Fractions, decimal fractions and percentages organiser, the majority of learners demonstrated high capacity. For each individual organiser, a small proportion of learners at S3 were in the low capacity region.

Evidence from educational research suggests that learning growth differs according to the stage of schooling. Bearing in mind that individuals experience growth spurts and plateaus in different ways, younger learners tend on average to advance in their learning more rapidly than older learners, for whom progress typically continues, but at a slower pace. The two SNSA norming studies^[4] carried out in Scotland during the academic year 2017 to 2018, in the first and second halves of the academic year, allowed us to confirm that these findings also hold for numeracy acquisition among Scottish learners.

Chart 2: Numeracy capacity across norming study time periods

Chart 2 shows that between November and March, learning progress with regard to numeracy development was most prominent among learners at P1. In November 2018, less than half showed high capacity, while in March 2018, a majority was recorded with capacity in this region. At P4, in March 2018 compared with November 2017, there were relatively smaller but still noticeable increases in the proportion of learners at high capacity in this learning area. Learning progress at P7 was less marked than in the other year groups, with similar proportions of learners in each of the capacity regions in November and March. At S3, there was again a noticeable increase in the proportion of learners with high capacity between the two norming studies.

2.3.2 Gender

Chart 3: Numeracy capacity distribution by gender and SNSA year

Chart 3 shows the proportions of numeracy capacity among boys and girls for all SNSA year groups, overall and when considering each of the four organisers. For P1 and P7, overall, the majority of boys and girls were in the high capacity region, while for P4, the majority of boys and girls were in the medium capacity region. In S3, the majority of girls was in the medium category overall, with slightly less than half of boys reaching the same level. There were some notable differences between the two gender groups, with more girls in the high capacity region at P1, and more boys in the high capacity region at P4, P7 and S3. Small proportions of girls and boys were in the low capacity region across all SNSA year groups.

At P1, girls showed slightly higher levels of capacity in each individual organiser and overall. While the difference between the results for boys and girls was small across all organisers, it was slightly more noticeable when considering only assessment content related to the organiser Fractions, decimal fractions and percentages. Both boys and girls mostly gained results in the high and medium capacity regions across all organisers, with very few learners recorded within the low capacity region.

At P4, slightly more boys than girls showed a high overall capacity, while a higher proportion of girls demonstrated an overall capacity in the medium region. These results were also observed when considering the four organisers individually. Whilst the difference between boys’ and girls’ capacity was small for all organisers, it was slightly more noticeable when considering only assessment content related to the organiser Number.

At P7, a larger proportion of boys demonstrated high capacity overall when compared with girls, and the differences were somewhat larger than among learners at P4. This finding holds for overall and when considering the four organisers separately. The largest difference in capacity between boys and girls within organisers was observed for the organiser Measurement, time and money, in favour of boys.

At S3, overall, there were also notably larger proportions of boys when compared with girls in the high capacity region. Larger proportions of girls attained a medium capacity when compared with boys. This difference was most marked when considering only assessment content related to the organiser Fractions, decimal fractions and percentages, where a majority of boys had results in the high capacity region for this organiser, while the results of girls were distributed in similar proportions across the medium and high capacity regions.

2.3.3 Scottish Index of Multiple Deprivation

This section reviews the distribution of learners for all SNSA year groups across categories which reflect the Scottish Index of Multiple Deprivation (SIMD). To simplify the display of results and aid their interpretation, we used three categories to indicate levels of socio- economic background, namely: 1–4, indicating the bottom socioeconomic quintile (that is, the most deprived children and young people, those in vigintiles 1 to 4); 5–16, indicating the three middle quintiles (that is, those in vigintiles 5 to 16); and 17–20, indicating the top quintile (that is, the least deprived children and young people, those in vigintiles 17 to 20).

Chart 4 shows the proportions of numeracy capacity for these SIMD categories for all SNSA year groups, overall and when considering each of the four organisers.

Chart 4: Numeracy capacity distribution by SIMD and SNSA year

At each year group, it can be seen that the proportion of learners in the high capacity category was noticeably larger in the SIMD category that reflects the uppermost socioeconomic status (i.e. less deprivation), while the relatively larger proportions of learners with low capacity were observed in the SIMD category reflecting lower socioeconomic status. This pattern was present in all four year groups, overall and when considering each individual organiser.

Chart 4 illustrates that the difference in numeracy capacity between children from the bottom quintile and the top quintile of SIMD was relatively smaller at P1, while it was more substantial at P4 and P7, and largest at S3. These observations apply both to overall capacity in numeracy and to each of the organisers in this learning area.

At P1, the majority of children demonstrated capacity in the high region for all organisers except Number, where slightly less than half of children in the bottom quintile demonstrated high capacity. In all other years, both overall and within organisers, high capacity for more than half of learners was mostly observed for those in the top quintile group. At S3, the year group where the largest proportion of learners was recorded in the low capacity region overall and for each organiser, among learners from the bottom quintile there tended to be roughly similar proportions in the low capacity category and the high capacity category overall and for all organisers except Fractions, decimal fractions and percentages.

2.3.4 Ethnic background

This section looks at differences in capacity between learners from ‘White Scottish’ and other ethnic backgrounds. Chart 5 shows the SNSA numeracy results for these two groups.

Chart 5: Numeracy capacity distribution by Ethnic Background and SNSA year

The results show that, generally, there were only very small differences across the two comparison groups, both in terms of overall capacity and when considering results for each of the organisers. At P1, among children from a ‘White Scottish’ background there were slightly higher proportions of learners with high capacity than among others. However, the proportions were of a similar size when considering only assessment material related to the organiser Number. Among learners at the higher year groups (P4, P7 and S3), both overall and for most of the organisers, there were slightly larger proportions with high capacity among learners from other ethnic backgrounds than among those from a ‘White Scottish’ background.

2.3.5 Free School Meal Registered

Chart 6 shows the numeracy capacity of learners according to groups defined by registration for free school meals (FSE). This chart distinguishes those with registered entitlement from all other learners.^[5]

Chart 6: Numeracy capacity distribution by Free School Meal Registered and SNSA year

In each of the four year groups, there were notably larger proportions in the high capacity group among learners without entitlement than among those with FSE, and there were correspondingly higher proportions in the low capacity group among learners with FSE. This pattern is similar across all year groups and also when considering results by organisers.

At P1 and P7, the majority of children without FSE showed high capacity, while for learners with this entitlement around a half for P1 and less than a half for P7 demonstrated capacity in this category. Differences between the two groups were roughly similar when considering each of the four organisers within these year groups. However, among P1 learners, the difference was somewhat smaller when considering only assessment content related to the organiser Information handling.

At P4 and S3, differences in proportions with high capacity between learners with and without FSE were similar to those in the other year groups, although at both year groups there were generally lower proportions of learners in this capacity region. The pattern was similar when reviewing results by organisers. At S3, overall and across organisers, the proportion of learners with FSE in the low capacity region tended to be equal to or larger than those with results in the high capacity region.

2.3.6 Additional Support Needs

Chart 7 shows the proportions of learners with high, medium and low capacity across SNSA year groups, according to whether or not learners were identified as having Additional Support Needs (ASN).

Chart 7: Numeracy capacity distribution by ASN and SNSA year

Across all year groups, it can be seen that the proportions of learners in the high category were notably larger among learners with no ASN, and similarly, there were larger proportions of learners with ASN in the low capacity category. This pattern is present in all year groups.

At P1, differences in overall capacity between the two groups were somewhat smaller than in the higher year groups: the majority of learners without support needs showed high capacity but about half of learners with ASN also showed high capacity. In P1 at organiser level, a large proportion of ASN learners showed high capacity in two of the organisers (Measurement, time and money and Information handling). Less than half of learners identified as having ASN had results in the high capacity category for the other two organisers (Number and Fractions, decimal fractions and percentages). Additionally, there were more learners with ASN in the low capacity category in all organisers.

At P4 and S3, most learners with and without ASN demonstrated medium capacity overall, and this was the general tendency also when considering only assessment material for each of the individual organisers. However, the proportion of learners with ASN in the low capacity category was notably larger than for learners without ASN, and notably smaller in the high capacity category than in the comparison group.

At P7, the majority of learners without ASN demonstrated high capacity, while learners with ASN mostly fell within the medium capacity category. These learners were also more prominent, proportionally, in the low capacity category.

2.3.7 Looked After Children at Home and Looked After Children Away from Home

Chart 8 shows the proportions of learners with high, medium and low capacity by categories of Looked After Children at Home (LAH) and Looked After Children Away from Home (LAA), in comparison with other learners, as classified within SEEMiS (the national database).

Chart 8: Numeracy capacity distribution by LAH/LAA status and SNSA year

Across all four year groups, it can be seen that among those learners who identified as LAH and/or LAA, there were notably smaller proportions demonstrating high capacity, while among other learners there were larger proportions demonstrating high capacity. Similarly, among learners who are LAH or LAA, there were higher proportions demonstrating low capacity than among other learners. This pattern is similar across all four year groups, both overall and when only considering assessment content related to individual organisers.

At P1, the differences between learners registered as LAH and/or LAA were somewhat smaller than in the other year groups, in particular, when considering only the organiser Information handling, where majorities in both groups of learners had results in the high capacity region. Differences among the two comparison groups were greatest at S3, where only few learners registered as LAH and/or LAA had results in the high capacity region.

2.3.8 English as an Additional Language

Chart 9 shows the numeracy capacity of learners according to groups defined by language background: English as an Additional Language (EAL) and all other children and young people. The category ‘Yes’ refers to those learners whose record in SEEMiS showed that they had English as an additional language. The ‘Other’ category comprises both learners for whom there was a ‘No’ as the entry for EAL, and those for whom there was no entry in this field.

Chart 9: Numeracy capacity distribution by English as an Additional Language (EAL)

When comparing proportions of high, medium or low capacity between groups of learners for whom English is an additional language (EAL) and those for whom English is their first language, we observed relatively small differences in favour of learners in the latter group, both in terms of overall capacity and when considering only individual organisers (Chart 9). Similar patterns were observed across all four year groups.

At P1, differences between the two groups were slightly larger than in higher year levels (P4, P7 and S3). When comparing proportions across the organisers, similar differences between the two groups in favour of P1 learners for whom English is their first language were observed. The largest difference was recorded when considering only content related to the organiser Measurement, time and money.