Programme for International Student Assessment (PISA 2022): Scotland's results - highlights

Report covering Scotland's performance in the Programme for International Student Assessment (PISA) 2022, covering maths, reading, and science.


Annex 2: How mathematics is Assessed

161. For each of the four mathematical processes examined in PISA 2022, a mathematics subscale was developed. PISA mathematics test item is designed to capture one of the processes, and students are not necessarily expected to use all three to respond to each test item.

Mathematical reasoning: i.e. "thinking mathematically", is the capacity to use mathematical concepts, tools, and logic to conceptualise and create solutions to real-life problems and situations. It involves recognising the mathematical nature inherent to a problem and developing strategies to solve it. This includes distinguishing between relevant and irrelevant information, using computational thinking, drawing logical conclusions, and recognising how solutions can be applied in a real-world context. Mathematical reasoning is also the capacity to construct arguments and provide evidence to support and explain ones' answers and solutions, and to develop awareness of ones' own thinking processes, including decisions made about which strategies to follow. Mathematical reasoning includes deductive and inductive reasoning. While reasoning underlies the other three mathematical processes described below, it nonetheless is different from them in that reasoning requires thinking through the whole problem-solving process rather than focusing on a specific part of it.

Formulating situations mathematically: mathematically literate students are able to recognise or identify the mathematical concepts and ideas underlying problems encountered in the real world, and then provide mathematical structure to the problems (i.e. formulate them in mathematical terms). This translation – from a contextualised situation to a well-defined mathematics problem – makes it possible to employ mathematical tools to solve real-world problems.

Employing mathematical concepts, facts and procedures: mathematically literate students are able to apply appropriate mathematics tools to solve mathematically formulated problems to obtain mathematical conclusions. This process involves activities such as performing arithmetic computations, solving equations, making logical deductions from mathematical assumptions, performing symbolic manipulations, extracting mathematical information from tables and graphs, representing and manipulating shapes in space, and analysing data.

Interpreting, applying, and evaluating mathematical outcomes: mathematically literate students are able to reflect upon mathematical solutions, results or conclusions and interpret them in the context of the real-life problem that started the process. This involves translating mathematical solutions or reasoning back into the context of the problem and determining whether the results are reasonable and make sense in the context of the problem.

Mathematical content

162. PISA 2022 developed a mathematics subscale for each of these four content domains:

Quantity: number sense and estimation; quantification of attributes, objects, relationships, situations and entities in the world; understanding various representations of those quantifications, and judging interpretations and arguments based on quantity.

Uncertainty and data: recognising the place of variation in the real world, including having a sense of the quantification of that variation, and acknowledging its uncertainty and error in related inferences. It also includes forming, interpreting and evaluating conclusions drawn in situations where uncertainty is present. The presentation and interpretation of data are also included in this category, as well as basic topics in probability.

Change and relationships: understanding fundamental types of change and recognising when they occur in order to use suitable mathematical models to describe and predict change. Includes appropriate functions and equations/inequalities as well as creating, interpreting and translating among symbolic and graphical representations of relationships.

Space and shape: patterns; properties of objects; spatial visualisations; positions and orientations; representations of objects; decoding and encoding of visual information; navigation and dynamic interaction with real shapes as well as representations, movement, displacement, and the ability to anticipate actions in space.

Real-world contexts

163. Mathematical reasoning and problem-solving take place in real-world contexts. There are four different contexts used in PISA 2022, which were also used in previous cycles:

Personal context: related to one's self, one's family or one's peer group. For example, food preparation, shopping, games, personal health, personal transportation, recreation, sports, travel, personal scheduling and personal finance, etc.

Occupational context: related to the world of work. For example, measuring, costing and ordering materials for building payroll/accounting, quality control, scheduling/inventory, design/architecture and job-related decision making either with or without appropriate technology, etc.

Societal context: related to one's community, whether local, national or global. For example, voting systems, public transport, government, public policies, demographics, advertising, health, entertainment, national statistics and economics, etc.

Scientific context: related to the application of mathematics to the natural world, and issues and topics related to science and technology. For example, weather or climate, ecology, medicine, space science, genetics, measurement and the world of mathematics itself

Level

Lower score limit

Percentage of students able to perform tasks at each level or above (OECD average)

Characteristics of tasks

6

669

2.0%

At Level 6, students can work through abstract problems and demonstrate creativity and flexible thinking to develop solutions. For example, they can recognise when a procedure that is not specified in a task can be applied in a non-standard context or when demonstrating a deeper understanding of a mathematical concept is necessary as part of a justification. They can link different information sources and representations, including effectively using simulations or spreadsheets as part of their solution. Students at this level are capable of critical thinking and have a mastery of symbolic and formal mathematical operations and relationships that they use to clearly communicate their reasoning. They can reflect on the appropriateness of their actions with respect to their solution and the original situation.

5

607

8.7%

At Level 5, students can develop and work with models for complex situations, identifying or imposing constraints, and specifying assumptions. They can apply systematic, well-planned problem-solving strategies for dealing with more challenging tasks, such as deciding how to develop an experiment, designing an optimal procedure, or working with more complex visualisations that are not given in the task. Students demonstrate an increased ability to solve problems whose solutions often require incorporating mathematical knowledge that is not explicitly stated in the task. Students at this level reflect on their work and consider mathematical results with respect to the real-world context.

4

545

23.6%

At Level 4, students can work effectively with explicit models for complex concrete situations, sometimes involving two variables, as well as demonstrate an ability to work with undefined models that they derive using a more sophisticated computational-thinking approach. Students at this level begin to engage with aspects of critical thinking, such as evaluating the reasonableness of a result by making qualitative judgements when computations are not possible from the given information. They can select and integrate different representations of information, including symbolic or graphical, linking them directly to aspects of real-world situations. At this level, students can also construct and communicate explanations and arguments based on their interpretations, reasoning, and methodology.

3

482

45.6%

At Level 3, students can devise solution strategies, including strategies that require sequential decision-making or flexibility in understanding of familiar concepts. At this level, students begin using computational-thinking skills to develop their solution strategy. They are able to solve tasks that require performing several different but routine calculations that are not all clearly defined in the problem statement. They can use spatial visualisation as part of a solution strategy or determine how to use a simulation to gather data appropriate for the task. Students at this level can interpret and use representations based on different information sources and reason directly from them, including conditional decision-making using a two-way table. They typically show some ability to handle percentages, fractions and decimal numbers, and to work with proportional relationships.

2

420

68.9%

At Level 2, students can recognise situations where they need to design simple strategies to solve problems, including running straightforward simulations involving one variable as part of their solution strategy. They can extract relevant information from one or more sources that use slightly more complex modes of representation, such as two-way tables, charts, or two-dimensional representations of three-dimensional objects. Students at this level demonstrate a basic understanding of functional relationships and can solve problems involving simple ratios. They are capable of making literal interpretations of results.

1a

358

87.6%

At Level 1a, students can answer questions involving simple contexts where all information needed is present, and the questions are clearly defined. Information may be presented in a variety of simple formats and students may need to work with two sources simultaneously to extract relevant information. They are able to carry out simple, routine procedures according to direct instructions in explicit situations, which may sometimes require multiple iterations of a routine procedure to solve a problem. They can perform actions that are obvious or that require very minimal synthesis of information, but in all instances the actions follow clearly from the given stimuli. Students at this level can employ basic algorithms, formulae, procedures, or conventions to solve problems that most often involve whole numbers.

1b

295

97.4%

At Level 1b, students can respond to questions involving easy to understand contexts where all information needed is clearly given in a simple representation (i.e., tabular or graphic) and, as necessary, recognize when some information is extraneous and can be ignored with respect to the specific question being asked. They are able to perform simple calculations with whole numbers, which follow from clearly prescribed instructions, defined in short, syntactically simple text.

1c

233

99.7%

At Level 1c, students can respond to questions involving easy to understand contexts where all relevant information is clearly given in a simple, familiar format (for example, a small table or picture) and defined in a very short, syntactically simple text. They are able to follow a clear instruction describing a single step or operation.

Contact

Email: keith.dryburgh@gov.scot

Back to top