Scottish Survey of Literacy and Numeracy 2011

 

Chapter 1: Introduction

1.1: What is the SSLN?

The Scottish Survey of Literacy and Numeracy (SSLN) is an annual sample survey which monitors national performance in literacy and numeracy in alternate years, for school children at P4, P7 and S2. It will also provide information which will inform improvements in learning, teaching and assessment at classroom level.

Approximately 11,000 pupils participated in the survey, which took place in May 2011. The survey consisted of a set of written and practical assessments and a pupil questionnaire, with a further questionnaire completed by about 5,200 teachers. The assessments used in the survey were designed to assess the wide range of knowledge, skills, capabilities and attitudes across learning identified in the Curriculum for Excellence (CfE) experiences and outcomes. They were designed to reflect the CfE requirements that pupils have achieved breadth, challenge and application of learning. The pupil questionnaire collected information on factors that are likely to affect learning, such as pupil attitudes and experience in class. The teacher questionnaire collected information on teachers' experience of delivering numeracy across the curriculum.

Following publication of the Experiences and Outcomes on 2 April 2009, Curriculum for Excellence was increasingly adopted in primary schools from August 2009 and formally rolled out in all secondaries from August 2010. This initially covered S1 only for the 2010/11 year, continuing upwards as the initial cohort progresses through secondary school years. The S2 cohort assessed in the 2011 survey (which was carried out in May 2011) was therefore not following Curriculum for Excellence. However, the survey provides a national performance benchmark on numeracy skills, and assesses skills which pupils should be experiencing as part of good learning and teaching practice.

The SSLN replaces the Scottish Survey of Achievement (SSA) which ran from 2004 to 2009. The SSLN has been developed to support assessment approaches for Curriculum for Excellence, and so results are not comparable with the SSA. The guidance for assessment for CfE is set out in the Assessment for Curriculum for Excellence: Strategic vision and key principles, published in September 2009, and in Building the Curriculum 5: A Framework for Assessment and its supporting suite of publications, first published in January 2010. The SSLN has been under development since 2009, with new assessment materials being trialled in schools throughout this period.

The SSLN is undertaken as part of a partnership between the Scottish Government, Education Scotland, the Scottish Qualifications Authority and local authorities.

1.2: How was the survey carried out?

Unlike many previous assessment surveys, all schools were asked to participate in the survey, ensuring that the demand on each school was minimised. The required sample size of about 4,000 pupils per stage, selected at random, was achievable with two pupils per stage (P4 and P7) in primary and twelve pupils per stage (S2) in secondary. Weighting was then applied to the data to account for the fact that the sampled pupils were representing different numbers of pupils in different schools.

Both publicly funded and independent schools were included. Special schools and schools with fewer than two or five pupils (at primary and secondary respectively) per stage were excluded, and resulting school participation rates for returning pupil assessments were over 90 per cent among publicly funded schools and just under 50 per cent among the independent sector. Weighting was also applied to adjust for non-response.

Pupils completed two booklets, each lasting about an hour, as well as taking part in a pupil teacher interactive assessment covering three tasks involving mental maths, estimation and measurement.

Tasks were either specifically developed for the SSLN by practising teachers and assessment experts, or, where previous SSA tasks were used or revised, these were re-assessed against CfE levels and experiences and outcomes. The assessments were constructed to include tasks with different degrees of challenge and across the range of topics within numeracy set out by the curriculum at each level.

Pupils were assessed at the following Curriculum for Excellence levels:

P4 First level
P7 Second level
S2 Third level

In contrast to the Scottish Survey of Achievement, the SSLN did not assess pupils against other levels. So, for example, although pupils in P4 may be reported as "performing very well at the first level", it is possible that some may be achieving many of the second level tasks as well; however, the SSLN does not capture this information. The principles of Curriculum for Excellence are clear, however, that the curriculum levels are not a barrier to pupils' progress in learning. In progressing through a level, though, pupils must demonstrate breadth and depth of learning and be able to apply their learning in different and unfamiliar contexts.

1.3: Interpretation of SSLN results

As in all sample surveys, since the SSLN is based on a sample of pupils rather than on the whole population, the results shown are estimates. That is to say there is an element of uncertainty within the results because the pupils sampled may not reflect the population exactly. To give a scale to this uncertainty, confidence intervals are produced to show the range of values within which one can be reasonably confident that the actual value would lie if all pupils were assessed. Ninety-five per cent confidence intervals for the main national estimates were calculated and were a maximum of ± two percentage points. This means that the true value of each estimate is likely to lie within two percentage points either side of the given estimate. Margins of error for subsets of the data (by gender, deprivation) are slightly larger. Where appropriate, confidence intervals are represented on charts by short lines to help demonstrate this level of uncertainty. Standard error data for the results, used to calculate these confidence intervals, are provided in the data tables.

The marks of participating pupils have been grouped into categories for ease of reporting. These categories refer to performance in the survey and are not meant to be used for general classroom reporting of performance. The following table gives the summary categories used for each performance level in the SSLN. For example, pupils correctly answering 75 per cent of tasks are described as "performing very well at the level". The assessments are designed to cover the full range of the curriculum at a given level, and so such a pupil might be expected in general to achieve at least 75 per cent of all tasks at their level. Pupils described as working within a level can achieve some of the outcomes expected for their age-group, but they are still working on achieving the others.

Percentage of tasks successfully completed in the SSLN	SSLN reporting category
75 per cent or more	Performing very well at the level
50 per cent or more, but less than 75 per cent	Performing well at the level
P4: less than 50 per cent, but more than 9 per cent	Working within the level
P7: less than 50 per cent, but more than 19 per cent
S2: less than 50 per cent, but more than 34 per cent
P4: 9 per cent or less	Not yet working within the level
P7: 19 per cent or less
S2: 34 per cent or less

These levels were set in consultation with Education Scotland, SQA and teachers, based on an analysis of the tasks involved in the assessment. The differing cut-off scores between "working within the level" and "not yet working within the level" were determined by estimating the number of marks that could potentially be obtained in the assessment using only skills acquired at the previous level. There were more tasks in S2 which used second level skills, and relatively few P4 tasks which used early level skills.

1.4: How will the findings be used?

The results of the 2011 SSLN will be used in line with the survey's three main objectives. These are:

• To monitor and report nationally on achievement in numeracy at the P4, P7 and S2 stages. The 2011 results will establish a baseline for future monitoring of numeracy achievement over time.
• To identify areas of numeracy strengths and weaknesses among pupils in Scotland to help inform policy initiatives and learning and teaching practices.
• To gather information and report nationally on pupils' and teachers' experience of learning and teaching numeracy, along with their views about this experience.

Learning and teaching resources have been developed by Education Scotland from the SSLN survey findings. This information is made available to teachers, schools and authorities to support and inform learning and teaching practice in the classroom. The resources are available on the following Education Scotland website http://www.educationscotland.gov.uk/sslnresource/, with further materials becoming available over the next months.

The survey contains a huge amount of data which cannot be summarised in this publication. Further analysis of the performance data, as well as the full set of pupil and teacher questionnaire results, are published in data tables alongside this publication. You are encouraged to look through these tables to obtain a full picture of the findings. The data will be further analysed over the coming months. The analysis contained in this report seeks to highlight the key messages and give a flavour of the range of analysis possible.

1.5: Further information

Further information on the SSLN, including more complete data tables and technical information about the survey, is available from www.scotland.gov.uk/ssln

There is a range of other reliable information on the performance of Scotland's school pupils.

Scotland participates in the OECD's triennial Programme for International Student Assessment (PISA) survey. This assessment is carried out by 15 year-olds in over sixty countries, including all OECD countries, and as such is a key international benchmark of performance. The results of previous PISA surveys are available at www.scotland.gov.uk/pisa

The Scottish Government also publishes analysis of SQA exam results and leaver destinations. The latest post-appeal data are available at www.scotland.gov.uk/stats/bulletins/00905 Public enquiries (non-media) about the information contained in this Statistics Publication Notice should be addressed to:

Phillipa Haxton

Scottish Government
Education Analytical Services Division
Area 2D South,
Victoria Quay
Edinburgh EH6 6QQ

Tel: +44(0) 131 244 0893
ssln@scotland.gsi.gov.uk

Media enquiries about the information in this Statistics Publication Notice should be addressed to

Mark Dunlop
Tel: +44(0) 131 244 3070