Achievement of Curriculum for Excellence (CfE) Levels 2018-19

Chapter 9: Background notes

9.1 Sources, coverage, methodology and quality assurance

9.1.1 Sources

The data included in the publication are provided to Scottish Government by local authorities and grant-aided schools. Independent schools are not included in the data collection. The assessments of children’s progress are based on teachers’ professional judgements in schools. A copy of the specification issued to data providers can be found here.

Teacher professional judgements of achievement of a level are based on all of the evidence collected by teachers during the ongoing assessment of children and young people’s learning. A wide range of evidence is collected in a variety of ways. This includes observing learners at work, assessing their work in class, standardised assessments and assessing children and young people’s knowledge and understanding by talking to them about their learning.

The Achievement of CfE Level census date was Monday 10 June 2019. Data was submitted by local authorities to Scottish Government by Friday 30 August 2019. A process of quality assurance between Scottish Government, local authorities and schools occurs before the production of these statistics.

9.1.2 Coverage

The data covers pupils in Primary 1, Primary 4, Primary 7 and Secondary 3 in mainstream schools and all pupils based in special schools/units. Data was provided for 229,255 pupils. Teacher judgements cover the four organisers of reading, writing, listening and talking and numeracy. Mainstream pupils for whom the teacher has been unable to make a professional judgement are not included in the published results (less than one per cent of pupils).

A very small percentage of children have long-term significant and complex additional support needs that mean that it is unlikely they will progress through the CfE levels during their time in education. These children are included within the data as ‘pupil following individual milestones’.

There has been continual improvement in the level of coverage for special schools and standalone special units. In 2015/16, the coverage was approximately 70 per cent. Since 2016/17 the approximate coverage has been consistently above 95 per cent.

9.1.3 Methodology

9.1.3.1 Gaelic medium education

Pupils based in Gaelic medium primary schools/classes will learn and develop their literacy and numeracy skills in both the medium of Gaelic and English.

Pupils in Primary 1 Gaelic medium primary schools will generally be immersed in developing their reading, writing, listening and talking and numeracy skills in the medium of Gaelic.

From around Primary 3, children based in Gaelic medium primary schools/classes will also start to develop their skills in reading, writing and listening and talking in the English language. It is expected that by the end of Primary 7, most children who have been based in Gaelic medium primary schools/classes should be achieving Second Level in reading, writing, and listening and talking in both Gàidhlig and English.

For pupils based in Gaelic medium primary schools/classes, the following data is collected and published:

	P1	P4	P7	S3
Gàidhlig Reading	√	√	√	√
Gàidhlig Writing	√	√	√	√
Gàidhlig Listening and Talking	√	√	√	√
English Reading			√	√
English Writing			√	√
English Listening and Talking			√	√
Numeracy		√	√	√

Achievement of CfE levels in Gàidhlig can be found in Chapter 5.

9.1.3.2 Pupils based in special schools / standalone special units

Special schools and standalone special units cater for children of all ages. The information gathered as part of this return does not include a specific stage for these pupils (i.e. they are simply recorded as being a pupil based in a special school or standalone special unit). Therefore, it is not possible to calculate the percentage of pupils who have achieved the CfE level relevant to their stage.

The data reported shows the overall picture of CfE levels that have been achieved for pupils based in special schools and standalone special units. These data are not included in the headline figures; they are reported in Chapter 6.

It has been reported that some pupils in special schools and standalone special units were recorded as ‘Not Assessed’ when these pupils are in fact working towards national qualifications and out of scope of this collection on Broad General Education performance. During 2019 we consulted with key stakeholders, including Education Scotland, special schools and standalone special units and local authorities, regarding how these pupils should be appropriately recorded. Based on their feedback we intend to introduce a new category for future Achievement of Curriculum for Excellence Levels data collections. The definition for this is:

For standalone special schools and units, use code 97 if:

(a) a pupil is in the senior phase and,

(b) the pupil is being taught at least one qualification, other award, or work- based learning, at SCQF Level 1 and above and,

(c) teachers would not normally make CfE level judgements for pupils in the senior phase.

9.1.3.3 P1, P4, P7 combined and Literacy variable

The publication tables include two derived variables: P1, P4 and P7 combined and literacy.

P1, P4 and P7 combined figures have been available on the school level dashboard since 2015/16 to provide data for small schools (where data for individual stages would be disclosive). P1, P4 and P7 combined figures are available at local authority and national level for comparison purposes.

The literacy variable has been created from the literacy organisers: reading, writing and listening and talking. A pupil is deemed to have achieved the expected level in literacy if they have achieved the expected level in all three literacy organisers: reading, writing, listening and talking. Pupils will not be included in this calculation if ‘Not Yet Assessed’ has been reported in one or more of the organisers. This literacy variable is used within the National Improvement Framework (NIF) basket of measures of the attainment gap.

9.1.3.4 Data matching

To reduce the burden on data providers, as per the Code of Practice for Statistics, pupil characteristic information was added to the Achievement of CfE Levels data by using previously collected data: Pupil Census 2018. The following variables were added:

• Ethnicity
• Additional Support Needs
• English as an Additional Language
• Scottish Index of Multiple Deprivation
• 6-fold Urban Rural Classification

Achievement of CfE Level data and Pupil Census data are initially matched based on Scottish Candidate Number (SCN) and school or, where an initial match is not made, based on SCN and other demographic information (e.g. sex and date of birth).

For 2018/19 a change was made to the approach taken to linking SIMD and Urban Rural Classification data.

Previously, where it was not possible to find an initial match between ACEL data and the Pupil Census using SCN and school, pupils appeared in the ‘Unknown’ SIMD and Urban Rural Classification categories. However for 2018/19, the Pupil Census was used to assign SIMD and Urban Rural Classification information whether the match was based on SCN and school, or SCN and other demographic information.

2016/17 and 201718 SIMD data has been revised on the same basis.

9.1.4 Data quality and quality assurance

9.1.4.1 Data quality and feedback from data suppliers

As part of the quality assurance process, feedback was sought from all data suppliers (local authorities and grant-aided schools) on the process of compiling the data and on factors which may affect data quality. Feedback was received from 31 out of 32 local authorities. The majority of data providers provided substantive feedback covering the assessment process followed by schools, their own quality assurance of the data and any outstanding concerns over the quality of the data.

A number of data suppliers indicated that they are fully confident in the ACEL data they have supplied. Most others indicated that their confidence levels were high and growing. This feedback is backed up by analysis of the 2018/19 data which shows increased consistency compared to earlier years.

However it should be noted that feedback from a small number of local authorities indicates that some concerns around confidence in data quality of Fourth Level judgements remain. Analysis of data on the achievement of Fourth Level indicates a greater range in results across local authorities than is observed at other Levels. This should be considered when interpreting these Fourth Level results.

9.1.4.2 Comparisons over time

This publication, and the associated supplementary tables, provide comparisons back to 2016/17, at a national and local authority level. When making such comparisons, it should be noted that both analysis of the data, and evidence provided to us by local authorities, suggests the robustness and consistency of the data has been increasing during this period.

In previous years these statistics have been labelled as ‘Experimental Statistics’ reflecting that they were new statistics in development. The Experimental Statistics label has been removed from this year’s statistics; for more information see Chapter 1.3.

Comparisons have not been made with data for 2015/16 - the first year of data collection – in this publication. Analysis of this data, alongside the 2016/17 data, highlighted inconsistencies between the two years. Due to this we do not recommend comparing 2015/16 data with data for subsequent years.

9.1.4.3 Comparing between local authorities

If making comparisons between local authorities we recommend keeping in mind the context of the authorities and their approach to assessment.

In particular, within some local authorities school pupils with complex needs may attend a special school or standalone special unit. It is not necessarily appropriate to assign these pupils to a specific stage and so pupils from special schools and standalone special units are excluded from the main analysis and are considered separately in Chapter 6. However, some local authorities do not have separate special schools or standalone special units and in these local authorities pupils with complex needs are integrated into their mainstream schools; these pupils have been included throughout this publication. This should be kept in mind when making comparisons between local authorities.

9.1.4.4 Assessment of children’s progress against CfE Levels

The expected standards under CfE were embedded in the experiences and outcomes from the outset; further to this Education Scotland published draft Curriculum for Excellence Benchmarks for literacy and numeracy in August 2016 in order to provide a more explicit and clear statement of standards. These standards were available to teachers ahead of the data collection for 2016/17. Final versions of the benchmarks were published in June 2017.

A national programme of Quality Assurance and Moderation has been put in place to provide more support and improve confidence and understanding amongst teachers, and in August 2017, Scottish National Standardised Assessments were made available for teachers to help inform their judgements.

9.2 School level data

School level results are also being released alongside this publication. They are available in the School Information Dashboard online via this link. Data will be published for all publicly funded primary and secondary schools subject to data protection limitations.

All school level results will be presented in ten per cent bandings (i.e. under 10 per cent, 10 per cent – under 20 per cent, … , 90 per cent or more). To prevent potential disclosure of information relating to individual pupils, any results relating to a grouping of 20 pupils or fewer will be suppressed. This means that around 19 per cent of primary schools and five per cent of secondary schools will have no information published for them.

As with the national and local authority level data, all school results include ‘Pupil following individual milestones’. This may have a particularly large impact on schools with an integrated special unit. Children who were recorded as ‘Not Assessed’ are not included in the calculations.

The data quality considerations described in Chapter 9.1.4 also apply to school level data. There is greater likelihood that an individual school’s results are affected by variations in assessment approach, socio-economic context and school size (for example) than is the case at the more aggregated local authority or Scotland level. If making comparisons between schools we recommend keeping in mind the context of the authorities and their approach to assessment.

9.3 Supplementary tables

The collection involves a large amount of data, which cannot be fully presented in this publication. This report seeks to highlight the key messages and give a flavour of the range of analysis possible. Detailed tables are published as supporting tables alongside this publication, and provide a fuller picture of the findings (http://www.gov.scot/ISBN/9781839603488). As with school level information, a disclosure control policy was applied to supplementary tables.

Within the supplementary tables for 2018/19 data, trend data has been included for the key measures. This includes the national overview, data by SIMD and local authority data.

9.4 Revisions policy

Data submissions for the latest year can sometimes identify required revisions for previously submitted data. Scottish Government will revise the previous years’ results at the time of publication of the latest year’s results. Data revisions must be received by October for inclusion in the next publication.

Achievement of Curriculum for Excellence Level 2015/16 data was revised and new supplementary tables published in 2016/17 following receipt of revised data from two local authorities and a minor coding revision for two special schools. (See Achievement of Curriculum for Excellence Level, 2016/17 for details).

Achievement of Curriculum for Excellence 2017/18 data was revised for the original publication of this report on 10 December 2019, on the new data matching basis (see Chapter 9.1.3.4), and to account for revised data received for one primary school.

Achievement of Curriculum for Excellence 2016/17 data was subsequently revised on 30 April 2020 to use the new data matching basis (see Chapter 9.1.3.4). This affects 2016-17 Achievement of Curriculum for Excellence Levels by SIMD in this report. It also affects 2016-17 Achievement of Curriculum for Excellence Levels by Urban-Rural classification which are not presented in this report but have been revised in the 2016/17 supplementary tables.

9.5 International and UK Comparability

England, Wales and Northern Ireland also report on literacy and numeracy performance based on teacher judgements, however, due to the differing education systems and curriculums, direct comparisons cannot be made.

England: https://www.gov.uk/government/organisations/department-for-education/about/statistics#statistical-collections

Wales: https://statswales.gov.wales/Catalogue/Education-and-Skills/Schools-and-Teachers/Examinations-and-Assessments

Northern Ireland: https://www.education-ni.gov.uk/topics/statistics-and-research/statistics

Scotland participates in the OECD’s triennial Programme for International Student Assessment (PISA) survey. This assessment is undertaken by 15 year-olds in over sixty countries, including all OECD countries, and as such is a key international benchmark of performance. The result of most recent PISA survey is available here: http://www.gov.scot/ISBN/9781839603730

9.6 Further information on attainment

There is a range of other information on the performance of Scotland’s school pupils.

The Scottish Government publishes analysis of school leaver attainment (qualifications) and destinations. The latest data is available at:

https://www2.gov.scot/Topics/Statistics/Browse/School-Education/Publications/Pub-SS-AIL

Further assessment and attainment information on schools can be found on the Scottish Government Education dashboards page.

The Scottish Qualifications Authority (SQA) publish attainment statistics which can be found at: https://www.sqa.org.uk/sqa/48269.8311.html

For updates on new Scottish Government education statistics, any forthcoming changes to Scottish Government education statistical publications and stakeholder consultations please register with Scotstat at the following website: http://www.scotland.gov.uk/Topics/Statistics/scotstat

Media enquiries about the information in this Statistics Publication Notice should be addressed to: Michael Berry, Tel: +44(0) 131 244 2967