Achievement of Curriculum for Excellence (CfE) Levels 2020-21

Chapter 8: Background notes

8.1 Sources, coverage, methodology and quality assurance

8.1.1 Sources

The data included in the publication are provided to Scottish Government by local authorities and grant-aided schools. Independent schools are not included in the data collection. The assessments of children's progress are based on teachers' professional judgements in schools. A copy of the specification issued to data providers is available at the Scottish Government website ( Scottish Exchange of Data: achievement of Curriculum for Excellence levels).

Teacher professional judgements of achievement of a level are based on all of the evidence collected by teachers during the ongoing assessment of children and young people's learning. A wide range of evidence is collected in a variety of ways. This includes observing children and young people at work, assessing their work in class, standardised assessments and assessing children and young people's knowledge and understanding by talking to them about their learning.

The Achievement of CfE Level census date was Monday 14 June 2021. Data was submitted by local authorities to Scottish Government by Friday 27 August 2021. A process of quality assurance between Scottish Government, local authorities and schools occurs before the production of these statistics.

8.1.2 Coverage

The data collected typically covers all pupils in Primary 1, Primary 4, Primary 7 and Secondary 3 in mainstream schools and all pupils based in special schools/units. However, for 2020/21 data was not collected for Secondary 3 or special school pupils. Data was provided for 168,556 pupils. Teacher judgements cover the four organisers of reading, writing, listening and talking and numeracy. Pupils for whom the teacher has been unable to make a professional judgement are not included in the published results. In 2020/21 teachers were unable to make a judgement for around 0.35% of primary school pupils across each organiser. This is slightly higher than in 2018/19 when teachers were unable to make a judgement for around 0.30% of primary school pupils across each organiser.

A very small percentage (less than one per cent) of children have long-term significant and complex additional support needs that mean that it is unlikely they will progress through the CfE levels during their time in education. These children are included within the data as 'pupil following individual milestones'.

8.1.3 Methodology

8.1.3.1 Pupils based in special schools or standalone special units

Data for special schools and standalone special units was not collected in 2020/21. For more information see Chapter 1.3. The text below refers to the data that would typically be collected and which was collected and published for 2016/17, 2017/18 and 2018/19.

Special schools and standalone special units cater for children of all ages. The information gathered as part of the ACEL return does not include a specific stage for these pupils (i.e. they are simply recorded as being a pupil based in a special school or standalone special unit and not as being in P1, P4 etc.). Therefore, it is not possible to calculate the percentage of pupils who have achieved the CfE level relevant to their stage.

The data (usually) reported shows the overall picture of CfE levels that have been achieved by pupils based in special schools and standalone special units. These data are not included in the headline national and local authority figures; they are reported separately.

It has been reported that some pupils in special schools and standalone special units were recorded as 'Not Assessed' when these pupils are in fact working towards national qualifications and out of scope of this collection on Broad General Education performance. During 2019 we consulted with key stakeholders, including Education Scotland, special schools and standalone special units and local authorities, regarding how these pupils should be appropriately recorded. Based on their feedback we have introduced a new category (code 97) for future Achievement of Curriculum for Excellence Levels data collections. The definition for this is:

For standalone special schools and units, use code 97 if:

(a) a pupil is in the senior phase and,

(b) the pupil is being taught at least one qualification, other award, or work- based learning, at SCQF Level 1 and above and,

(c) teachers would not normally make CfE level judgements for pupils in the senior phase.

8.1.3.2 Derived variables: P1, P4, P7 combined and literacy variable

The publication tables include two derived variables: P1, P4 and P7 combined and literacy.

P1, P4 and P7 combined figures have been available on the school level dashboard since 2015/16 to provide data for small schools (where data for individual stages would potentially reveal information about individual pupils). P1, P4 and P7 combined figures are available at local authority and national level for comparison purposes.

The literacy variable has been created from the three literacy organisers: reading, writing and listening and talking. A pupil is deemed to have achieved the expected level in literacy if they have achieved the expected level in all three literacy organisers. A pupil is deemed not to have achieved the expected level in overall literacy if they have failed to achieve the expected level in one or more of the individual organisers. If a pupil has not been assessed in one or more of the organisers ('Not Assessed'), their results are not included in calculation of the overall literacy variable.

These two derived variables are used in the key measures in the National Improvement Framework (NIF):

• Percentage of P1, P4 and P7 pupils combined achieving expected Level in Literacy;
• Percentage of P1, P4 and P7 pupils combined achieving expected Level in Numeracy;
• Percentage of S3 pupils achieving Third Level or better in Literacy and;
• Percentage of S3 pupils achieving Third Level or better in Numeracy.

The last two measures are not available in this report as data for S3 pupils were not collected in 2020/21; for more information see Chapter 1.3.

8.1.3.3 Gaelic medium education

Pupils based in Gaelic medium primary schools/classes will learn and develop their literacy and numeracy skills in both Gaelic and English.

Pupils in Primary 1 Gaelic medium primary schools will generally be immersed in developing their reading, writing, listening and talking and numeracy skills in the medium of Gaelic.

From around Primary 3, children based in Gaelic medium primary schools/classes will also start to develop their skills in reading, writing and listening and talking in the English language.

It is expected that by the end of Primary 7, most children who have been based in Gaelic medium primary schools/classes should be achieving Second Level in reading, writing, and listening and talking in both Gàidhlig and English.

For pupils based in Gaelic medium primary schools/classes, the following data was collected and published in 2020/21:

	P1	P4	P7
Gàidhlig Reading	✓	✓	✓
Gàidhlig Writing	✓	✓	✓
Gàidhlig Listening and Talking	✓	✓	✓
English Reading			✓
English Writing			✓
English Listening and Talking			✓
Numeracy		✓	✓

S3 data were not collected for 2020/21; for more information see Chapter 1.3

Achievement of CfE levels in Gàidhlig can be found in Chapter 5.

8.1.3.4 Data matching

To reduce the burden on data providers, as per the Code of Practice for Statistics, pupil characteristic information was added to the Achievement of CfE Levels data by using previously collected data from the Pupil Census 2020. The following variables were added:

• Ethnicity
• Additional Support Needs
• English as an Additional Language
• Scottish Index of Multiple Deprivation
• 6-fold Urban Rural Classification

Achievement of CfE Level data and Pupil Census data were initially matched based on Scottish Candidate Number (SCN) and school or, where an initial match is not made, based on SCN and other demographic information (e.g. sex and date of birth).

8.1.4 Data quality and quality assurance

8.1.4.1 Data quality and feedback from data suppliers

As part of the quality assurance process, feedback was sought from local authority data suppliers on the process of compiling the data and on factors which may affect data quality. Feedback was received from 27 out of 32 local authorities. The majority of data providers provided substantive feedback covering the assessment process followed by schools, their own quality assurance of the data and any concerns over the quality of the data.

In general most local authorities indicated that they were confident in the data quality with a number citing further improvements planned for future data collections. A small number of authorities noted that they still had concerns about the consistency of judgements across schools within the authority and, therefore, across the country as a whole.

Many authorities noted that the COVID-19 pandemic had had an impact on the data submitted this year. Whilst most local authorities said that they were confident that the data provided was an accurate reflection of the assessment that had been possible a number reported that the pandemic meant teachers had reduced opportunities to gather evidence on which to make judgements. This was due, for example, to school closures and self-isolation of pupils and teachers.

The pandemic has also impacted on the nature and quantity of the moderation exercises (which help develop a shared understanding of standards and expectations) which some schools and local authorities have been able to conduct. In particular local authorities mentioned that face-to-face moderation could not take place and that moderation exercises were often limited to in school rather than between schools. This may have impacted on the consistency of some of the teacher judgement data this year.

8.1.4.2 Official Statistics

From 2015/16 to 2017/18 Achievement of CfE Level publications were Official Statistics labelled as 'Experimental Statistics'. This was to reflect the fact that these were new statistics in development, published to involve users and stakeholders in their development and build in quality and understanding at an early stage. From 2018/19 Achievement of Curriculum for Excellence Level statistics are no longer labelled as Experimental Statistics.

The factors that led to the removal of the Experimental Statistics label can be found in the Evidence Paper.

8.1.4.3 Comparisons over time

This publication, and the associated supplementary tables, provide comparisons back to 2016/17, at a national and local authority level. When making such comparisons, it should be noted that both analysis of the data, and evidence provided to us by local authorities, suggests the robustness and consistency of the data has changed during this period.

Comparisons have not been made with data for 2015/16 - the first year of data collection – in this publication. Analysis of this data, alongside the 2016/17 data, highlighted inconsistencies between the two years. Due to this we do not recommend comparing 2015/16 data with data for subsequent years.

The time period covered by the 2020/21 statistics means that the results will be affected by the coronavirus (COVID-19) pandemic. This is likely to have had an impact both on learning and teaching (and therefore the CfE levels achieved) and on the quality of the data submitted for some schools. This should be kept in mind when making comparisons between years.

Developments in the ACEL data collection and publication since its introduction

Chart: ACEL Development

Flow chart showing the developments in the ACEL data collection and publication since its introduction

2015/16

1st year of data collection

Analysis of 2015/16 data, alongside the 2016/17 data, highlighted inconsistencies between the two years.

Due to this we do not recommend comparing 2015/16 data with data for subsequent years and therefore 2015/16 data is not presented in this publication or the associated supplementary tables.

2016/17 – 2017/18

Experimental Statistics

These were new statistics in development, published to involve users and stakeholders in their development and build in quality and understanding at an early stage.

The robustness and consistency of these statistics increased over time. This should be kept in mind when making comparisons between years

2018/19

Official Statistics

From 2018/19 it was decided that ACEL statistics would no longer be labelled as Experimental Statistics.

The factors that led to the removal of the experimental label can be found in a paper, available here (http://www.gov.scot/ISBN/9781839603488).

The robustness and consistency of these statistics increased over time. This should be kept in mind when making comparisons between years.

2019/20

Data collection was cancelled

The ACEL collection and publication was cancelled in 2019-2020 due to the difficulties in collecting data whilst schools were closed due to COVID-19.

2020/21

Primary pupils data collection only

The ACEL publication cover Primary school children (P1, P4 and P7) only in 2020-2021.

Secondary school and special school data was not collected due to other pressures on these schools including implementation of the SQA National Qualifications Alternative Certification Model which was used to award National 5s, Highers and Advanced Highers in 2021.

The time period covered by the 2020/21 statistics means that the results will be affected by the coronavirus (COVID-19) pandemic. This should be kept in mind when making comparisons between years.

8.1.4.4 Comparing between local authorities

If making comparisons between local authorities we recommend keeping in mind the context of the authorities and their approach to assessment.

In particular, in some local authorities, pupils with complex needs are integrated into their mainstream schools; these pupils have been included throughout this publication. However, within other local authorities school pupils with complex needs may attend a special school or standalone special unit. It is not necessarily appropriate to assign these pupils to a specific stage, and so, these pupils are usually treated separately in this publication. This year data was not collected for pupils in standalone special schools. The treatment of pupils with complex needs across different local authorities should be kept in mind when making comparisons between local authorities, or between individual schools.

8.1.4.5 Assessment of children's progress against CfE Levels

The expected standards under CfE were embedded in the Experiences and Outcomes from the outset; further to this Education Scotland published draft Curriculum for Excellence Benchmarks for literacy and numeracy in August 2016 in order to provide a more explicit and clear statement of standards. These standards were available to teachers ahead of the data collection for 2016/17. Final versions of the benchmarks were published in June 2017.

A national programme of Quality Assurance and Moderation has been put in place to provide more support and improve confidence and understanding amongst teachers, and in August 2017, Scottish National Standardised Assessments were made available for teachers to help inform their judgements.

8.2 School level data

School level results are also being released alongside this publication. They are available in the School Information Dashboard online via this link. Data will be published for all publicly funded primary schools subject to data protection limitations.

All school level results will be presented in ten per cent bandings (i.e. under 10 per cent, 10 per cent – under 20 per cent, … , 90 per cent or more). To prevent potential disclosure of information relating to individual pupils, any results relating to a grouping of 20 pupils or fewer will be suppressed. This means that around 19 per cent of primary schools will have no information published for them.

As with the national and local authority level data, school results include 'Pupil following individual milestones'. This may have a particularly large impact on schools with an integrated special unit.

Children who were recorded as 'Not Assessed' are not included in the calculations.

The data quality considerations described in Chapter 8.1.4 also apply to school level data. There is greater likelihood that an individual school's results are affected by variations in assessment approach, socio-economic context and school size (for example) than is the case at the more aggregated local authority or Scotland level. If making comparisons between schools we recommend keeping in mind the context of the authorities and their approach to assessment.

8.3 Supplementary tables

The collection involves a large amount of data, which cannot be fully presented in this publication. This report seeks to highlight the key messages and give a flavour of the range of analysis possible. Detailed tables are published as supporting tables alongside this publication, and provide a fuller picture of the findings (see supplementary tables). As with school level information, a disclosure control policy was applied to supplementary tables.

Within the supplementary tables for 2020/21, trend data for 2016/17 to 2020/21 has been included for pupil characteristics. This includes data by Additional Support Needs, English as an Additional Language, urban rural classification, sex, ethnicity and Gaelic education.

8.4 Revisions policy

Data submissions for the latest year can sometimes identify required revisions for previously submitted data. Scottish Government will revise the previous years' results at the time of publication of the latest year's results. Data revisions must be received by October for inclusion in the next publication.

Achievement of Curriculum for Excellence Level 2015/16 data was revised and new supplementary tables published in 2016/17 following receipt of revised data from two local authorities and a minor coding revision for two special schools. (See Achievement of Curriculum for Excellence Level, 2016/17 for details).

Achievement of Curriculum for Excellence for 2016/17 and 2017/18 data was revised in 2019, to use the new data matching basis introduced in 2018/19 and to account for revised data received for one primary school for 2017/18.

With additional time series data now being published (see Chapter 8.3), the opportunity has been taken to revise 2016/17 data by ethnicity, English as an Additional Language and Additional Support Needs. These revised figures take account of an improvement made to the data linkage process between ACEL data and Pupil Census data which was introduced for the 2017/18 publication.

8.5 Rounding

Figures used in the commentary of this report are based on the unrounded data which can be found in the supplementary tables. This means that they may not always match with figures that are derived using the rounded data displayed in tables and charts.

8.6 International and UK comparability

England, Wales and Northern Ireland also report on literacy and numeracy performance based on teacher judgements, however, due to the differing education systems and curriculums, direct comparisons cannot be made.

England: Statistics at the Department for Education

Wales: Examinations and assessments

Northern Ireland: Department of Education Statistics

Scotland participates in the OECD's triennial Programme for International Student Assessment (PISA) survey. This assessment is undertaken by 15 year-olds in over sixty countries, including all OECD countries, and as such is a key international benchmark of performance. The result of most recent PISA survey is available at the Scottish Government website ( Programme for International Student Assessment (PISA) 2018: highlights from Scotland's results).

8.7 Further information on attainment

A range of other information on the performance of Scotland's school pupils is available.

The Scottish Government publishes analysis of school leaver attainment (qualifications) and destinations. The latest data is available at the School education statistics website.

Further assessment and attainment information on schools can be found on the Scottish Government Education dashboards page.

The Scottish Qualifications Authority (SQA) publish attainment statistics which can be found at the SQA website.

For updates on new Scottish Government education statistics, any forthcoming changes to Scottish Government education statistical publications and stakeholder consultations please register with Scotstat at the following website: ScotStat Register: guidance - gov.scot (www.gov.scot)