Producing official statistics

Revisions and corrections

Aims

All Scottish Government official statistics are governed by the Code of Practice for Official Statistics. One of the principles of the code is to publish "a Revisions Policy for those outputs that are subject to scheduled revisions" and where applicable, to "correct errors discovered in statistical reports, and alert stakeholders promptly".

This document sets out the principles used by the Scottish Government Statistician Group when dealing with planned revisions and corrections. It is expected that each topic area will produce specific policies for their own statistics.

Revisions

Any revisions to statistics can be inconvenient to users and can undermine confidence and trust. However in some cases the user need for timely figures makes it desirable to release statistics on a provisional basis before they can be finalised making subsequent revisions necessary. This is often done on a regular basis and perhaps each time a series is published but can also arise from an urgent requirement to publish an official statistic. In other cases revisions to previously issued figures result from improvements in statistical methods, or due to the availability of more and fuller data. Usually such changes are planned in advance but occasionally work on the latest statistics leads to an urgent need to make revisions.

Our approach to revisions:

• A revisions statement for each output that is subject to regular scheduled revision will be published on the Scottish Government website.
• All figures which are expected to be revised will be clearly marked as provisional and an indication given of when revisions are expected to be published. If possible an indication of the possible scale and nature of likely revisions will be given.
• We will give as much notice as possible of changes to methodology, definitions or any other changes likely to lead to revisions along with an indication of their possible scale and nature.
• As far as possible we will consult users before making changes that affect statistics (for example, to coverage, definitions, or methods) or to publications.
• Revisions will be subject to the same arrangements for publication and pre-announcement as other releases. Notification of forthcoming revisions will be included on the Scottish Government forthcoming publications schedule. We will also notify key users by e-mail.
• We will provide a statement explaining the nature and extent of revisions at the same time as they are released.
• Ideally previous versions of publications and data tables containing unrevised figures will be replaced and archived. This may not always be feasible. As a minimum all unrevised figures will be clearly marked and contain a link to the revised figures.
• We will ensure that where time series are revised, or changes are made to methods or coverage, we produce consistent historical data, wherever possible, and user guidance. If it is not possible to produce consistent historical data then advice on potential effects will be given where possible.

Corrections

Errors in statistics can seriously undermine trust and confidence and can adversely affect policy decisions and debate. However careful decisions need to be made about the amount of time and effort involved in checking relative to the user need for timely statistics and required levels of accuracy for the main purposes. Systems should be designed to minimise the risk of processing error and assure the quality of source data and processed results.

Unfortunately, sometimes errors do happen in statistical outputs and protocols need to be put in place to ensure that our response is proportionate and swift. The Code of Practice for Official Statistics states that we should “correct errors in statistical reports, and alert stakeholders promptly”, but the mechanism for correcting mistakes and who to alert depends on the severity of the mistake.

Whilst the procedures which are appropriate in each case will vary, the basic data quality checks which are commonly carried out are being set out in guidance being produced by the Improving Quality and Dissemination Cross-Office Group and will be published shortly.

Our approach to corrections:

• The Code of Practice should not be interpreted in such a way as to inhibit quality assurance. Often policy colleagues and others with knowledge of the context will be able to identify errors that checking systems will not, so it is important to involve them.
• Classifying mistakes often involves subjective decision making. The branch head may want to consult their senior statistician and/or the Chief Statistician about the error and the consequences.

Matters which are considered:

• The importance of the statistic e.g. is it a headline figure or being used in significant ways?
• The size of the mistake particularly relative to any stated levels of accuracy
• The recipients of the mistake
• The consequences of the mistake – does it alter the main message of the statistics or any likely use?
• The response should be based on a consideration of the above matters.
• Statistical staff must ensure they have ascertained the full extent and nature of the error before issuing corrected figures.
• Minor errors will be corrected in the next edition of the publication. The correction will be made clear and reasons explained.

For other errors we will:

• Correct the material on the website, making clear that this has been done.
• Notify users of any substantial errors or errors which could affect their own work.
• Where errors have been identified that will take time to correct advance notice should be given if possible with the expected release date and an indication of possible scale.

In the case of major corrections we will also issue a formal Statistical News Release subject to the normal procedures for such releases.

Other than for very minor errors:

• Branch procedures will be amended to ensure that such a mistake does not recur.
• Where the error is in data provided by others, action should be taken to prevent recurrence. Where the provider is a public body it may be appropriate to involve policy colleagues and Ministers.
• Consideration should be given to publishing an explanation of the above action where this would assist public confidence and the future avoidance of error.

Responsibilities

The Head of Profession (Chief Statistician) has final responsibility for deciding on any action necessary to comply with these procedures. However, responsibility for the day-to-day implementation of the principles and processes outlined in this statement is vested in Heads of Branch and their line managers. Desktop instructions outlining branch procedures for the above processes and principles should be in place for every dataset held by the branch and branch heads will review these on a regular basis.

The Improving Quality and Dissemination cross-office group acts as a network and provides assistance with revisions and corrections queries. This is done via email correspondence and regular meetings.