Guide to basic quality assurance in statistics

Planning of data quality assurance

QA of the data we use is a fundamental part of ensuring that:

• data is used in an appropriate way
• the risk of errors in our statistics is minimised

Errors in published statistics can have serious consequences:

• important decisions may be taken based on incorrect information
• the trust of internal and external customers in statistics produced may be affected
• an error may have a significant political impact (for example, if a correction affects the achievement of a government target)

If an error does occur, correction of the error is likely to require a considerable amount of extra effort from the statistical staff involved, and others.  Correction of errors should adhere to the Scottish Government corporate revisions policy and steps you take may include:

• correcting the publication online
• adding an explanation of the error to the website
• informing Ministers and other customers of the mistake

Although there is always some risk that errors may occur, it is very important that sufficient time is set aside to quality assure the data before analysis work is started. There should be a clear plan of the quality assurance that is to be carried out, often drawing on previous experiences, and this should be used to decide on the time required (ideally putting in some slack in case of delays).

A review of the processes and problems encountered last time the data were analysed and published may help prioritise work, and may prevent the repetition of mistakes.

It is worth discussing what issues there were last time a piece of work was carried out, and what the impact of these issues was. Input from analytical, policy and other colleagues at the start is better than being told at the end ‘we always have problems with that data, with the interpretation, with the policy impact etc’.

It is also important to make it clear to others involved why the QA process is important and how long it is expected to take.

Official and National Statistics have pre-announced publication dates.  However, it is important to avoid the situation where, if the data arrives late and, in order to publish the statistics on time, quality assurance checks are reduced (as illustrated below), increasing the risk of errors.

To minimise the risk of data arriving late, it is important not to just expect the data to arrive on the date originally agreed with the data provider.

It may be helpful to keep in touch with the data provider on a regular basis to ensure that there are no unforeseen problems that may affect when the data will be received (for example, due to technical problems or lack of staff).

It may also be possible to see an early version of the data to obtain a feel for the format the final dataset will arrive in, and possibly carry out some basic checks.

For external contractors it is also worth checking to see if there is a contractual agreement specifying when the data will arrive and what format it will be in.

If data are received late, options include:

• pulling in extra people to enable the checks to be carried out in a shorter period (where resources are available)
• delaying the publication (allowed if there is a very clear, non-political, reason – for example, if it is unavoidable that data will arrive late from external data providers). The Office of the Chief Statistician can advise on the steps to take if a publication date is to change.