Scottish economy statistics: quality assurance

The accuracy of statistical outputs in the general statistical sense is the degree of closeness of estimates to the true value.

Sampling Errors

Due to the numerous data sources used in the production of economic statistics for Scotland and the complexities in the range of methodologies adopted, it is impossible to produce reliable estimates of the standard errors of published estimates. However, OCEA produce revisions tables in all its main outputs to highlight where and why estimates have been revised.

For most of the OCEA products (GERS, I-O, GDP, IME, RSIS and SNAP) the data are collected by other government departments (mainly ONS, HMT and HMRC). Where the data are collected for UK purposes, but the Scottish extract is not sufficient for similar use in Scotland, top-up surveys are organised. The Scottish Government currently pays for survey boosts of the Monthly Business Inquires, the Business Register Employment Survey, the Annual Business Inquiry and the Labour Force Survey.

For the GCS, where most data are collected in a Scottish Government survey, probability sampling is used, and the sample is stratified in favour of larger companies and known exporters.

For all data processing outliers are investigated and removed where appropriate.

Coverage Errors

For all OCEAs statistical outputs where sample information is weighted to the total population, the Interdepartmental Business Register (IDBR) is used as a sampling frame.

OCEA currently receive quarterly excerpts of this relating to all local units operating in Scotland. As this is a dynamic, administrative source, it is not always accurate and up to date. OCEA statisticians make adjustments to the register for known inaccuracies.

More work should be carried out to establish a reliable business register for Scotland which can be adjusted back through time to allow better time-series analysis of weighted estimates on a comparable basis through time.

Measurement Errors

OCEA Statisticians carry out sense checks on all data collected locally and via other organisations. Measurement errors are routinely identified and appropriate action taken - follow up, deletion or imputation.

Non-response Errors

This is an issue in particular for the GCS. The questionnaires ask for details of exports of goods and services from Scotland to the Rest of the World and to the Rest of the UK. Many companies, especially large national and multinational companies, often have difficulty in isolation Scottish activities from UK-wide activities. Many companies supply a best guess where appropriate.

Processing Errors

These are rare. If the error results in a major change to a published figure, an erratum would be issues and the figures re-published. Users would be informed and a statement place on the website. This has not happened to date.

For minor errors, these would be routinely corrected at the next publication for regular statistics and details of the revisions would be explained, where appropriate, in the accompanying Statistics Publications Notice.

Modelling Assumptions and Associated Errors

OCEA adopt different models relating to the Scottish economy where full ranges of data are not available, or where data come from a variety of data sources, and these may be of varying quality.

The Input-Output Tables and Multipliers is a good example where significant value is added by combining supply and demand estimates, together with estimates of flows of goods and services between industries and final demand sectors, to form a more holistic analysis of the economy in Scotland for a particular year. This forms the basis of the multiplier analysis and moreover has been extended to form a carbon Assessment model.

Models, by their nature, involve some subjective judgement. Where possible, OCEA will explain to users what assumptions have been made and how these assumptions might impact on the analysis or end product.

The GERS methodology uses some modelling work in apportioning UK expenditure and revenues to Scotland. The detail underpinning these decisions is fully documented on the web and in the publication. A sensitivity analysis is also provided.

Seasonal Adjustment

For GDP, IME, RSIS and SNAP, outputs are produced on a seasonally adjusted basis. Component series are tested for seasonality. The X-12-ARIMA technique is used where appropriate to remove regular seasonal peaks and troughs so that the underlying trends and other features of the data are easier to identify.

Imputation

OCEA Statisticians often made professional decisions involving imputation, especially where missing values occur in samples which are weighted to form time series estimates.

Mistakes

Mistakes are rare, but when they occur, measures are taken to avoid similar mistakes in the future. These are more likely to occur when statistics are produced in a hurry or on an ad-hoc basis. Regular peer review of the main outputs limits the chance of mistakes of any significance being made.

Revisions

Planned revisions in OCEA follow standard, well-established and transparent procedures. All statistical outputs contain table of revisions to previously published figures, and the reason for the revisions is published in the Statistics Publication Notice.

OCEA have a published revisions policy.