User Guide to Recorded Crime Statistics in Scotland

14. Producer's Quality Assurance investigations and documentation

This section looks at the Scottish Government’s checks to consider whether the derived aggregated statistics are meaningful and seem correct.

14.1 Scottish Crime Recording Board

The Scottish Crime Recording Board (SCRB) was set up ‘to act as the guardian of, and ultimate decision maker on, issues related to the Scottish Crime Recording Standard’. It has three high level objectives: to support the production of accurate and objective statistics on crime in Scotland; to ensure crime data are comprehensive, consistent, transparent and trustworthy; and, to take into account the needs of both users and providers in the production of crime statistics, and to ensure this process is undertaken in a manner consistent with the Code of Practice for Official Statistics. The SCRB also provides support to HMICS in their Crime Audits.

Further details are available in the Scottish Crime Recording Board chapter.

14.2 QA Stage Five - Scottish Government Quality Assurance and publication of National Statistics on Recorded Crime in Scotland

Scottish Government statisticians receive data from Police Scotland on a quarterly basis. The data are in the form of aggregate information, showing the number of crimes and detections recorded each quarter by crime code for each local authority in Scotland.

The data supplied is based on an extract from Police Scotland’s live database. This data extract is referred to as MKQ (Made Known Quarterly) as the records are of crimes and offences that have been brought to the attention of the police. The dataset contains revisions to earlier quarters, most often due to details changing as crime investigations progress (see Stage Three), as well as figures for the latest quarter.

The data is used to produce the annual statistical bulletin series Recorded Crime in Scotland and to help with answers to ad-hoc queries, parliamentary questions and Freedom of Information requests. The statistical bulletin provides users with valuable insight into how the volume and nature of crime in Scotland changes over time. The publication provides detailed information about crime trends over the past ten years, and graphs with accompanying data showing trends back to 1971.

14.2.1 Quality Assurance processes

Whilst Scottish Government statisticians investigate the data provided in detail each quarter (the checking of data across the four quarters of the financial year period to ensure data consistency both as the year progresses and against the previous year’s quarterly data), their main focus is on the end-year aggregates for inclusion in the annual statistical bulletin. The Quality Assurance (QA) checks take two forms: an in-depth look at how the current year’s figures compared with those of the previous year; and a trends analysis looking for irregularities in time series data for broad aggregates.

14.2.1.1 Detailed current year checks

These checks are generated by the Scottish Government’s SAS system and used to process the long-term current and historic information for publication. The system compares the latest year’s figures to previous years for crime codes, by each local authority. It highlights instances where the data might warrant further investigation based on whether all the following conditions are met:

• The absolute percentage difference is greater than or equal to 25%,
• AND the absolute difference in number is greater than or equal to 20,
• AND the previous year’s figure was greater than or equal to 10.

14.2.1.2 Manual long term analysis

This check looks at longer term trends at a crime group level for each local authority. This analysis highlights unusual breaks in the trend, prompting further investigation.

This all generates a detailed list of queries on anything that may at first appear unusual in the data, that the Scottish Government sends to Police Scotland’s Analysis and Performance Unit (APU) for further investigation.

In response to the queries raised from the above checks, Police Scotland provide Scottish Government Statisticians with explanatory commentary based on their knowledge and intelligence, such as local initiatives and operations, which is then reflected where appropriate in the statistical publication.

During the QA checking process, it is possible for errors to be found in both the new reporting year’s data as well as the data for previous years. Where errors are identified in the new reporting year’s data, corrections are made. While we do not routinely revise figures for previous years, we are committed to correcting errors in the data and providing suitable explanations for any changes made to previously published data.

Following the conclusion of correspondence with Police Scotland, if Scottish Government Statisticians believe further investigation is required to understand the nature of any unusual or significant changes in the data, they have the option of raising this through the SCRB (for further information, see the SCRB chapter). This could lead to the SCRB asking the Crime Registrars to conduct a tailored audit of the relevant crime type to determine whether any change in recording practice had affected the final statistics. Further to this, statisticians would also seek to explore other sources of information to assess if they corroborate any new or developing trends in recorded crime (see the Corroborative analysis section below).

Once that stage of QA is completed, the data are then used to produce a set of data tables.

14.2.1.3 Revisions analysis

The Recorded Crime in Scotland National Statistics bulletin is based on snapshot information extracted immediately following the year to which they relate (i.e. the 2020-21 statistics will be based on data extracted and submitted by Police Scotland in April 2021). By comparing these figures with the equivalent snapshot in earlier years, statisticians can ensure that no systemic bias is introduced, despite any actual change in the data. There are therefore no planned revisions to report for future releases of the National Statistics – notwithstanding that corrections to past data will always be made or highlighted to users where an error is discovered.

The bulletins include analysis on the extent of revisions made to recorded crime data following its original submission to Scottish Government Statisticians by Police Scotland. This compares the crime and offence data extracted by Police Scotland in the year it was initially extracted (i.e 2020-21 data extracted in April 21 and published in September 2021) with the same set of data submitted one year later (i.e. 2020-21 crime and offence data extracted in April 2022). For more information see the Revisions Policy chapter.

This analysis confirms that the extent of further amendment to police crime and offence records is minimal at the Scotland level – giving users confidence that the published statistics remain a sufficiently accurate measure of police recorded crimes and offences. This analysis is completed annually to ensure that the extent of further amendment to data once it has been submitted to the Scottish Government remains minimal at the Scotland level and consistent with earlier years.

It is important that the impact of the ongoing revisions described above continues to be monitored – as an aid to QA of the information. This relates both to Police Scotland’s management information reports and the Scottish Government’s National Statistics. Where relevant, explaining the extent of changes in the data is important to help user understanding, especially when considering changes over time.

14.2.1.4 Corroborative analysis

Within the bulletins on recorded crime, Statisticians will provide comparisons with other published data where this is felt to be of value to users. To date the main focus has been on comparisons with the Scottish Crime and Justice Survey.

a. Comparison with the Scottish Crime and Justice Survey (SCJS)

The Scottish Crime and Justice Survey (SCJS) provides a complementary measure of crime compared with police recorded crime statistics. The survey provides information on the criminal justice system, on people's experience of civil justice problems and people's perception of crime.

One of the key strengths of the SCJS is its ability to capture crimes that are not reported to, and subsequently not recorded by, the police. SCJS provides an estimate of all crimes committed and therefore, can be used in conjunction with Recorded Crime to estimate the proportion of crimes not reported to the police. The latest published SCJS data (relating to 2019-20) estimated that 40% of crimes, as defined by the SCJS, were reported to the police.

The Scottish Government’s Recorded Crime bulletin contains a section Overall trends - Police Recorded Crime and the SCJS. The analysis is also threaded throughout the Recorded Crime in Scotland commentary, ensuring that readers are reminded of the scope of the Recorded Crime in Scotland statistics as a set of the full picture of crime in Scotland.

The dual purpose of the analysis is to present a picture of crime in Scotland informed by both SCJS and Recorded Crime in Scotland (maximising value and minimising risk of misinterpretation), and also to highlight how we can use the two statistical products together as complementary sources i.e. describing the additional analysis that is possible, the results of that analysis and what it means.

b. Homicide

In addition to the Recorded Crime in Scotland bulletin, the Scottish Government also publishes an annual bulletin on Homicides in Scotland. The data used in this bulletin are collected separately from the data included in the Recorded Crime in Scotland bulletin where the data are extracted from police recording systems at different time points. In this bulletin, homicide cases are included against the year in which the incident that led to the homicide is first recorded by the police. This is not necessarily the year in which the victim dies, the year in which the perpetrator is brought to trial for the crime, or the year in which the case is finally disposed of by the courts. This leads to minor differences between the figures reported in the Recorded Crime in Scotland bulletin and the Homicide publication, as well as those reported by Police Scotland. This is explained in greater detail in the Recorded Crime in Scotland publication.

c. Hospital emergency admissions

The Recorded Crime in Scotland bulletins include analysis comparing the number of emergency hospital admissions for assault. These are published in the Unintentional Injuries statistical publication from Public Health Scotland, part of NHS Scotland.

It is not possible to make direct comparisons between the two sources (as not all cases of police recorded attempted murder or serious assault may include an emergency admission to hospital – treatment might only be required within the Accident & Emergency department or the complainer may not wish to seek medical advice). However, as these sources both generally relate to how Scotland’s emergency services respond to the most serious types of assault, it would be anticipated that both should show similar trends over time. The analysis in the bulletin confirmed this was the case – with the same trend of significant reductions in activity apparent for both the police and hospital admissions. This provided an additional level of assurance on the Police Scotland data regarding attempted murder and serious assault.

d. Fire-raising

The Recorded Crime in Scotland bulletins include analysis comparing the number of crimes for fire-raising collected by Police Scotland with data on the number of fires attended by the Scottish Fire and Rescue Service (SFRS). These are published in the SFRS statistical bulletin Fire and Rescue Statistics in Scotland (Incident data).

Whilst it is not possible to make direct comparisons between the two sources (as many fires are caused accidentally rather than as a result of a crime, and a fire may have ended before SFRS was required to attend), both relate to how Scotland’s emergency services respond to fire. As such it would be anticipated that both should show similar trends over time. The analysis in the bulletin confirmed this was the case – with the same trend of significant reductions in activity apparent for both the police and fire services. This provided an additional level of assurance on the Police Scotland data regarding fire-raising.

e. International or sub national comparisons

Reassurance may also be gained if similar trends witnessed in Scottish data are seen by other countries, particularly those in the rest of the United Kingdom (accepting that definitions and counting rules are likely to be different). Shared knowledge and collaboration can lead to greater intelligence and insight into the data and the trends they show.

Statistics for the recorded number of crimes in Scotland are not directly comparable with statistics collected in England and Wales for the recorded number of notifiable offences. This is mainly due to differences in the counting rules; for notifiable offences the counting system is, wherever possible, victim based rather than offence based. Further information is available in the Comparability across the UK chapter.

f. Other bulletins

The Scottish Government publishes companion statistical bulletins to Recorded Crime in Scotland that are based on individual level crime and offence data on a number of topics. Further information on these other sources of data are available within Chapter 22: Other Scottish Government publications using police data.

Future Analysis

Scottish Government statisticians will continue to scope out further sources of information as potential candidates for new corroborative analysis in future bulletins, as and when they become available.

14.3 Technical report on the comparability of recorded crime data

14.3.1 Switching source of Recorded Crime statistics from Legacy Force data to SCOMIS.

Prior to 2013-14, and with the establishment of Police Scotland, a new IT system (ScOMIS) was developed. This collated crime reports from across the variety of systems used by the eight legacy police forces into one central information management system and supplied the Scottish Government with recorded crime data. Previously, the Scottish Government collected recorded crime data from the eight legacy police forces, who in turn had extracted the data from their own systems.

To enable a full check of whether data extracted from ScOMIS were comparable with legacy force data, ScOMIS was populated with data for 2008-09 onwards. These data were then compared with the data previously submitted by the legacy forces.

The Scottish Government were satisfied that crime data, Groups 1 to 5, are comparable pre and post reform. This means there is no break in the time series for this data.

However the overall comparability for offences data, Groups 6 and 7, was affected by some of the issues relating to legacy force data and changes in recording practices. For Group 6 – Miscellaneous offences, all crime codes are fully comparable from 2008-09 onwards and overall comparisons for Group 6 – Miscellaneous offences should only be made for 2008-09 onwards. There are greater comparability issues for Group 7 – Motor vehicle offences and overall comparisons for Group 7 – Motor vehicle offences should not be made prior to 2013-14.

For further information see the Technical Report.

14.3.2 Switching source of Recorded Crime statistics from SCOMIS to SEBP.

14.3.2.1 Background

Between 2013-14 and 2019-20, the National Statistics on Recorded Crime in Scotland were produced using data extracted from a single Police Scotland IT system, called the Scottish Operational and Management Information System (ScOMIS). This data was then provided to the Scottish Government on a quarterly basis for the production of the National Statistics (see Annex 1 for more information).

In 2019, Police Scotland developed a new data repository called the Source for Evidence Based Policing (SEBP). This was designed, in part, to replace ScOMIS and therefore become the new official source of information for the recorded crime National Statistics.

At the March 2021 Crime Board, members approved the switch from ScOMIS to SEBP for the production of the National Statistics. This commenced with the 2020-21 reporting year.

Since then, Justice Analytical Services, with support from Police Scotland analysts, have completed a technical review of the impact this switch in sources had on the 2020-21 crime data, – with a summary provided below.

14.3.2.2 Methods

Police Scotland analysts provided a comparable dataset extracted from both ScOMIS and SEBP. These datasets included the number of crimes both recorded and cleared up, by local authority for the 2020-21 financial year. As these were drawn from a live operational database at a different time from the extract used for the National Statistics, the findings below are based on 2020-21 figures that will differ slightly from those presented elsewhere in the bulletin.

14.3.2.3 Total recorded crimes

There was a strong degree of alignment between ScOMIS and SEBP for recorded crime in 2020-21. The technical review found that a very small number of additional crimes (representing just 0.2% of all crimes recorded in 2020-21) were included in SEBP but not ScOMIS. Both sources also showed strong alignment in terms of clear-up rates for recorded crimes. In 2020-21, the clear-up rate recorded by ScOMIS was 56.6%, whilst for SEBP it was 56.7%, a difference of 0.1 percentage points. As such the impact on time-series analysis of changing sources to SEBP is negligible at the Scotland level.

This minimal difference largely related to some additional crimes of Vandalism etc. being recorded in SEBP rather than ScOMIS. As such the inclusion of those crimes in SEBP represents a small improvement to the administrative processes used by Police Scotland to capture crime data for statistical purposes.

14.3.2.4 Total recorded offences

There was also a strong degree of alignment between ScOMIS and SEBP for recorded Miscellaneous offences in 2020-21. This technical review found that a very small number of these cases (representing just 0.2% of all crimes recorded in 2020-21) were included in SEBP but not ScOMIS. As such the impact on time-series analysis of changing sources to SEBP for Miscellaneous offences is also negligible at the Scotland level.

The largest identified difference between the two sources was for Group 7 – Motor vehicle offences, where some offence types such as Unlawful use of a motor vehicle, Speeding etc. had additional offences recorded in SEBP but not ScOMIS. This discrepancy represented only 3% of all Motor vehicle offences recorded in 2020-21 and as such the alignment between the two sources was still considered to be good.

14.3.2.5 Summary of findings

As highlighted above, this 2020-21 technical review on the impact of changing from ScOMIS to SEBP, found strong alignment between both administrative systems for recorded crime. Some small discrepancies were identified, specifically around crimes of vandalism (some of which appear in SEBP but haven’t been drawn through within ScOMIS). Whilst they have a small to negligible impact at the national level, they are concentrated amongst particular crime types within specific local authorities and therefore may have a slightly greater impact in these areas.

Further information on this change in sources for the National Statistics, including the analysis underpinning the above summary, is available to users on request (JusticeAnalysts@gov.scot) .

14.4 Summary of Scottish Government policy and guidance on quality

The Scottish Government produces and publishes official statistics in accordance with the UK Statistics Authority’s Code of Practice for Statistics, and other relevant guidance, including the Government Statistical Service (GSS) quality guidance.

Recorded crime statistics are subject to assessment by the Office for Statistics Regulation, to ensure that the statistics meet the professional standards set out in the Code of Practice for Official Statistics. The code of practice covers a range of criteria, one field being the quality of statistics. For more information see the UK Statistics Authority chapter.

It is possible for errors to be found in previous years data during the QA checks. While figures are not routinely revised by the Scottish Government, errors in the data are corrected, and suitable explanations for any changes to previously published data are provided. The process for doing this is set out in the Scottish Government Official Statistics Policy - Revisions and Corrections.