Recorded Crime Statistics in Scotland: user guide

15. Risks, limitations and challenges of the recorded crime data

15.1 Main Risks

Ethical recording of crime is integral to modern policing and it is vitally important that crime recording and disposal practices are capable of withstanding rigorous scrutiny.

The HMICS 2020 Crime Audit stated that:

"Despite a slight increase in recent years recorded crime in Scotland remains at one of the lowest levels since 1974. A wide range of users rely on recorded crime statistics to monitor trends, to develop policy, and to carry out research. The public can use the statistics as a source of information to assess how safe their local area is and whether crime is decreasing or increasing. The police can use crime statistics to monitor trends and variations, ensuring that resources are deployed appropriately. It is essential that crimes are recorded accurately by Police Scotland, and that users have confidence in the crime statistics published by the Scottish Government."

The main risks are around users misunderstanding or misusing the Scottish Government National Statistics. However, there are also risks around the collection, collation and presentation of the data in the publications.

Recorded Crime in Scotland needs to be correctly understood and interpreted to avoid misuse, and there are risks and limitations of the data that could affect their application.

For users of recorded crime statistics, it is important to understand exactly what the data means and what its limitations are. The QA chapters are intended to increase users' understanding of how the data is collected and statistics are developed, which should in turn improve the credibility and trust in the data. Further detail on the users and uses of crime are available in the Users and uses of recorded crime statistics chapter.

This User Guide will be reviewed annually in order to ensure it remains both up-to-date and fit for purpose. Changes to this User Guide will be approved by the Scottish Crime Recording Board (SCRB).

15.2 Limitations of the statistics

There are limitations to the Recorded Crime in Scotland data as published, partly due to the limitations and risks associated with using Police Scotland administrative data:

• Recorded Crime in Scotland is a statistical report of Police recorded crime. It is important to note that this is not all crime. Some incidents are not reported to the police and so may not be recorded as crimes and only data from Police Scotland is included in the main findings and tables in the statistical bulletin. We do however include a chapter on the Scottish Crime and Justice Survey which adds additional data, to give a wider picture of all crime.
• The statistical return from which most of the figures in the Recorded Crime bulletin are taken is a simple count of the numbers of crimes and offences recorded and cleared up by Police Scotland at local authority level. One return is made for each local authority in Scotland and these are aggregated to give a national total.
• It is not currently possible to publish recorded crime data at a lower geography than this. However, information on a subset of crimes and offences are collected at a low area level (data zones) for the Scottish Index of Multiple Deprivation (SIMD). This currently includes crimes of violence; domestic house breaking; drugs crimes; minor assault; and crimes of vandalism;
• Collecting aggregated numbers of crimes and offences means there is no information about victims, perpetrators or about the individual crimes or offences and incidents. It is not at present feasible to publish recorded crime data with victim or perpetrator data included.
• It should be noted that the nomenclature of some crime codes do lend themselves to providing information about victims, such as Sexual assault which has separate crime codes for victims aged 16 and over, between 13 and 15 years, and under 13 years.
• Scottish Government statisticians have supplemented this limited detail about crimes by conducting manual reviews of crime records in targeted areas in order to improve understanding and gain valuable insight into specific types of crime, including Common Assault, 'Other sexual crimes', Fraud and most recently Cyber-crime.
• Only data from Police Scotland are included in the main findings and tables in the bulletin. Data on the total number of crimes and offences recorded and cleared up by the British Transport Police and the Ministry of Defence Police in Scotland are included in the Annex section of the bulletin.
• Amendments (such as the deletion of incidents found on investigation not to be criminal) will always arise after the end of the financial year and submission of data to the Scottish Government. However, new analysis on the extent of further amendment to police crime and offence records following their original submission, confirmed such changes to be minimal at the Scotland level. See Revisions Policy for more information.
• Crimes and offences are included against the year in which they are recorded by the police. This is not necessarily the year in which the crime or offence took place, the year in which the accused is brought to trial for the crime or offence, or the year in which the case is finally disposed of by the courts. This is an important distinction.
• Trends in Recorded Crime in Scotland do not always demonstrate the obvious outputs; for example, if Police Scotland were to carry out a campaign on domestic violence, the increased attention and reporting of this type of crime could imply a rise in domestic violence, whereas the reality may be a positive picture of increased police activity, ultimately resulting in less domestic violence.

15.3 Challenges in using administrative data

Data producers should be very clear with users about the issues around the use of administrative data for statistical purposes. There can be limitations arising from differences in definitions preferred in the statistical and operational situations, changes in the operational definitions and circumstances over time, or a lack of standardisation in data collection procedures, IT systems and differing local policies and priorities.

Possible challenges in using administrative data for statistical purposes with respect to Recorded Crime in Scotland statistics are shown in Table 15.3

Broadly these risks to the quality of the Recorded Crime in Scotland data can occur at:

• the data collection stage – what is entered into the police recording system and how it is categorised;
• at the transfer into SEBP for transmission to the Scottish Government;
• at the statistical analysis and report writing stage; or
• at the interpretation and use stage.

Potential risks in reporting and collecting crime data include under or over reporting, misclassification, processing errors, IT collation errors, analysis and interpretation errors.

Table 15.3: Challenges and mitigations of using administrative data

Risk

Lack of standardised application of data collection:

• inconsistencies in how divisions of Police Scotland interpret guidance
• differences in the use of local systems for recording crime
• the distortive effects of targets and performance management regimes
• differing local priorities, e.g. some police areas might require higher levels of accuracy for certain crime types or information, but less so for other aspects that are important to the Scottish Government

Mitigating action/people

Data is entered according to the Code of Ethics.

Crime managers check their team's data, crime registrars provide consistent advice and next level check, HMICS audit the data and provide feedback.

APU provide oversight of collection.

Risk

Variability in data suppliers' procedures:

• Police Scotland may not have direct control over the development of guidance for data entry
• local checking of the data can be variable and might not identify incorrect coding or missing values
• local changes in policy could impact on how the data are recorded or on the coverage of the statistics

Mitigating action/people

Crime registrars advise on the SCRS and Counting Rules.

Crime managers provide first level of local checking, with crime registrars providing independent review.

Focus on certain crime types in certain areas or divisions may be reflected in crime reports. Police Scotland, with a single management structure should reduce the variability.

Risk

Quantity of data suppliers:

• large number of police officers entering the data, spread geographically.
• there are many data collectors (police officers in local authorities) providing their data to an intermediary organisation for supply to the Scottish Government via APU.

Mitigating action/people

The single Police Scotland, with a single management structure should reduce the risk, improving as Police Scotland beds in more.

Risk

Complexity and suitability of administrative systems:

• administrative datasets can be complex containing large numbers of variables; it takes time, and therefore resource, to extract the data required by the Scottish Government
• data collation can be hampered by IT changes in Police Scotland
• data might need to be manipulated by Police Scotland to meet the structural requirements of the Scottish Government, leading to potential for errors

Mitigating action/people

Recorded crime has been published for many years and the relationship between the administrative data and the published outputs are well established. However, IT changes need to be carefully managed, particularly with the proposed introduction of a new national IT system for Police Scotland.

Risk

Public perceptions:

• lack of knowledge about use of personal data for statistical purposes
• concern that personal data should be sufficiently anonymised and secured

Mitigating action/people

The Scottish Government publication contains advice to data users about the limitations of the data. Police Recorded Crime data is provided at local authority level and contains no personal data.

15.4 Influence of Targets

The possible influence of targets on recorded crime statistics is a potentially significant issue.

Police Scotland has developed an outcomes focused performance framework to measure progress against delivery of the policing priorities outlined in the Annual Plan. The performance framework provides a mechanism to demonstrate progress towards their outcomes. Gathering data, narrative and insights together provides Police Scotland with an evidence base that demonstrates how they are achieving their outcomes.

It has been recognised by Police Scotland and its many scrutiny agencies, including public scrutiny, that targets could be a likely source of bias in recording crime. There is potential for such targets to bias data collection, and targets and inputs may be interpreted differently in different areas. These risks are recognised and acknowledged by those involved in the production of crime statistics, and to counter some of the potential biases that could be introduced into the data, there are layers of independent audit of recorded crime data.

Mitigating action has been introduced to protect the accuracy and quality of the underpinning data. The Police Scotland safeguards that exist are strongly promoted within Police Scotland to protect the data quality regardless of potential pressures (as described in the Operational context and administrative data collection chapter).

Given the perceived risk of targets introducing bias into crime recording, the independence of crime registrars is paramount. They are organisationally external to operational decision-making and therefore can ensure that the data is accurate to enable better, quicker and more consistent decision making on crime recording and interpretation of the SCRS.

The Scottish Government carries out comparisons of Recorded Crime in Scotland against other sources (as discussed under the Corroborative analysis section) to check for possible distortion effects of targets on Recorded Crime in Scotland, with sense checking underpinning the early checking of Recorded Crime in Scotland data provided from Police Scotland.

HMICS also stated in its Crime Audit 2016 that:

"Throughout the UK, there has long been concern that quantitative performance frameworks and targets, such as the one used by Police Scotland in its first three years of operation, may unduly influence crime recording decisions. In our audit, as in 2014, we found no overt evidence of performance targets affecting crime recording. Indeed, our 2016 audit results for violent crime suggest that any performance pressures that may have existed have eased further." In its Crime Audit 2020 HMICS reported no change to this stated position.

15.5 Proportionality of Approach

To ensure that the scale of investigation and documentation for the Recorded Crime in Scotland statistics are proportionate to the statistical output, we used the 'Risk/profile matrix' proposed in the OSR's Administrative Data QA Toolkit. This section describes the analysis behind the decision to set the level of assurance^[5] of Recorded Crime in Scotland as A3.

The level of assurance fundamentally influences the nature of the activities required for QA of the recorded crime data and is determined by considering the public interest profile of the statistics and the degree of concern about the data quality.

The assessment is that Recorded Crime in Scotland are level 3, i.e. statistics of potentially high data quality concern and high public interest. However, this level is also relevant where the data quality concerns are less but there is a high public interest in the statistics.

Recorded Crime in Scotland statistics are of high public interest and rank in the "higher" category of the public interest dimension.

In terms of the data quality dimension, the UKSA have noted that "in Scotland, there is less direct evidence of data problems at the operational level^[6]", and so the data quality concerns may be considered to be medium level. Nonetheless, in the current climate of concern around police recorded data elsewhere in the UK, they have been rated as the higher level of concern. The high profile nature of these statistics ensures they are likely to remain at A3 for the foreseeable future.

The consequence of the A3 risk level is that, as the statistical producer, the Scottish Government must carry out a comprehensive assurance and audit approach.

The Scottish Government will continue to monitor the profile and quality of these statistics with Police Scotland and other justice partners, and will inform users of any developments in this area, through the SCRB.

Level of concern over data quality	Public Interest Profile: importance for informing decisions
	Lower	Medium	Higher
Lower level ('low')	A1	A1/A2	A1/A2
Medium ('medium')	A2	A2	A2/A3
Higher level ('high')	A3	A3	Recorded crime statistics A3