User Guide to Recorded Crime Statistics in Scotland

13. Data suppliers' Quality Assurance principles, standards and quality checks

This section focuses on the quality assurance (QA) procedures that Police Scotland have in place. It also considers the role of the Scottish Police Authority (SPA) in holding Police Scotland to account, for example by scrutinising their performance data.

13.1 Initial Quality Assurance processes

13.1.1 Incidents

When incidents are reported by the public to Police Scotland, the Service Advisor and Controller will carry out a thorough check of all the information being submitted (or equivalent person if the incident is being reported at a police station). This then allows a Service Advisor and/or controller to make a grading decision on the severity of the incident – which will in turn dictate the police response. For the higher graded calls (i.e. those of a more severe nature that may require an emergency response) – a resource will be dispatched to deal with the incident. For the lower graded calls, where an immediate response is not critical, the call may be transferred to the Resolution Team, who will liaise with the caller and may arrange an appointment with an officer as necessary or record a crime direct.

In addition to the checks made at the time of the initial report, some divisions also have incident review officers who will assess incidents that have been phoned in to ensure information has been entered and recorded accurately. This tends to focus on high-risk areas, such as victim-based incidents (like violent or sexual crime).

An incident will remain open until it has been closed by the area control room or Resolution Team, for onward management within the local division. When an incident has been closed, it must show whether or not a crime has been committed. If a crime has taken place then a crime report must be raised, and cross referred to the original incident (and vice-versa). If no crime has occurred (and the incident has been closed as a no-crime incident) then the incident must contain sufficient information to dispel the initial inference of criminality. An incident that provides a disposal of ‘no complaint’ is insufficient to negate any initial suggestion at the time of reporting that a crime had occurred. In some cases the Officer attending may report back the details as to the outcome of the incident to the Control Room, to allow application of the decision directly onto the incident.

HMICS conducted an audit of call handling and logging in 2015, followed up by update reports in 2017 and 2018 and by a review of the early implementation of the Contact Assessment Model in 2019. Both HMICS and Police Scotland also carry out regular audits of crime-related incidents, to ensure information has been recorded accurately (See Stage 2) and that all crime-related incidents lead to the creation of an associated crime report (covered in more detail below). This work also includes sampling those incidents that were closed as non-crimes to ensure this decision was accurate and based on sufficient information to justify the decision not to crime.

13.1.2 Recorded Crime

Once a crime report has been established, this becomes the main vehicle for recording the details as investigations progress.

The crime report undergoes a significant amount of scrutiny between being raised and filing the report (i.e. closure of the crime report once investigation complete).

An officer first enters the details of the crime (manually directly onto the crime recording system, by telephone through dictation of the circumstances, or by mobile working device) into the crime recording system. These details include location, time, victim details such as gender, age and address, witness details, and various flags such as alcohol involved, weapons used etc.

Officers are required to complete the crime report as soon as possible, preferably by the end of their tour of duty. There can be circumstances when a crime report is delayed, however the Scottish Crime Recording Standard states that this delay should not extend beyond 72 hours unless there are exceptional circumstances. All delays in recording crime which extend beyond 72 hours must have an explanation detailed in the incident report.

Once the initial report is raised, it is reviewed by the Crime Management Unit (CMU). Each division has a CMU, consisting of a crime manager and a supporting team. The CMU reads the circumstances of the case and assesses whether additional information is required to determine what crime has occurred, or whether the number of crimes implied in the crime report have all been counted correctly. They may also consider whether the nature of the crime requires a different team to the on-scene officer to take forward further investigation of the case (for example sexual crime). For these incidents the CMU will forward the crime report to that team.

Whilst the CMU may change the crime type or number of crimes relating to a case – most often they will seek clarification from officers on specific details, to ensure crimes are recorded accurately. From the initial entry of the crime report into the crime recording system to the final sign-off, the CMU has the ability to dip into cases and flag up queries and reminders for the necessary information required to sign-off the crime report. Through this process dialogue and feedback will flow between the relevant officers, their supervisors and the CMU – as records are updated and investigations progress. This will continue until such time as the officer’s supervisor and/or CMU is satisfied that the crime report can be closed.

Each crime report retains an auditable trail of the queries raised by CMU as the case moves forward and officers’ responses to them. The CMU can ask the Crime Registrars (who audit the quality of recorded crime, see Section 2) to intervene as necessary, to ensure cases are appropriately closed and filed.

The points overleaf illustrate some of the more generic queries or comments that can be raised by the CMU as part of their role of quality assuring crime information.

In incidents which potentially include multiple crimes – the CMU will remind officers to ensure that information is appropriately cross-referenced with other relevant records, to help ensure the total number of crimes are counted correctly.

Similar to the above, the CMU will seek to tease out the precise number of crimes in cases where there may be more than one complainer or more than one type of crime. An example could be where a report of vandalism implies a second complainer beyond the main one referred to in the original report (through damage to two people’s property). In this case the CMU would seek clarification about what happened to this second person and whether a second crime had taken place. Many incidents can also contain multiple types of crime – such as vandalism, breach of the peace and assault, and the CMU will review these carefully to ensure all crime is being classified and recorded in line with the counting rules and standards.

In cases involving injuries to a complainer, the CMU will prompt and remind officers to include updates on medical treatment and prognosis - including where insufficient information on injuries has been included in the initial record. This is a critical QA function as the nature of assault-based injuries will determine whether an incident is statistically counted as a serious assault or a common assault).

The CMU also reminds officers to notify it if new information comes to light which may require changes to a crime’s classification. Further updates should be timely and complainers should be kept informed.

In a separate exercise, where a crime is cleared up (detected) a case officer from the case management team will review a final report on the case for the Procurator Fiscal – albeit they do not review the information in the actual crime report.

The above section demonstrates that throughout the process of recording incidents and crimes, a considerable amount of scrutiny takes place involving a large number of officers and dedicated staff, to ensure crimes are recorded consistently and ethically.

13.2 Proactive Systemic Data Checks

The primary standard for recorded crime data in Scotland is the Scottish Crime Recording Standard (SCRS). The Standard advises on the recording of crime and is supported by Counting Rules. The Counting Rules provide a national standard for the recording and counting of crimes and offences recorded by Police Scotland, known as 'recorded crime'. Further information is available in the SCRS chapter.

The SCRS also requires that regular audits of crime recording are carried out to check that incidents are recorded correctly and to check whether recorded crimes have been counted and classified correctly. There are two regular audits of the data with respect to compliance with the SCRS: the first is a systematic and routine check by the Crime Registrars; the second, an independent regulatory audit by HMICS (as detailed below). Police Scotland audits are carried out in accordance with an established methodology developed by Police Scotland.

The ultimate responsibility for ensuring compliance with the SCRS (and associated counting rules) lies with the Chief Constable of Police Scotland, discharged on a daily basis to the appointed Crime Registrars. All crime reports are subject to the rules set out in the SCRS which are under constant review. Updates will normally be effective from 1 April each year.

The Command Area Crime Registrars sit within Governance, Audit and Assurance and operate independently of Divisions. In addition to their function of ensuring compliance with the SCRS, they provide guidance and training. Further information is available in the Crime Registrars chapter.

Each of the thirteen Police Scotland divisions has a Crime Manager who is responsible for local crime recording practices in their respective divisions.

The Crime Registrars are responsible for conducting audit of recording practices relating to:

Crime Related Incidents and Associated Recorded Crime

Crime Records Reclassified to ‘No Crime’ - these are reversals of an original assumption that an incident was a crime, based on additional information – for example a broken window may at first sight be considered an act of vandalism but further investigation confirms it was an accident.

The audit methodology is presently under review following HMICS Audit 2020 Inspection report.

The audits do not presently cover the ‘clear-up’ (detections) of crimes and offences.

This ongoing scrutiny provides an additional checking function highlighting consistency or differing practices between divisions, identifying common occurring errors where more explicit guidance might be necessary and differing performance between individuals’ judgements. Each audit is followed up with targeted training and support, whereby a report on each division’s results is prepared – to facilitate discussions with the local crime management team around any problems identified. The Registrars also retain the ability to conduct short-term follow up audits if significant room for improvement is flagged up for a particular division or crime type.

The results of these audits are provided to the Audit Risk and Assurance Committee of the Scottish Police Authority. In addition to this, the SCRB receives reports from the National Crime Registrar on the internal audit programme of police crime and offence data, and progress made in meeting recommendations from HMICS and previous internal audits. This gives everyone involved in the process of quality assuring and (or) producing the National Statistics an opportunity to discuss the results of these audits and raise any queries.

The results of these audits should also be seen in perspective with the other quality checks in place, especially the daily checking of all current records by the divisional supervisors.

Recorded crime data is collated by the Police Scotland in-house Demand & Productivity Unit which works to strict quality procedures. The Demand & Productivity Unit provides data for Police Scotland to assess its own performance and to report to the SPA.

13.3 Quality assurance safeguards

Recorded Crime in Scotland data is produced by Police Scotland about Police Scotland. Therefore, strict independence of reporting from core business is required to avoid any undue influence on the data.

Crime managers, who carry out the first level of checking of crime reports, have no responsibility to achieve targets and should therefore be free of influence, however their division does aim to achieve increased performance.

Crime registrars have no responsibility to achieve targets and are conscious of their role in maintaining an ethical approach to reporting and HMICS provides independent scrutiny of both Police Scotland and the Scottish Police Authority to deliver services that are high quality, continually improving, effective and responsive to local needs.

The SPA exists to hold Police Scotland to account. They have a set of Performance Standards, derived from the Policing Principles, to hold the Chief Constable to account. They use Recorded Crime in Scotland as part of the evidence base to inform member briefings.

The SCRS is generally updated on an annual basis and published in April each year. Changes are approved via the Scottish Crime Recording Board (see Chapter 9).

One of the advantages of Police Scotland is the opportunity to improve the consistent application of the SCRS as only through such an approach can the public have full confidence and understanding as to the true extent of crime taking place.

13.4 Quality reports for the data

Crime registrars carry out an annual audit. These audits are fed back to divisional commanders, assistant chief constables in each command area and Police Scotland audit and risk board. The reports contain detail appropriate to the audience and actions, such as training and guidance, are taken on the basis of them.

The SPA has a statutory role in scrutinising the performance of Police Scotland. Established under the Police and Fire Reform (Scotland) Act 2012, its main functions are to maintain the police service, promote the policing principles and continuous improvement in the policing of Scotland, and to hold the chief constable to account. The SPA also provides independent scrutiny of Police Scotland’s Performance by assessing delivery of both Police Scotland and SPA in delivery of the current Strategic Police Plan and associated business planning. The SPA has a role in mitigating any risks of inconsistency or reporting conflicts through the Policing Committee and subsequently to the SPA Board.

It is evident from the work undertaken by crime registrars that high levels of service and investigation are being delivered across Police Scotland, despite some recording process errors being identified. Crime registrars are focussed on ensuring that such errors are overcome and that improvement activities will improve SCRS compliance rates moving forward^.

13.5 Her Majesty's Inspectorate of Constabulary in Scotland (HMICS) audits

Reviews of incident and crime recording have formed a regular part of the HMICS scrutiny programme in recent years. Crime Audit 2020 was laid before parliament in March 2021, which follows similar audits of crime recording carried out by HMICS in 2016, 2014 and 2013 (conducted shortly after the creation of Police Scotland). These previous assessments were broadly satisfied with the arrangements at the time, while making specific recommendations for improvements, which were followed up in subsequent reviews. Following the most recent audit (Crime Audit 2020) an Action Plan to address the recommendations is being progressed by Police Scotland.

These audits provide an independent assessment of the state, efficiency and effectiveness of crime recording by Police Scotland and the extent to which recording practice complies with the SCRS and Counting Rules. They test the accuracy of both incident and crime recording in Scotland, and provide an independent layer of assurance to the data.

13.5.1 HMICS Crime Audit 2020

The most recent audit of incident and crime recording in 2020 found Police Scotland’s compliance with the Scottish Crime Recording Standard and Counting Rules is generally good at over 90%, although below the service’s own target of 95% compliance. 91.4% of incidents had been closed correctly (Test 1) and 90.8% of crime was counted and classified correctly (Test 2). These compliance rates were lower than those found in the previous audit in 2016 when the results were 92.7% and 95.1% respectively.

Key Findings:

There remains scope for improvement in the recording of sexual crime. 90.3% of sexual incidents were closed correctly (90.0% in 2016). However, only 86.1% were counted and classified correctly, lower than 2016 (91.4%). Only 80.1% were recorded within 72 hours of being reported to the police (83.9% in 2016).

Of the 478 sexual crime records audited 58 (12.1%) were cyber-enabled crimes and 39 (67.2%) correctly had a cyber marker applied.

There remains scope for improvement in the recording of violent crime. 89.6% of violent incidents were closed correctly and 91.1% of violent crimes were counted and classified correctly, both lower than the audit results of 2016 (93% and 96% respectively). 92.2% of violent crimes were recorded within 72 hours of the incident being reported to the police (94.6% in 2016)

In the 2020 Crime Audit HMICS looked at domestic abuse offences, a category not examined in previous audits. Compliance for domestic abuse offences is good. 94.6% of domestic abuse incidents were closed correctly, a higher compliance rate than the other categories examined. 92.6% of domestic abuse crime records were counted and classified correctly. 95.2% of domestic abuse crimes were recorded within 72 hours of the incident being reported to the police, with ten of the thirteen divisions achieving over 95% compliance.

No-crimes relate to incidents where a crime report was created, but following investigation found not to be a crime. The audit found 94.4% of no crime decisions were correct, which is good. In 2016, the result was 96%. Three divisions achieved 100%: Lothians and the Scottish Borders, Greater Glasgow and Lanarkshire.

Divisional results varied, and compliance rates were lower than those found in 2016 in many areas. One division, Lanarkshire, performed better than Scotland as a whole in all categories of audit . The performance of one division, Dumfries and Galloway, had deteriorated since 2016 when it was found to perform consistently well. HMICS was told this was due to a period of temporary staff shortages.

Having identified that many issues in earlier audits were yet to be resolved, HMICS closed all previous recommendations and incorporated all outstanding issues within a set of six new recommendations for PS around their operational oversight of crime recording. HMICS urged Police Scotland to use the recommendations as a basis for an improvement plan which could be monitored and reported on regularly. This resulted in the Police Scotland Action plan. HMICS continues to monitor the progress of the Action plan and the six recommendations.

The full audit report can be accessed online. You can also access information on previous audits on the HMICS website.

13.5.2 Future HMICS Audits

No date has yet been identified for a future Crime Audit, however HMICS has undertaken in its Scrutiny plan 2022-2025 to undertake a further Crime Audit in 2024 which should provide sufficient time to allow the roll out of Police Scotland’s new crime management system to be fully implemented.

13.6 Conclusion (from the audits)

Evidence from the various audits conducted suggests that the accuracy of recorded crime data remains good at the national level (although some variation persists at the divisional level).

The value of both Police Scotland and HMICS audits is of most benefit when used constructively to improve officers’ understanding of the crime classification system and how to get the classifications correct, first time. Internal circulations within Police Scotland suggest ongoing improvement, and as part of its standard functions the SCRB will continue to consider how all the audit activity undertaken can best inform and provide assurance to users of both Police Scotland’s management information and the published National Statistics.

13.7 QA Stage Three - Updating records and revisions analysis

This stage considers how records can change over time as the information is refined. It considers how to present statistics derived from snapshot information and how such changes could be reported on.

13.7.1 Operational processes

One of the challenges of the recorded crime databases is that they are dynamic, i.e. constantly changing in real time as police investigations develop. Care is taken to analyse data from well regulated extracts so to ensure consistency of reporting.

Most aspects of recording crimes are subject to revisions from the initial recording of an incident to the final recording of a specific crime. Details of the crime report might change as investigations progress and intelligence is gleaned. Further information on this is available in the Scottish Crime Recording Standard and Counting Rules and the No Criming chapters.

Once the crime report is effectively finalised, changes only take place to the report if further information comes to light, even though other related information might provide a contextual update – i.e. if a court comes to a different decision than the police.

13.7.2 Quality Assurance procedures: Police Scotland Management Information

Police Scotland advise that when presenting analysis based on quarterly extracts of data, care needs to be taken when interpreting the results, given amendments to crime reports can always arise after the initial crime report is entered onto the crime recording system. Care needs to be taken when making statements, especially about fluctuations, as there will always remain the possibility that figures could be revised in future extracts of the data.

Further information on additional checks are available in the section/chapter on Producer’s QA investigations and documentation.

13.8 QA Stage Four - Police Scotland internal and external reports – scrutiny and checking

This stage provides details on the Management Information (MI) reports Police Scotland produce for both internal purposes and public use. It also provides details of the data scrutiny and QA arrangements in place to mitigate the risk of errors occurring in the data extraction and report preparation processes.

13.8.1 Publications and Management Information Reports

Police Scotland publishes statistical information on a quarterly basis in the form of MI releases, typically around two months after the period to which they relate. These include:

• A quarterly MI Report for Scotland as a whole.
• A quarterly MI Report for each Council Area.
• A quarterly MI Report for each Police Division.

The figures in these reports provide provisional data for the current year and revised data for the previous year. These MI releases are not classified as Official Statistics and Police Scotland is not a registered producer of Official Statistics. They are produced to demonstrate Police Scotland’s commitment to transparency (alongside other regular reporting activity to the Scottish Police Authority).

The data for these releases is also the main source of information for the Scottish Government’s Recorded Crime in Scotland publication (an annual National Statistics product). The figures in the Q4 MI release will in theory be consistent with the annual National Statistics publication (although they may end up differing as a consequence of amendments following QA carried out by Scottish Government statisticians – see Stage 5).

Police Scotland manages performance internally through the use of dynamic dashboards and oversight from various boards such as Strategic Operational Performance Board, Operational Delivery Board and Regional Delivery Board. Performance reporting consists of:

• Annual Police Plan Bi-Annual Report (produced at the end of quarter 2 and combined with Chief Constable’s Year End report at the end of quarter 4). This fully supports the legislative requirement to report on progress of the activities contained within the Annual Police Plan.
• Quarterly Performance Report (produced at the end of each quarter). This supports legislative reporting requirements but also provides a wider and more detailed picture of traditional policing performance measures, supplemented with additional insight and narrative to the many different aspects of policing and corporate support functions.
• Chief Constable’s Year End Report (produced at the end of quarter 4). Section 39 of the Police and Fire Reform (Scotland) Act, 2012, SPA must prepare the Annual Review of Policing as soon as practicable after the end of each reporting year. In service of this report, the Chief Constable is required to provide, within three months of the end of the reporting year 2021/22, an assessment of the performance of the Police Service.

13.8.2 Scrutiny and Quality Assurance processes

This section provides details about the scrutiny and QA Police Scotland carry out in the process of preparing reports for public and internal consumption. This is set out in three sections: (a) routine data scrutiny and checking (b) diagnostic and analytical scrutiny; and (c) research and cross referencing of data.

13.8.2.1 (a) Routine data scrutiny and checking

The SEBP Data Repository is used by Police Scotland to pull together crime data from legacy forces and aggregate the information needed for national reporting purposes.

The SEBP Data Repository is used by the Statistical team within the DPU for many of their routine report outputs and adhoc analysis. Both SQL and SPSS are used to extract and process the data. Results are often exported to Excel as an intermediary step before preparing the final published product.

The SPSS code is often complex and well documented, with considerable attention to detail, e.g. re-coding records to correct recording differences between divisions.

As part of the processing of data, the SPSS code often includes routines to produce frequencies and cross-tabulations of existing and derived variables. This acts as a double-check for potential errors occurring as a result of recoding and aggregation, mitigating the risk of errors being introduced in the processing stage.

Processed data is scrutinised carefully when translated into final reports. This involves consistency checking between result sets, ensuring consistency between products, and checking totals, percentages etc. within result sets are calculated correctly. Reports are peer reviewed and signed off at a senior level prior to publication.

It is important that the data underpinning publications is held in a snapshot form (so that the content of publications can be replicated) and that the option remains for additional historical scrutiny and analysis if required.

Reports are shared widely across the police service and with relevant partner bodies (such as local authorities and the SPA). Feedback received may include queries on the data itself, but more often will ask for reasons behind levels or changes in the data, in order to understand trends or unusual figures.

It is rare for errors to be made in publications. When made, these are corrected either immediately or in the next release (depending on severity and frequency).

13.8.2.2 (b) Diagnostic and analytical scrutiny

A considerable amount of scrutiny over the data set is conducted within Police Scotland, mainly looking at emerging trends, geospatial analysis and outlier/exception spotting. Much of this analysis feeds into internal strategic reports that provide intelligence complimenting the data. Police Scotland use diagnostic products to interrogate its data and analyse trends. Exception reporting is undertaken whereby regular business intelligence analysis is carried out to understand trends for a range of crime types and performance indicators. This analysis is used to highlight both high and low levels of data in real time.

13.8.2.3 (c) Research and Cross-referencing of data

The incident recording system (STORM Unity) gathers information about the incident leading to the decision of whether a crime has been committed, before opening a crime report on the crime recording system. The crime report holds the incident reference number (where applicable), so that crimes may be related to incidents. Many incidents recorded do not result in a crime report and a unique incident may generate a number of different crimes.

At present the quarterly MI reports provide information on the number of incident types by force, division and local authority for the year to date compared with the previous year’s figures.

13.9 Future developments

A new national IT crime system will be rolled out across Divisions by the end of 2023/24. With any developments in this area, the Scottish Government and Police Scotland will consider what impact this may have to the provision of data and the subsequent publication of the bulletin. Such developments could provide the opportunity to revisit the data collection method, as well as the availability of greater granularity of data.