User Guide to Recorded Crime Statistics in Scotland

10. Quality Assurance

A wide range of mechanisms are employed to ensure that police recorded crime data is robust. This section (which comprises Chapters 10-15) documents the steps undertaken to quality assure data that is captured and published as management information by Police Scotland, and analysed and turned into National Statistics by the Scottish Government. This includes a summary of the quality checks made at each stage of the data journey, from capture to publication.

The process of producing National Statistics on police recorded crime is complex. Data is collected in real time, as Police Officers respond to incidents and enter information into their Crime Management Systems. This is then updated on a continuous basis as investigations proceed and new information or evidence is obtained. Key decisions are made early in the process, including whether or not an incident constitutes a crime and if so what the classification of that crime should be, ultimately for statistical purposes.

National Statistics and management information are derived from snapshots of crime recording databases. Due to the evolving nature of crime investigations, the timing of data extraction from Police Scotland databases will always have some impact on the resulting figures. A summary of how this affects the production of analysis on recorded crime is included in the discussion of data revisions within the Data suppliers' Quality Assurance principles, standards and quality checks and Producer's Quality Assurance investigations and documentation chapters.

The current database systems are complex, with data being collected at various sites on different data systems. The quality assurance (QA) processes in place are focussed on the accurate capture of data, consistency of recording, and the accurate transfer of processed data into a range of publications.

Not all crimes are reported to the police, and hence recorded by them. If a crime is left unreported or unrecorded, it will not be contained in the recorded crime statistics^[4]. It is also worth noting that further decisions relating to the crime after Police Scotland’s involvement e.g. following a court decision, will not be reflected in these data. Further information is available in the Risks, limitations and challenges of the recorded crime data chapter.

The UK Statistics Authority (UKSA) Administrative Data Quality Assurance Toolkit provides guidance about acceptable levels of QA in respect of administrative data. This user guide draws on that guidance, recognising that QA of administrative data is more than just checking that the figures add up. It is an ongoing iterative process to assess the data’s fitness to serve their purpose. It covers the entire statistical production process and involves monitoring data quality over time, and reporting on variations in that quality.

10.1 Summary of safeguards

Safeguards are in place throughout the collection and production lifetime of Recorded Crime statistics. These are shown schematically in Figure 10.1, along with who carries them out in Table 10.1. The safeguards exist to ensure that the data collection and analysis produce high-quality, complete statistics.

Table 10.1: Validation checks carried out during collection and publication of police recorded crime in Scotland

1. Action

Crime recording should be carried out in accordance with the Police Scotland Code of Ethics.

Who does it?

Police Scotland – reporting officer

2. Action

Police officers/staff use legacy systems to record a crime.

Who does it?

Police Scotland – reporting officer/ police staff

3. Action

Incident recorded with an appropriate crime type according to the SCRS and Crime Counting Rules.

Who does it?

Police Scotland – reporting officer/police staff

4. Action

Crime Management staff in each police division oversee the crime records.

Who does it?

Police Scotland – Crime Management Staff

5. Action

Check downloads – daily check to ensure data has been provided from each crime recording system.

Who does it?

Police Scotland – ICT and Demand Productivity Unit

6. Action

Check compliance with the SCRS.

Crime Registrars provide independent scrutiny to assure the quality of crime records by undertaking audits. The national structure and links between crime registrars help to ensure data is consistent.

Who does it?

Crime Registrars

7. Action

Check extracts are complete and correct – sense check and comparisons on legacy system data.

Who does it?

Police Scotland – Demand Productivity Unit

8. Action

Crime reviews (from point 1 to point 8). HMICS assess the state, efficiency and effectiveness of crime recording by Police Scotland and the extent to which recording practice complies with the Scottish Crime Recording Standard and Counting Rules.

Who does it?

Her Majesty’s Inspectorate of Constabulary in Scotland (HMICS)

9. Action

Carry out checks according to the Government Statistical Service guidance to verify the validity of Police Scotland data. These consider whether the derived aggregated statistics are meaningful, and whether changes in time series and discontinuities can be explained.

Who does it?

Scottish Government, Justice Analytical Services

10. Action

Check for areas of sensitivity and potential data confidentiality issues.

Who does it?

Scottish Government, Justice Analytical Services

11. Action

Peer reviews within Justice Analytical Services. Publication according to the Code of Practice for Statistics

Who does it?

Scottish Government, Justice Analytical Services

Recorded Crime in Scotland published

10.2 Structure of the Quality Assurance section

This QA section is structured in accordance with the UKSA’s aforementioned guidance on QA of administrative data, and the evidence is arranged around the key QA stages in the ‘data journey’:

Chapter 11 - Operational context and administrative data collection

In this chapter, we demonstrate and describe for users how we understand the environment and processes in which Recorded Crime in Scotland data are compiled and the factors which might affect the quality of the data. Includes QA Stage One - Dynamic capture of incident and crime data at source and QA Stage Two - Classifications of Crime and Incidents.

Chapter 12 - Communication with data supply partners

This chapter provides evidence of how the Scottish Government maintains and develops strong and effective relationships with Police Scotland and the Scottish Police Authority (SPA) as data suppliers. It provides the clarity and structure to enable us to maintain a good understanding of the approaches adopted by Police Scotland and the SPA to ensure consistency in recording and quality levels.

Chapter 13 - QA principles, standards and quality checks by suppliers

This chapter describes the principles, processes and checks that are conducted by Police Scotland, and how these results are used to ensure continuous improvement of the statistics. The work of other independent auditors, including HMICS and the SPA is also included here. Includes QA Stage Three - Updating records and revisions analysis and QA Stage Four - Police Scotland internal and external reports – scrutiny and checking.

Chapter 14 - Producer’s QA investigations and documentation

This chapter includes the QA conducted by the Scottish Government. We set out how the findings from our own QA checks are supplemented by the knowledge gained through reviewing the other practice areas outlined above, and used to inform a published statement that sets out the basis of the producer’s judgment about the quality of the administrative data. Includes QA Stage Five - Scottish Government Quality Assurance and publication of National Statistics on Recorded Crime in Scotland.

Chapter 15 – Risks, limitations and challenges of the recorded crime data

This chapter includes an explanation of the limitations and risks of the underlying data, so that users can appreciate how these will affect their use of the statistics.

When considering the QA steps undertaken at each of these stages, relevant aspects of the UK Statistics Authority’s Quality Management Actions (from the Quality Assurance of Administrative Data tool kit) have been considered. The key QA components that are currently in place or are being developed are listed below.

Quality Management Actions currently in place or under development

Investigate

• Police Scotland scrutiny of its data
• Scottish Government scrutiny of the data provided
• Crime Registrar’s audit
• HMICS audit
• Bias, error and revisions analysis

Manage

• Scottish Crime Recording Board
• Strengthening relationships
• Feedback loops & learning processes
• Publishing audit reports
• Corroborating with other sources

Communicate

• Documentation of the statistical processes
• Regular dialogue between suppliers and providers
• Describe bias and errors to users
• User friendly website
• Statistics related updates for users (via ScotStat)

This QA section provides evidence on:

the suitability of the administrative data for use in producing National Statistics;

factors that the statistical producer needs to take into account in producing the National Statistics;

the information that users need to know to make informed use of the statistics.

This section should satisfy users that safeguards are in place to measure and report crime statistics, and that steps are in (or continue to be put in) place to improve processes and systems. Its purpose is to build users’ confidence in the use of administrative data for statistical purposes and support their correct application in analysis. It also demonstrates what we do to deliver the highest quality data for users in the most proportionate way.

While the assurance processes and systems detailed must be taken in context with the information on limitations and risks of the data, the Scottish Government believes that there is sufficient evidence to provide users with confidence in the Police Recorded Crime Statistics.