Scottish Crime and Justice Survey 2021/22: Main Findings

2. Impact of the COVID-19 pandemic on the SCJS

Response to the COVID-19 pandemic

In order to help prevent the spread of COVID-19, the Scottish Government suspended face-to-face interviewing for the SCJS and the other household surveys on 17^th March 2020.

The last findings for the SCJS, produced from interviews which took place before the suspension came into force, were published in March 2021 and covered the 2019/20 survey year (as well as biennial self-completion results for 2018/19 and 2019/20 combined). These results were not significantly impacted by the suspension of face-to-face interviewing due to COVID-19 as the fieldwork suspension was at the very end of the fieldwork period (note the SCJS fieldwork typically begins in April and runs for 12 months).

To fill the evidence gap created by the suspension of the SCJS, the Scottish Victimisation Telephone Survey (SVTS) was developed. This was a standalone survey of experiences and perceptions of crime, safety and policing in Scotland during the COVID-19 pandemic. The results were based on around 2,700 telephone interviews conducted in September and October 2020. This survey was a discrete and additional collection to the SCJS and not a replacement. More information on the background and findings from this survey can be found on the SVTS website.

No Scottish Crime and Justice Survey interviews took place in the 2020/21 survey year and as a result no data were published for that year.

Resumption of interviewing and changes to the survey design

Due to the relaxation of COVID-19 restrictions, it was possible for SCJS fieldwork to resume in late 2021 (with interviews conducted between November 2021 and December 2022). The results of this fieldwork sweep are presented in this report.

While restrictions had been relaxed, at the start of the fieldwork period, it was still not possible to conduct interviews face-to-face in respondents’ home, the approach used pre-pandemic. Instead, interviews were carried out remotely, either by telephone or video. This approach was in line with public health measures in place at the time. Limited adaptations were made to the questionnaire to accommodate telephone interviewing.

Between November 2021 and April 2022, interviews were conducted entirely using a ‘knock-to-nudge’ approach. This involved interviewers initially calling at sampled addresses to introduce the survey on the doorstep, randomly select and adult to take part, encourage them to do so and arrange a time for the interview to be conducted remotely (by phone or video chat). In-home, face-to-face interviewing resumed in April 2022. Despite this, respondents were still offered the option of a remote interview, if that was preferred given any ongoing concerns about risks of infection.

The 2021/22 questionnaire was largely left unchanged from the 2019/20 questionnaire with the exception of a few amendments:

• a reduced self-completion section
• the amendment of the format of some questions for telephone interviewing
• the introduction and removal of a small number of questions

The sampling approach for the SCJS remained broadly the same as previous years, with one member of the household randomly selected by the interviewer to take part in the survey. Assumptions around response rate were revised.

Summary of comparability analysis

It is important to consider whether the change in approach to interviewing had an impact on the comparability of the findings presented here with earlier years. For example, was there a significant shift in the profile of individuals responding to the survey? To examine this, analysis was conducted by Ipsos on behalf of the Scottish Government to examine:

• variation in response rates across key geographic variables (i.e. deprivation, urban/rural and police division)
• change in the profile of respondents pre- and post-pandemic
• differences in the respondent profile and victimisation rates between the two fieldwork stages and the different modes of interview

A summary of their findings are provided below and the full report, entitled ‘Scottish Crime and Justice Survey: Analysing the effects of using a mixed-mode approach to adapt to COVID-19 challenges’, can be read on the supplementary documents page.

Between 2019/20 and 2021/22, response rates fell from 64% to 47%. While this represents a large fall, the pattern of the reduction was consistent across different areas of Scotland. For example, the most and least deprived areas fell from 57% and 66% to 43% and 49% respectively, with a similar trend seen between urban and rural areas.

Secondly, the composition of the achieved sample in the post-pandemic wave was compared against that of the pre-pandemic wave across a range of estimates. These included 20 geographic measures, household level characteristics, individual level characteristics and various substantive measures. Overall, for most variables that we would expect to be relatively stable, the differences between the pre- and post-pandemic waves (after weighting) were relatively small. However, for a limited number of key variables, the changes in estimates may be more than expected. These included tenure and educational attainment:

• owner-occupation increased by 2 percentage points
• respondents with no educational qualifications decreased by 5.5 percentage points

This means that non-response bias might have impacted the post-pandemic slightly more than the pre-pandemic wave. However, the scale of this is likely to be small, especially on key substantive measures such as victimisation.

Finally, across a wide range of estimates the differences across modes and between the different stages were small. There were a small number of estimates where there were differences by mode and stage, the analysis did not find any evidence of change to how people answer questions. This suggests that the move from interviewing face-to-face in-home to remote interviewing did not have a major impact on the results in relation to measurement error and are unlikely to have introduced discontinuity into the data series for the SCJS.

Implications for quality and designation of latest SCJS findings

In July 2022, the Office for Statistics Regulation supported a proposal by the Office for National Statistics to temporarily suspend the National Statistics status of the Crime Survey for England and Wales (CSEW)^[7]. This precautionary action was primarily taken due to the shorter fieldwork period (taking place over 6 rather than 12 months). ONS also note that while they saw improvements in the response rate, they were still to return to pre-pandemic levels and the spread of interviews across the year was uneven^[8].

It is important to acknowledge the impact that the COVID-19 pandemic had on the SCJS, both in terms of the need to take a mixed mode approach to interviewing and the overall reduced response rate. However, we do not expect that the small shifts described above will have had a significant impact on the comparability of these results with earlier years. As such we have taken the decision to maintain the National Statistics designation for this 2021/22 sweep of the survey. We will continue to monitor the measures described above as part of producing the 2023/24 survey. As the interviews for this sweep will be almost entirely conducted face-to-face, there should be less of an impact than what was seen in 2021/22.

Limitations on the self-completion data

The self-completion aspect of the survey asks respondents about particularly sensitive topics and therefore respondents answer this section confidentially either online or via a paper questionnaire (where a telephone or video interview was conducted) or using the interviewer’s tablet (when an in-home interview was conducted). The consequences of the COVID-19 disruption to SCJS quality and designation outlined above also apply to the survey’s self-completion section, however, there is the added challenge that fewer respondents complete the self-completion modules compared with the main survey and, there is a requirement to have two-years of self-completion data for standard results (and due to suspension of fieldwork in 2020, we only have one). Therefore, statisticians are continuing to review the collected self-completion data for 2021/22 and will inform users through SCOTSTAT what might be published from this data in 2024.