Scottish Crime and Justice Survey: user workshops - summary

11. Annex C: Paper for Workshop 3 on Questionnaire Development

1. Introduction

The Scottish Crime and Justice Survey team are in the process of a re-procurement exercise. The user engagement portion of this exercise began with a Consultation (hosted on Citizen Space) and a set of associated discussion events. Both the consultation and discussion events gave users and stakeholders the opportunity to provide comment on the strengths and limitations of the survey, in addition to reflections on how the survey might be adapted through re-procurement. Our series of User Workshops build on three key themes arising from the consultation and discussion events. This workshop, Workshop 3 on 'Questionnaire Development', builds on the extensive feedback and input we have received on this topic thus far.

This paper will: i) outline the key considerations and limitations we must acknowledge when undertaking questionnaire development work; ii) summarise the 'questionnaire development' feedback we have received so far; iii) sketch out our plan for our upcoming questionnaire development phase and, iv) outline some possible topics for the open discussion.

During the workshop itself, the SCJS team will provide an overview of these areas before opening up to suggestions and questions from attendees.

2. The Scottish Crime and Justice Survey (SCJS)

The SCJS is a large-scale social survey which asks people about their experiences and perceptions of crime in Scotland. Over the years, the SCJS has asked over ninety thousand people about their views and experiences of crime and justice in Scotland. It has found a 46% fall in crime between 2008/09 and 2019/20 and improvements in people's feelings of safety. Importantly, it has also identified varying levels of both crime victimisation and feelings of safety amongst different groups in society.

The SCJS is a key source of evidence on crime victimisation in Scotland, as well as issues relating to policing and the criminal justice system. It is used extensively by the Scottish Government to inform the design and effective delivery of policy as well as by academic stakeholders and justice organisations. The SCJS and Police Recorded Crime are essential complementary sources of data that, together, present a fuller picture of crime in Scotland. The SCJS captures crime that does not come to the attention of the police, and helps validate the police recorded crime data and ensure crime and justice policy is based upon a comprehensive understanding of the victimisation rate in Scotland.

3. Covid-19 Impact on SCJS

In 2020, all Scottish Government face-to-face interviewing, including the SCJS, was suspended and the Scottish Victimisation Telephone Survey (SVTS) was developed. The Scottish Government introduced the SVTS as a discrete collection to the SCJS and the results are based on a sample of around 2,700 telephone interviews conducted in September and October 2020.

The current survey year (2021/22) has also been affected by research restrictions as a result of COVID-19. This year, the SCJS is being carried out using a mixed-mode approach. 'Knock to Nudge'^[7] is being used to offer respondents an interview by telephone or via video call, until such time it is deemed appropriate to include an in-home face-to-face option as well. The self-completion section of the survey is currently completed by the respondent online or on paper.

4. Re-Procurement

The Scottish Crime and Justice Survey (SCJS) in its current form was established in 2008/09, although a crime survey has run in Scotland since 1982. From October 2015, the SCJS has been delivered by Ipsos & ScotCen but this contract is coming to an end. Therefore, to ensure the continuation of the survey and the continued provision of evidence on crime victimisation in Scotland, a re-procurement process is needed. Our proposed procurement timeline would ensure that a Supplier would be in place in October 2022 and able to begin fieldwork in Spring of 2023.

5. Re-Procurement Work Streams

• Mapping Users of the SCJS Data: Identifying users of the SCJS data and reviewing how, and to what extent, they engage with the survey in order to become better informed about user requirements.
• Literature Review: Reviewing the criminology literature to locate the SCJS, its key concepts and ideas within the wider discourse on crime and victimisation.
• Review of International Crime Surveys: Reviewing existing, international crime surveys to identify areas of best practice and lessons to be learned.
• Formal Consultation & Discussion Events: A formal, written consultation hosted on Citizen Space, accompanied by live discussion events, designed to enable users to provide feedback on the SCJS and how it might be adapted. This consultation closed on the 9^th December.
• Options Appraisal: Different survey design options for the next iteration of the SCJS appraised by a wide range of internal and external users and stakeholders.
• User Workshops: Three user workshops to collect further user and stakeholder feedback on i) Options for survey design, ii) User engagement strategies and iii) Questionnaire Development.

6. Workshop Aims

In this workshop we aim to :

• Communicate the feedback we have received so far on the topic of questionnaire development;
• Outline upcoming plans for questionnaire development work;
• Invite input on both of these topics.

7. Considerations and Limitations

When discussing the topic of questionnaire development, it is important to highlight certain constraining factors:

Survey Length: The current survey length is approximately 40 minutes (although this can vary). It is not recommended to increase the length of the survey as it is important to avoid over-burdening respondents. This means that we are limited in how many questions we can add, and that we must also consider removing questions when adding new ones.

Adequate Sample Size: It is not consistent with UK GDPR requirements to collect personal data that cannot be used to produce robust analysis. Therefore, if a question or topic relates to something that is likely to be a particularly rare experience, or for some other reason will not result in an adequate level of data for analysis, then it is unlikely to be adopted into the SCJS.

Time Series Comparability: In order to identify trends and make meaningful comparisons over time, questions asked within the survey need to be comparable year on year. This means that any desire to adapt questions must be weighed against a desire for comparability.

Competing Demands: Demand for questions to be added to the SCJS is always higher than survey capacity. Therefore, the SCJS team – in conjunction with stakeholders – must make a decision about which questions to prioritise. Preference will be given to questions that will produce data for which there is a clear use and user demand.

Appropriateness of the SCJS : When considering the addition of new question topics, it is important to determine whether the SCJS is the appropriate place for these questions, and whether these questions align with its function as a victimisation survey. There may be cases when a question would be better suited for inclusion in a different survey or, perhaps, whether data should be collected using a different method.

Lead Time: When considering the addition of new questions, it is important to acknowledge the significant 'lead time' of the SCJS. For example, if a new question were to be added to the first survey under the new contract, the question would be asked during the 12 month fieldwork period, Spring 2023 to Spring 2024. Data arising from this question would be included in the main findings report, published in early 2025. This significant lead time means the SCJS is not an appropriate place for urgent data collection.

8. Feedback Received So Far

Feedback received in consultation submissions, during discussion events and during informal meetings with stakeholders has been wide and varied. Questionnaire development, both the addition of new question modules and the amendment of existing questions, has been a very popular topic. Thus, the following bullet points are a summary of pieces of feedback received most commonly, as opposed to a comprehensive overview of all feedback received. To note, the wording below is reflective of the feedback itself and is not necessarily reflective of the views held by the SCJS team:

Cyber Crime:

• It is important to review this questionnaire module if the SCJS is to keep pace with changing trends, and understandings, of crime.
• Need to define what cyber-crime is and how it relates to other types of crimes, e.g. stalking, harassment, sexual offences.
• Perhaps the 'SCJS Crime' total should include cyber crime

Partner Abuse:

• Questions should be aligned with the most recent legislation and should cover aspects of abuse like coercive control and the impact on victims.
• Need to adapt existing questions to enable respondents to indicate that: their abuser was the person who contacted the police and that a civil case (and not just a criminal case) arose from their experience.

Violence Against Women and Girls (VAWG):

• Enthusiasm to increase number of questions that focus specifically on VAWG.
• Information should be gathered on respondents' views on conviction rates for sexual crimes
• Experiences of justice process, specifically on if such experiences are re-traumatising.

Hate Crime:

• Enthusiasm for the SCJS to collect information specifically on hate crime and to align with recent Hate Crime legislation
• Hate crimes often unreported to police so survey a good instrument to collect information.

Restorative Justice:

• To align with Scottish Government's commitment to the availability of restorative justice by 2023, could we ask if people have been offered Restorative Justice?
• Could also ask if people would be willing to use Restorative Justice.

Feelings of Safety:

• Current feelings of safety questions could be extended to collect more information.
• For example, in addition to the existing questions, we could ask respondents about feelings of safety when travelling to and from work, or when visiting certain locations like local parks.

Stalking & Harassment:

• To acknowledge the way in which cyber-crime can overlap with both stalking and harassment.

• To adapt question wording to include the sending of gifts, letters and cards as an example of behaviour that could be considered harassment

• For the question that asks whether an individual has shared pictures of you to be amended to included 'shared or threatened to share'

Sentencing & Wider Criminal Justice System:

• To ask respondents whether the criminal justice system 'gives sentences which reflect the seriousness of the crime. Then asking them this for a number of different offence groups including sexual offences, non-sexual offences, minor offences etc.
• To ask about people's experiences of the criminal justice system, including questions on whether the experience has been re-traumatising.
• To ask about confidence levels in COPFS.

Identifier/ Demographic Questions:

• Collecting information on whether respondents are care experienced
• Collecting information on whether respondents are BSL signers.

9. Plans for Questionnaire Development Work

Partner Abuse

Work has already begun on the development of the Partner Abuse module of the SCJS and the Questionnaire Development Update, published in November 2019,

outlines the projects undertaken so far, including a review of the existing questions and stakeholder engagement on the topic. This work was paused as a result of changing priorities at the onset of the pandemic, however, the SCJS team is keen to return to the development of this module in the spring of 2022 in the hope that any changes made will be included in the first year of the new SCJS contract.

The work is likely to take learning from the various projects undertaken in 2019, however, we will also re-engage stakeholders in the process. When developing the module our goals will be to assess and then respond to user need; align with wider SG strategies as well as the most recent legislation on the topic.

Cyber Crime

As stated in the Questionnaire Development Update, published in November 2019, new questions on cyber-crime/online behaviours were introduced to the questionnaire with effect from the 2018/19 SCJS. The team made a commitment to continue reviewing this module and the decision has been made to undertake further development work from the spring of next year. As above, the hope is that any changes made will be included in the first year of the new SCJS contract.

10. Suggested Areas for Discussion

The following are some suggested areas for discussion, however, any and all comments are welcome:

• Further views on the 'Feedback received so far'.
  • Do you agree that these areas should be considered for development?
  • Do you have specific comments on what changes should be made, or what questions should be added, in each of these areas? How will this data be used?
• Plans for Questionnaire Development Work
  • Do you have any comments on the outlined plans?
  • Do you have any suggestions for how this development work should be undertaken?
• Additional Comments
  • Are there any additional areas that you believe would benefit from questionnaire development work?
  • Are there any comments on the 'Considerations and Limitations' within which the SCJS team have to work.
  • Are there any questionnaire modules that you think could be asked less frequently (perhaps every 2 years)? This would free up space to introduce new questions.