Citizens' Jury on QCovid: Report on the jury's conclusions and key findings

2. Methodology

Overview

To understand the views of the public towards QCovid® and similar risk prediction models, Ipsos designed and facilitated a Citizens' Jury. A Citizens' Jury brings together a randomly selected group of people who broadly represent the entire community. The people who attend listen to evidence from a range of experts, learn about key issues, discuss them with one another, and then deliberate and draw conclusions together.

The jury of 25 people from across Scotland met online across six three-hour workshops throughout February and March 2022. The jury was convened to answer the following key question:

"What are the risks and benefits of using public health data to predict people's risk of dying from Covid-19?"

More specifically, they explored the risks, benefits and ethical concerns related to each potential use of a model like QCovid®, and the principles that would make its use acceptable.

Ethics Panel

An independent Ethics Panel was established to advise the Scottish Government on its ethical assessment of QCovid®. It was agreed by the ethics panel that an essential component of the ethical assessment was understanding how the public viewed the use of a risk prediction model such as QCovid® in Scotland. The panel also provided a check and challenge to the process, ensuring that the methodological design was appropriate, relevant, robust and accessible.

The Ethics Panel was made up of one Scottish Government official as well as academics and specialists independent to the commissioning and convening organisations. A list of Ethics Panel members can be found in the appendix.

Sampling and recruitment

The Sortition Foundation,^[4] a recruitment organisation specialising in representative random sampling, conducted the recruitment for the Citizens' Jury by sending out 6,000 invitation letters across Scotland, using the Royal Mail Postcode Address File. Those living in more deprived areas were over-sampled to account for the lower response rates that are typically found in these areas. The full recruitment report can be found on the Sortition Foundation's website.^[5]

Based on all those who registered their interest in joining the jury, a randomised stratified selection process then took place that broadly reflected the demographics of Scotland, including age, gender, region, ethnicity, disability and deprivation (based on the Scottish Index of Multiple Deprivation, SIMD).^[6] Ethnic minority groups were over-sampled at the selection stage to ensure sufficient representation of these groups. An additional measure was also included in the selection process to ensure a range of views were represented in terms of attitudes towards personal health data use by the Scottish Government.

Overall, 30 people were selected from the members of the public who registered to participate (allowing for some dropping out before or during the process). A table summarising the demographic profile of the final selected and confirmed sample can be found in the appendix.

Once recruited, participants were onboarded by the Ipsos research team. To support and enable participation in all workshops, and in line with industry standards, participants were each paid £50 per workshop, resulting in a total of £300 for full participation. Where necessary, participants were provided with laptops and dongles to provide a connection to the internet and were supported with training on how to use the technology and access the meeting platform. This allowed us to increase the diversity of those taking part.

Jury process and structure

The QCovid® Citizens' Jury was grounded in recognised industry best practice and followed key principles of deliberative engagement. More information on these principles can be found in Ipsos UK's best practice guide, developed in partnership with experts in the field, such as Imperial College and Involve.^[7]

The jury comprised six online Zoom sessions scheduled over four weeks in February and March 2022 (summarised in Figure 2).

During the learning and deliberation phase of the jury, participants learned about each of the QCovid® tools in turn: the clinical tool, the public-facing tool, the population tool using non-anonymised data, and the population tool using anonymised data. For each of the tools, expert speakers delivered presentations to introduce the tool and outline the key considerations of each. After each presentation, members moved into small breakout groups to discuss and reflect on what they had learned. In the breakout discussions, members agreed on clarification questions which were then answered by the speakers in the main plenary, or via a Question and Answer (Q&A) document which was shared with participants on an ongoing basis with written responses provided by the speakers, the Scottish Government and Ipsos.

Sessions two to six each began with the chair reflecting on what participants had discussed in their groups at the previous workshop. This provided a space for participants to reflect on where they had got to, along with discussions they had had with friends and family. Stimuli was in the form of presentations, Q&A sessions, and case studies. The range of stimuli supported participants to reflect on their own experiences, as well as to consider relevant situations and experiences different from their own.

Based on rapid analysis of the discussions by the research team, and reviewed by facilitators of breakout room discussions, the final session provided participants with draft overarching principles on each of the QCovid® tools for review and ratification in breakout rooms. The rapid analysis has since been validated with systematic analysis, which was conducted following fieldwork to inform this report.

Overview of jury sessions

An overview of each session, including dates, times, content and presentations, is summarised in table 1 below. A list of presenters is provided in the appendix.

Materials

Materials were developed by Ipsos and approved by the Scottish Government. Presentations were developed and recorded in advance by expert speakers and then recordings were played back live during the main plenary sessions and the speakers joined the sessions to answer questions from participants. Any questions that were not answered during the session were compiled in the Q&A document. Presentation recordings were hosted on YouTube and shared via private links for members to watch again in their own time.

Interpretation of findings

The principles set out and discussed in this report are intended for consideration in the possible future use of QCovid® or similar risk models by the Scottish Government.

This exercise supported participants to express a range of views on the tools which QCovid® or similar risk models could deploy, and of their expectations and understanding of the ethical considerations in relation to using public health data in this way. This report synthesises those diverse and sometimes inconsistent expressions to draw out major themes of discussions and to draw attention to the way that participants – individually and collectively – described what mattered to them and why. On occasion, the report refers to verbatim assertions by participants and their understanding of the tools. These are not intended as authoritative statements of fact, but even when misaligned to the ways in which QCovid® has or would work in practice, they tell us something valuable about how key messages, support, or services can been perceived and understood by members of the public.

Further, it should be noted that whilst the method of qualitative analysis is systematic and rigorous and the conclusions robust (being based on groups that are reflective of the diversity of the wider public), the analysis does not seek to quantify findings nor does it indicate statistical significance from a representative sample. This report offers a valuable insight into public perspectives on the key questions posed to them after receiving and deliberating on key information relevant to the questions. As such, it opens up a deeper understanding of public perceptions of the risks and benefits of QCovid®, and the ethical considerations that resonate most.

Report structure

The following chapter summarises the overarching themes which cut across the whole of the jury's discussions on QCovid®, as well as the specific principles relating to each tool. The subsequent chapters detail the findings related to the individual tools. In each of these chapters we provide a brief overview of the content presented to the jury, followed by a summary of the ratified principles and a summary of the discussion points - risks, benefits and ethical considerations - which led to these principles. Relevant quotes and screenshots from the fieldwork are used to reflect the discussions held throughout the workshops.