Scottish jury research: findings from a mock jury study

2. Methodology

2.1 Summary of approach

In Scotland, the Contempt of Court Act 1981 prohibits questioning jurors who have participated in actual criminal trials about their discussions during deliberation. The research questions set out in Chapter 1 were therefore addressed by a large-scale mock jury experiment.

64 juries watched a video of either a mock rape or a mock assault trial, then deliberated in groups for up to 90 minutes before returning a verdict. In order to assess the effect of the Scottish jury system's unique features on jury reasoning and decision-making, the 64 mock juries varied with respect to:

• Number of verdicts - 32 juries had two verdicts (guilty and not guilty) and 32 had three verdicts (guilty, not guilty and not proven) available to them.
• Jury size - 32 juries deliberated in groups of 12, and 32 in groups of 15.
• Majority required - 32 juries were asked to reach a unanimous verdict, on which each member of the jury was agreed, and 32 were asked to return a simple majority verdict (seven out of 12 or eight out of 15 jurors).

In combination with trial type (rape or assault), this resulted in 16 possible combinations, with each combination run four times (see Table 2.1, below).

Table 2.1 Summary of conditions for mock juries

Condition number	Jury size (15 or 12)	Majority required (simple majority or unanimous)	Number of verdicts (three or two)	Trial type	Number of experiments (mock juries) per condition	Number of jurors per condition
1	15	SM	3	Rape	4	60
2	15	SM	2	Rape	4	60
3	15	U	3	Rape	4	60
4	15	U	2	Rape	4	60
5	12	SM	3	Rape	4	48
6	12	SM	2	Rape	4	48
7	12	U	3	Rape	4	48
8	12	U	2	Rape	4	48
9	15	SM	3	Assault	4	60
10	15	SM	2	Assault	4	60
11	15	U	3	Assault	4	60
12	15	U	2	Assault	4	60
13	12	SM	3	Assault	4	48
14	12	SM	2	Assault	4	48
15	12	U	3	Assault	4	48
16	12	U	2	Assault	4	48
Total					64	864

The remainder of this chapter explains each aspect of the research methods in more detail, including efforts to maximise realism.

2.2 The participants ('mock jurors')

'Mock jurors' were recruited in Edinburgh and Glasgow from members of the general public eligible for jury service by Ipsos MORI specialist recruiters, using a recruitment questionnaire developed by the research team. We recruited more jurors than required for each session, to ensure that we were able to run each jury with either 12 or 15 jurors after allowing for 'no shows' on the day. In practice, this meant that there were sometimes more jurors than required. In total, 863 jurors participated in deliberations across the 64 juries.^[30] An additional 105 'spare' jurors watched the trial videos and completed a questionnaire before being sent home (see 2.4, below, and questionnaire C in Annex E).

Recruiters obtained informed consent to participate at the time of recruitment. The researcher responsible for each jury also reminded participants at the outset that they were volunteers and were free to leave at any time. All mock jurors were given a £50 thank-you payment in recognition of the time they had given up to participate, and a helpline leaflet in case they had found aspects of the evidence or process distressing.

2.2.1 Mock juror characteristics

Quotas for recruiting jurors were based on the profile of the Scottish population aged 18-75 (75 was set as an upper limit, since those aged 71+ can be excused from jury service on grounds of age). While there is no directly relevant Scottish data, research in England and Wales found that - contrary to common myths about certain groups being under- or over-represented in jury composition - "serving jurors were remarkably representative of the local community in terms of ethnicity, gender, income, occupation and religion".^[31]

Table 2.2 shows the profile of jurors who (a) participated (n = 969) and (b) actually went on to deliberate (after 'spare' jurors were sent home, n = 863), compared with the profile of the Scottish population aged 18-75. Overall, the profile was fairly similar to that of the Scottish population in terms of gender and age group (although slightly more women participated overall, the gender balance of those who deliberated was closer to the actual population figures^[32]). In comparison with the population as a whole, slightly fewer jurors indicated that their highest qualification was at school-level or below. However, 10% of participants did not answer the survey question about their highest qualification. As lower educational attainment is often associated with non-response in questionnaires, ^[33] it is possible that this group includes more people with no or low qualifications than with higher level qualifications.

Table 2.2 - Profile of jurors who participated in mock juries

	All jurors who participated	All jurors who deliberated	Scottish population aged 18-75*
Gender
Male	44%	46%	49%
Female	55%	53%	51%
Describes self in another way / not answered	1%	2%	NA
Age
18-34	31%	32%	31%
35-54	39%	38%	37%
55+	28%	29%	32%
Prefer not to say / not answered	1%	1%	NA
Highest qualification
Standard grade or below	29%	29%	36%
Higher or equivalent	17%	17%	18%
HND or equivalent	16%	16%	12%
Degree-level or higher	28%	28%	30%
Unsure / Refused / Not answered / Other	10%	10%	4%
Sample size	969	863

* Source for population figures: Gender and Age = NRS Mid-year population estimates 2016 (the most up to date estimates available at the time quotas were set). Highest qualification: Scottish Household Survey data, restricted to 16-74 year-olds.

2.3 The trial videos

Each mock jury watched one of two filmed case simulations of around one hour in length - a rape trial (in which the accused claimed that the complainer consented to sexual intercourse), or an assault trial (in which the accused claimed he acted in self-defence). Summaries of the content of the trials are included in Annex D. Short extracts from the trial films are available to watch online:

• Assault trial: https://youtu.be/gxeU-sFzOxQ
• Rape trial: https://youtu.be/kDAGaSedje8
• Judge's opening and closing directions: https://youtu.be/ecemRns-gDk

The trials were filmed by a professional film company^[34] at the High Court in Edinburgh. All roles were played by actors, with the exception of a retired judge who performed that 'role' in the films. The trials were fully scripted by the research team, with the scripts reviewed by members of the Scottish Government's Research Advisory Group (RAG) (which included a judge, a sheriff, an advocate and a solicitor). A second advocate and retired sheriff, independent of the RAG, reviewed the scripts and observed rehearsals and filming, providing advice on realistic delivery to the actors.

The legal instructions given to the jury were based on the standard directions recommended in the Judicial Institute for Scotland's Jury Manual,^[35] supplemented by advice from the RAG, independent legal advisors, and the judge who performed that role in the films.^[36]

The purpose of this research was to test whether - whilst holding other aspects of the trial constant - the different features of the Scottish jury system appear to incline juries in one direction or another in terms of both verdict choice, and deliberation process. It also aimed to explore understandings of the not proven verdict in particular. The scripts for both trials were therefore deliberately drafted to generate a degree of ambiguity, in order to encourage debate within the jury room about guilt and acquittal, and to maximise the likelihood that jurors would consider the difference between the not guilty and not proven verdicts.

In line with Table 2.1 above, eight versions of each trial film were produced. These were entirely identical (within trial type), with the exception of the very final section, in which the judge tells the jury about the verdicts available and the majority required for a verdict to be returned. When the jury was required to try and return a unanimous verdict, a supplementary direction from the judge was recorded (to be played if the jury failed to reach a verdict after 70 minutes). This indicated that a verdict could now be accepted provided that no more than two members of the jury disagreed with it (i.e. a near unanimous verdict).

A small focus group of 'mock jurors' was present for the live rehearsals of each trial, which preceded filming. The research team also piloted the films with eight mock juries prior to the main fieldwork. Both exercises were used to check that participants found the simulations sufficiently realistic, and that they provoked an appropriate level of ambiguity and discussion. Minor changes were made to the scripts following the live rehearsals, and minor editing changes were made to one of the filmed case simulations following the pilot.

2.4 The mock jury process

The 64 mock juries took place over eight Saturdays. Four juries took place simultaneously (in separate rooms) in the morning, and another four in the afternoon. The process was as follows:

• Jurors received a short briefing from the researcher responsible for their jury, took the juror affirmation, and then watched the film allocated to their jury.
• Jurors took the standard juror affirmation.^[37]
• Jurors watched the trial video.
• The researcher distributed a short self-completion questionnaire (referred to in this report as the 'pre-deliberation' questionnaire), recording jurors' initial views on the appropriate verdict and their level of confidence in those views.^[38] The researcher gave any 'spare' jurors (in excess of the 12 or 15 required for deliberations) a slightly different questionnaire, which also included questions about the not proven verdict (asked of other jurors after deliberation).^[39]
• 'Spare jurors' were then sent home and the remaining 12 or 15 jurors were instructed to begin their deliberations. The researcher left the room after starting the audio and video-recording equipment and jurors deliberated for up to 90 minutes.
• Once juries indicated that they were ready to return a verdict, the researcher asked them who spoke for the jury, and whether they had been able to reach a verdict (and, for unanimity conditions, whether this was a verdict on which they were all agreed).
  • If a jury asked to reach a unanimous verdict was unable to do so after 70 minutes of deliberation, they were played a supplementary video direction from the judge. This informed the jury that they could now return a verdict if ten out of 12 or 13 out of 15 jurors agreed on it. If, after a further 20 minutes (that is, a total deliberation time of 90 minutes), the jury was still unable to reach a verdict, the outcome was recorded as 'hung'.
• Once the verdict had been recorded, the researcher administered another self-completion questionnaire (the 'post-deliberation' questionnaire^[40]), which covered:
  • Jurors' individual views on the appropriate verdict, and their reasons for favouring that verdict.
  • Whether they had changed their mind during the course of deliberations and their reasons for doing so.
  • Their level of confidence in whether the jury's verdict was the right one.
  • Their views on their ability to participate in the jury's discussions.
  • The influence they felt they had over the jury's decision.
  • Their level of satisfaction with the experience of being a juror.
  • Their understanding of the not proven verdict.
  • Basic demographic information about each juror (gender, age group, working status and highest educational qualification).

2.5 The data

Data from the mock juries was collected and triangulated from three main sources.

2.5.1 Jury 'metadata'

Information about each jury was recorded on a 'jury record sheet' by the researcher overseeing it.^[41] This included: the time the jury began deliberating, the time at which they returned a verdict, what that verdict was, and, for any near unanimity verdicts in the unanimity condition, the size of the majority.

2.5.2 Participant (mock juror) questionnaires

Ipsos MORI's specialist teams scanned and edited data from the questionnaires that jurors completed before and after deliberating. Specialist teams also coded data from open-text questions, using a framework developed by the research team following review of a sample of the data. An SPSS (quantitative analysis software) data file was produced for analysis by the research team.

2.5.3 Jury deliberations

All deliberations were audio and video-recorded. Audio recordings were professionally transcribed. The research team coded their content thematically with the use of NVivo (qualitative analysis software), using a code frame developed by the team after reviewing a sample of deliberations.

The research team also reviewed the video-recordings of the deliberations using an observation sheet.^[42] This was used both to capture the themes discussed during deliberations (as reflected in the NVivo coding), and to record elements of how juries reached their decision (for example, the tone of discussion, levels of participation, and numbers of dominant jurors). To ensure consistency, sheets were completed independently by two members of the research team for each jury.^[43]

The final data from observation sheets was merged with the jury metadata and questionnaire data for analysis.

2.5.4 Data analysis and presentation

The study generated both quantitative data (from juror questionnaires and metadata) and qualitative data (for example, about the ways in which not proven was discussed within deliberations). It also captured data at both jury and individual juror level.

Any differences in the quantitative findings presented in the main body of the report are statistically significant (at the 5% level) unless otherwise stated.^[44] Further detail about the statistical tests applied is included in Annex H.

2.6 Study limitations

In comparison with many previous mock jury studies, this research was significantly more realistic. In particular:

• It used filmed trials rather than transcripts, and jurors were directed on the legal tests to be applied.
• It included a lengthy group discussion element (some mock jury studies either omit this altogether, or allow very limited time for deliberation).
• Each mock jury included either 12 or 15 people (groups of six to eight are common in mock jury research, which is significantly smaller than the real juries the findings are intended to apply to - see Annex B for further discussion of methods and realism in mock jury research).

All these elements increase the robustness of the research and the confidence that can be placed in its conclusions. However, as with any mock jury research, it is subject to several limitations that must be kept in mind when interpreting the findings.

First, participants knew that they were not acting as jurors in a real trial. The one-hour video simulations were engaging and benefitted from input from legal practitioners to maximise realism, but they inevitably involved a substantially streamlined account of the criminal trial, with none of the periods of delay and disruption that often mark real proceedings. The vast majority of mock jurors appeared to take the proceedings seriously - as indicated by the fact that discussions between jurors who disagreed regularly became animated. However, it is not possible in any mock jury study to control for any impact that the artificial nature of the experience might have had on how they deliberated or which verdict they returned.

Second, sample size is a common issue in jury research and although the study was the largest of its type in the UK, involving almost 1,000 individual jurors, the total number of juries (64) was still relatively small. As such, it is unlikely that anything other than large differences in verdicts between juries would have been picked up. As discussed in Chapter 3, however, the much larger sample of jurors included in this study means that it is possible to identify significant differences at juror level. Though translation from juror- to jury-level findings is not straightforward, these findings indicate the likelihood of particular features tilting juries in one direction or another.

Third, the findings in this report are based on jurors' responses to two specific trials. As discussed above,^[45] the trials in this research were deliberately designed to prompt debate between verdicts, in order to test whether, all else being equal, particular features (number of verdicts, etc.) influenced jurors in one direction or another, and to encourage discussion of the not proven verdict. The fine balance of the trials was therefore appropriate to the specific aims of this study. However, the exact pattern of verdicts returned is unlikely to reflect the pattern of verdicts that would be returned by juries in a wider range of differently balanced cases. Thus, while we can reach informed conclusions from this study on, for example, whether a change in the majority required would incline more jurors towards or away from a specific verdict, we cannot estimate the likely scale of those impacts across a larger number of differently balanced trials.