Pre-recorded evidence and juror decision-making: evidence review

2. Understanding Jury Simulation Methods

This chapter begins with a brief account of the search criteria used to underpin this Evidence Review. As noted above, the primary focus is upon research studies that rely upon trial simulations to develop insight into juror attitudes and jury outcomes. The majority of discussion in this chapter is devoted, therefore, to highlighting some key methodological considerations that must be borne in mind when evaluating the persuasiveness of such studies. Researchers are often required to balance expediency and the need to isolate specific variables with the imperative to develop realistic and engaging trial stimuli in their study design.

This chapter highlights the difficulties that can result in terms of the external validity – that is, the relevance to the ‘real’ world – of studies that rely on brief vignettes, trial transcripts, or short and overly simplified trial reconstructions. It also reflects on the somewhat uncertain implications of relying on convenience samples drawn from the student population as jury participants, and highlights the considerable importance of including measures for tracking influence that are not only transparent but also embedded within the realities of the task of collective verdict deliberation which confronts jurors in actual criminal trials. This methodological understanding is necessary to effectively evaluate the strength of resultant substantive claims in relation to the use of pre-recorded evidence.

2.1 Parameters of the Literature Search

In terms of inclusion criteria, it was originally intended that the search would encompass material published between the years 2000 to 2017, but it soon became apparent that a number of significant studies were published prior to this, so these were also included in the review. The search focused on the main English speaking jurisdictions that use juries in criminal cases, namely Scotland, England and Wales, Northern Ireland, the Republic of Ireland, Canada, the Australian jurisdictions, New Zealand and the US jurisdictions; but did not exclude empirical research undertaken by researchers working in other jurisdictions where it was clearly of relevance.

In order to identify relevant sources, an extensive search of electronic databases was undertaken. This encompassed legal databases (such as Westlaw, Lexis Nexis and HeinOnline), scientific databases (such as PsychARTICLES), multidisciplinary databases (such as JSTOR and Ingenta Connect) and the databases of the major academic journal publishers (such as Wiley Online Library, Sage Journals Online and Springer Link). Some of these databases (such as JSTOR) also index academic books, but a specific search of other sources (such as Google Scholar and library catalogues) was undertaken to identify relevant material in book format. A snow-balling technique (checking the references cited in each study) was then used to identify relevant soures of evidence that may not be indexed electronically. In addition, a further search was undertaken to identify relevant Government reports (by searching Government websites and national libraries in relevant jurisdictions) as well as reports published by law reform bodies such as Law Commissions and third sector bodies.

Following these searches, and preliminary review of all papers and reports of potential relevance, 52 papers of direct relevance were identified and evaluated. These papers – alongside others pertaining to the surrounding context – are fully referenced in the discussion below. Priority of focus has been given within the Review to research studies that have relied on the most rigorous methodologies, arisen in legal systems of most relevance to Scotland, and / or been undertaken most recently, on the basis that these hold the greatest evidential weight.

2.2 Simulation Studies: The Importance of Method

A number of different methods might be used to generate insight into the factors (including in relation to mode of evidence delivery) that influence jurors. Observation of courtroom proceedings can be instructive to a degree, as can interviews with practitioners. However, to the extent that such methods rely on the subjective interpretation (whether by the observing researcher or the practitioner) of jurors’ body language and other signaling behaviours during trial proceedings, or conjecture based on verdict outcome about what likely occurred behind the closed doors of the jury room, they can only provide partial insights and may not be especially robust. In particular, in situations where – as is the case in this Evidence Review – the objective of research is to test in a relatively controlled fashion the effect of a specific trial factor, these methods are often of limited value. Thus, while observational and interview-based research will be drawn upon where applicable and appropriate in this Evidence Review, its primary focus will be upon studies that set out to generate and evidence ‘first hand’ insights into the dynamics and substance of jury deliberations.

In several jurisdictions, including Scotland (s. 8 Contempt of Court Act 1981 ), restrictions have been imposed that prohibit any direct questioning of real jurors in respect of the content of their verdict deliberations. These restrictions, though supportive of the solemnity and sanctity of trial proceedings, impose a significant obstacle on our ability to properly understand verdict outcomes, and the factors that influence jurors’ decision-making. In an effort to overcome this, researchers (particularly psychologists) have developed a variety of methods for stimulating, tracking and recording juror influences. Such jury simulations hold significant potential to bridge (at least partially) the existing knowledge gap between what happens in the courtroom and resultant verdict outcomes. However,the reliability of their findings hinges upon their methodological rigour and the degree to which they have attended to the evidential and practical realities of the trial environment they seek to emulate. It is fair to say that, in this respect, jury simulations to date have been rather mixed. Indeed, as discussed below, there has been a wide spectrum in relation to the complexity and realism of the trial stimulus relied upon, diverse approaches to the recruitment and demographic representativeness of participants, and considerable variety in terms of the measures utilised to track jurors’ substantive responses.

2.3 Trial Scenarios and Simulations

In their review of existing jury simulations, Lieberman and Sales acknowledge that there have been a number of studies in the past in which the research instruments relied upon are “so far removed from the dynamics of an actual trial that it is difficult to place high confidence” in their findings (1997: 592). In some cases, only a very brief trial vignette has been used; in others, excerpts from a written trial transcript have been relied upon. Such written formats, though relatively easy to set up, deprive participants of a key source of information available to real jurors, who are known to devote considerable attention (for better or worse) to the non-verbal behaviour of the parties. Other researchers have relied instead on a videotape of a trial re-enactment. This allows observation of trial parties’ demeanour and personal characteristics, but it remains unclear whether observing a videotaped trial creates a distance to proceedings which reaffirms jurors’ sense of the artificiality of the simulation. This may be particularly relevant in this Evidence Review, where some studies have specifically set out to explore the impact of video-mediated testimony whilst situating it within the broader context of a videotaped trial. Where researchers have instead used live trial re-enactments, they have generally concluded that they provide more realistic and engaging stimuli (Finch & Munro, 2008; Ellison & Munro, 2014). It should be noted, however, that live re-enactment comes with a risk of subtle variation in delivery across trial conditions, which, depending on the focus of the study, may create problems.

Whether the stimuli are presented in written or visual form, there are additional challenges associated with the experimental context. Resource constraints entail that many of the more mundane delays associated with real jury service cannot be replicated and ‘shortcuts’ in evidence and procedure are required in order to capture the key facts and legal arguments in an appropriate time-frame. Moreover, given that simulation studies are typically designed to test the influence of a particular set of conditions, it is common practice to minimise participants’ exposure to too many other factors by restricting the stimuli scenario (Kramer & Kerr, 1989). Concerns have been expressed that this may encourage jurors to rely on conjecture and stereotype to ‘fill in the gaps’ in a way that may not occur in a real trial (Visher, 1987; Diamond, 1997).

In some studies, ‘shadow’ juries have been utilised instead of mock simulations. Participants are recruited to observe the unfolding of a real trial from the public gallery and then asked to deliberate under the observation of the researchers. This addresses, of course, the above concerns regarding the realism of the trial stimuli. However, not only is this a cost-intensive method, it is also one tied very concretely to the particular trial observed, providing a one-time-only stimulus that cannot be manipulated to build up an understanding of the scale, function and consequences of a specific variable (for example, the characteristics of the parties or a particular fact). Thus, this method provides rich but relatively limited insight; and its findings relate confidently only to factors that proved influential for shadow juries, with limits remaining on the ability to extrapolate to the real jury even within the same case.

Both simulation and shadow jury methods encounter further challenges, moreover, as a result of the inevitable knowledge on the part of participants that they are not acting as real jurors in the case before them. The significance to be attributed to this role-playing dimension is unclear, however. Existing research does not support the implication that mock jurors will necessarily take their task less seriously, at least to the extent that this is reflected in their propensity to make legal mistakes vis-à-vis real jurors (Reifman et al, 1992); and there is evidence of mock jurors engaging conscientiously with their role and expressing stress regarding their verdicts (Bray & Kerr, 1987; Finch & Munro 2006, 2008; Ellison & Munro, 2010a). Moreover, studies that have set out to test for a verdict impact as a result of role-playing have produced inconclusive results (Wilson & Donnerstein, 1977; Zeisel & Diamond, 1978; Kerr et al, 1979).

2.4 Sampling Issues

A large proportion of existing jury simulation studies have relied on undergraduate students to act as participants, often in exchange for course credit. While this offers a convenient and cost-efficient approach to sampling, students do not represent a typical cross-section of the jury-service eligible population, at least in demographic terms (Sears, 1986). What impact this has on jury deliberations remains unclear, however (Bornstein, 2017). Simon and Mahan (1971) found that students displayed attitudinal differences which made them more likely to acquit than jurors drawn from a more representative jury pool; meanwhile, Sealy and Cornish (1973) found, to the contrary, that students were predisposed to be harsher on the accused than the ‘typical’ juror. Of course, it might be argued that the significance of using student participants lies not so much in attitudinal differences vis-à-vis the general population, but more in cognitive differences which will affect their ability to engage with the deliberation process (Miller et al, 1977; see also Weiten & Diamond, 1979). While there is some support for this, it is worth noting studies which have shown no statistically significant differences between student and other jurors in relation to, for example, their understanding of judicial instructions (Rose & Ogloff, 2001).

A further consideration to be borne in mind is the self-selecting nature of participants. Unlike real jury service, which can be compelled by the state, participants engage voluntarily in simulation studies. Attention to demographic profiles can ensure that samples match broadly either to the general population or to the profile of those who act as jurors locally or nationally, but self-selecting research participants may still differ attitudinally or cognitively from their ‘real’ counterparts. They may take part because of a particular interest in the subject matter or an inherently more socially conscious or altruistic nature, for example (Braunack-Mayer, 2002). It is currently unclear, however, what impact this may have on their approach to the substantive task.

2.5 Measures of Influence

Many researchers have asked participants to rate their evaluation of particular factors associated with the trial scenario, or their preferences for a verdict, on graded scales. This provides an externally observable measure that can be useful for statistical analyses, but it may not yield particularly valuable insights into the real process of jury deliberation, where a collective and binary verdict is required.

This maps on to a broader distinction within this field of existing research as between mock juror and mock jury studies. In the former, the individual is taken as the primary unit of analysis. In the latter, the jury is recognised to be more than the sum of its individual parts and group decision-making is built in as an integral part of the study design (MacCoun & Kerr, 1988; Young et al, 1999; Devine et al, 2004). In conjunction with the concerns raised above regarding jurors ‘filling in the gaps’ in abbreviated trial reconstructions, deliberation may be particularly important. Indeed, previous studies have maintained that within group discussion, “the knowledge, perspectives and memories of the individual members are compared and combined, and individual errors and biases are discovered and discarded, so that the final verdict is forged on the shared understanding of the case” (Ellsworth, 1989: 206).

Where deliberation has been included as a component of the research design in previous studies, it is also worth noting that there has been divergence regarding the size and composition of mock juries, as well as the duration of discussion and the level of direction given to jurors on the pertinent legal tests. Several studies have departed from the jury numbers required in real cases within their jurisdiction to make discussions more manageable, particularly within constrained timescales. Few studies have used mock juries of 12 members or more, with most using 6 or 8. The impact of this on the tone, substance and outcome of deliberations remains unclear. While a number of studies have suggested that jury size does have an impact on tone and outcome (Lempert, 1975; Roper, 1980), others have concluded that it does not (Mills, 1973; Kerr and MacCoun, 1985; see also Mukhopadhaya, 2003).

2.6 Creating an Evidence Base

Generating stronger evidence from jury simulations requires a study design that involves carefully constructed and controlled stimuli, developed with an eye to the realities of the courtroom, using a representative sample of participants, and invoking measures to track juror influence that include a period of collective verdict deliberation. But even with this more rigorous approach, there remains in simulation studies a number of confounding variables relating to the type of trial, the fact pattern underpinning it, the attributes of the parties and their legal counsel, the substantive and procedural rules of the legal system, and the attitudinal preferences and discursive dynamics at play within any given jury. As subsequent chapters illustrate in respect of the impact of pre-recorded evidence in particular, this can make it difficult to extrapolate across individual studies to confidently identify a coherent set of findings.