Mediation in civil justice: international evidence review

Assessment of the Evidence Base

The evidence base on mediation is variable, making comparison and transferability problematic.

There have been a number of systematic reviews conducted which focus on mediation, as well as small-scale, empirical studies, however experimental, comparable studies that would allow generalization of findings from one context to another are lacking. The most reliable studies that produce empirical data have generally been conducted in the USA.

There are some important and oft-cited systematic review articles that highlight the patchy nature of the evidence base on mediation internationally, making clear that there are a number of variables that are rarely investigated and that those that are may be difficult to draw conclusions from due to the limited nature of the studies. There are also several significant gaps in the evidence base. As Wissler states:

To date, some mediation outcomes have been examined in many studies, but other outcomes have been assessed in only a few studies… Our ability to draw clear conclusions about the relative effectiveness and efficiency of court-connected mediation, neutral evaluation, and traditional litigation is limited by the small number of studies with reliable comparative data based on the random assignment of cases to dispute resolution processes and the use of statistical significance tests.^[3]

An evaluation of the mandatory mediation system in Ontario, which examined the first
23 months of the programme in Ottawa and Toronto, is an example of robust mediation research. In total, 3810 evaluation forms were forwarded to the evaluators, who were external to the court.^[4] This large-scale dataset provides more robust evidence on experiences, viewpoints, and attitudes, allowing greater certainty to be attributed to the findings. Research in Maryland can also be considered robust, if smaller in scale. A research team in Maryland was asked by the courts to review mediator style and impact on long and short-term outcomes.^[5] This research had a carefully thought out, longitudinal design, allowing the researchers to see whether mediation continued to have positive impacts beyond the immediate or short timescale. This robust research design allows greater certainty to be attributed to findings.

The complexity of mediation within the civil justice system makes comparable studies very difficult to design and execute in a meaningful way, and they are therefore lacking. Transferability from one context to another is also problematic, due to the range of different designs, approaches, and models used, as well as local legislative regimes and types of mediation. Differences in what a particular programme may deem as success, given its own drivers (court backlog or greater access to justice, for example) also influences its design and measurement, and therefore how it can be transferred to other contexts.

To illustrate this point, the examples of an English mandatory pilot in the London County Court in 2004 and the Ontario mandatory mediation programme (evaluated in 2001) will be used.^[6] Despite both programmes being mandatory there were very different outcomes. In the English county court – where a voluntary scheme had been in place for some years, but uptake was low – the new pilot automatically referred parties to mediation, with the option of opting out, but with the risk of incurring cost sanctions for unreasonable refusal to participate; in Ontario (at this time only in Ottawa and Toronto) parties also had the ability to opt-out by applying to the judge and judicial discretion was used to decide whether to grant the request. In Canada there were only a few opt-outs, whereas in England around 80% of parties opted out, with judges communicating concerns about forcing parties into mediation. Genn points out that in England the Halsey ruling coincided with the beginning of this pilot. The Halsey judgement ruled that the court had no power to force parties to mediation and that force could be seen as a contravention of human rights.^[7] Genn states that the discrepancy between these two pilots in terms of results was:

a classic example of policies colliding and of the danger of extrapolating from one culture to another. The applicability across jurisdictions of procedural innovations depends on, among other things, the culture of litigation, the formal court structures for dispute resolution, the characteristics of disputes, and costs rules.^[8]

The majority of empirical studies in mediation are focused on short-term settlement and satisfaction rates for users, with more limited interest in wider and longer-term outcomes; cost and time savings for users and the civil justice system are also of interest.

Settlement and satisfaction rates are clear measures of success in the literature on mediation in the selected jurisdictions. The literature surveyed here was strongly focused on short-term, often immediate, assessment of these outcomes by parties, lawyers, and to a lesser extent, mediators. Research in Maryland was interested in longer-term outcomes and style of mediation, but was one of the few studies that looked at a longer-term result. There was some interest in compliance and, to a lesser extent, how parties measured their sense of empowerment, their relationship with the opposite party, and whether they felt they could use mediation techniques beyond the mediation itself to help handle disputes more effectively. Cost and time were also significant outcomes for users and the civil justice system, though empirical evidence that guarantees cost and time savings is lacking. It would appear that mediation can save both time and money, for users and the civil justice system, but that this cannot be assured for every case. This assessment of the evidence is summed up by the National Alternative Dispute Resolution Advisory Council (NADRAC) of Australia based on their assessment of the mediation evidence:

For example, the research literature on mediation suggests that rates of agreements seem to be consistent across diverse forms of mediation and service types (about 50-85%), and that there is high client satisfaction rate with mediation. Compared with adversarial methods, mediation produces higher compliance and lower re-litigation rates. Outcomes are at least as positive for mandated as for voluntary referrals. There is low awareness of ADR, and low uptake of voluntary ADR. The long term impacts, substantive fairness and overall cost effectiveness of mediation are unclear.^[9]

Defining what is considered as 'success' in mediation is not straightforward. Agreeing with our assessment of the evidence, Mack notes that settlement and satisfaction rates are the most commonly used measures of success, but that these are not necessarily sophisticated enough to be helpful in measuring the complexity of mediation.^[10] Few cases that enter the legal system actually go to trial, and the 'leaky funnel' can be hard to monitor, but is important to understanding what impact mediation may have had.^[11] Local legal culture will have an impact on the success or otherwise of ADR and makes comparison between local courts and systems difficult, therefore what measures should be used to study the 'effectiveness' of mediation is unclear.^[12] Critically, also, baselines are lacking in the civil justice system: if we do not know where we started with mediation and other ADR, we cannot be sure what impact any intervention has had.^[13] Therefore, though settlement and satisfaction rates can be recorded, their true meaning is less clear – particularly when trying to compare the impact of one system with its potential impact elsewhere. The question of success will be revisited below.

Gaps in the Evidence Base

There are a number of key gaps in the evidence base that are important to fill to better understand and measure outcomes.^[14]

• Drivers to engagement and settlement: there is limited evidence on why certain parties negotiate and settle and why others do not. Though some evidence in the USA and Canada looks to explore barriers and enablers of settlement there is no systematic evidence exploring the reasons for settlement. Though there is evidence that mandatory mediation increases take up, this does not equate to better outcomes. Free services – such as in Small Claims – may also increase uptake, but the discussion in this area remains predominantly based on assumption. With a range of settlement rates (see section 'Settlement Rates') it would be useful to know what makes a case more likely to settle and why a substantial set of cases are not settled. It is not possible to make this assessment based on the current evidence base.

• Characteristics of parties and the dispute: detailed evidence on what the case was about and the characteristics of who took part, as well as the history of the dispute and whether it then had to return to court for trial due to non-compliance or appeal, is not available. Mediation is a private and confidential process and this means such data is limited. It was also found that there had been a general shift away from close monitoring of mediation in the selected jurisdictions, with sources in Queensland citing lack of government funding for monitoring, and sources in Florida noting an ideological shift away from seeing simplistic monitoring of cases as useful.

• Private, pre-court mediation: there is a very limited evidence base on mediation outside of court processes, hence why this review focuses in the main on court-connected programmes.

• Quality of outcomes: settlement and satisfaction in the short-term are key measures of the success of mediation. However, the quality of these outcomes, particularly in the longer term, lacks evidence. The USA has stronger evidence around parties' contentment with the outcome but this is rarely followed through in the long-term to establish if it is consistent, saves further court time in appeal, whether the dispute resurfaces, or the agreement fails.

• Awareness and provision of information about mediation: though the literature mentions the importance of party and lawyer understanding of mediation, it does not give guidance on how best to educate and inform.

• Mid-value claims: Small Claims cases are frequently discussed in the literature, with mediation having good outcomes in these courts; high value claims also appear to do well in mediation, often due to maintenance of relationships, the privacy of the process, and more creative solutions available. There is limited evidence on mid-value claims.

• Behaviour of mediators, lawyers, the judiciary and other court staff: though there is some evidence on how these stakeholders see mediation and believe the mediation has gone, there is a lack of evidence on what is happening in practice, as opposed to perception of events. With each group significant to the process and outcome of mediation more research on their behaviour would be highly useful. Research in Maryland shows that mediator behaviour does have an impact on short- and long-term outcomes, but is one of the few studies to do so. With clauses in some jurisdictions demanding lawyers inform clients about mediation and other forms of ADR, it would be significant to see how this works in practice. Judges have a great deal of discretion in many areas, and their attitudes towards mediation, means of referral, and judgements about opt-out are significant. This extends to court staff when they have the duty to divert cases to mediation.

• Cost and time savings: it is difficult to accurately quantify the cost and time savings mediation can make. Though there is general agreement such savings can be made, the evidence base is limited. A more specialist assessment of this is needed.

• Negative and unintended consequences: there is the potential for negative and unintended consequences from mediation, particularly when it is seen as a panacea. More research and evidence is needed on these issues, such as how power differentials manifest during the mediation, what case types mediation is most suitable for, and whether settlement is always a positive outcome. Some further comment will be made on this below.

Approach to Selecting Studies for Inclusion

The review took the form of a literature review and discussion with key informants. It was not comprehensive in nature, but explored the available literature and evidence on mediation in the civil justice systems of the selected jurisdictions.

This review is not systematic or comprehensive, but designed to provide an overview of the operational features of each jurisdiction and to provide available evidence on outcomes for users and the civil justice system for each jurisdiction. Evidence was selected for inclusion based on several criteria: (i) the robust nature of it; (ii) how frequently it was cited in other research; (iii) the relevance of its findings to key outcomes. The outcomes that were selected as key came from initial reading of the literature and discussion with policy colleagues. It was very clear that settlement and satisfaction rates are key measures of mediation success and these were therefore included, as well as efficiency issues around cost and time savings. Beyond this the literature and key informant discussions suggested that there were wider outcomes of significance too: compliance, maintenance of relationships, and ability to handle disputes more effectively in the future. These were evidenced to a greater or lesser extent, but have been drawn out where possible.

This study ranks evidence in terms of its robustness. In social research, large-scale, generalisable (meaning comparable and transferrable) empirical studies are generally viewed as the most robust. These are strongest when they use experimental methods, with comparison available through a control group. Smaller-scale but clearly thought-out empirical research is considered robust, though its findings have to be more cautiously interpreted due to the lack of ability to generalize beyond the specific setting. Theoretical pieces, from more objective authors – academics, for example – are the next most robust pieces, with articles, blogs, and opinion pieces from those immersed in the field at a final level. Court evaluations and data are generally considered robust pieces of work, but their findings would not extend beyond reporting on that one pilot or intervention and they are therefore a step down from the most robust research. In this study, research has been considered in this way, with all of these kinds of research and writing included, but with a focus on finding, including, and relying on the most robust levels of research to a greater extent than those of lower quality.