Reconviction Rates in Scotland: 2010-11 Offender Cohort

6 Comparing reconviction rates across administrative areas

6.1 Reconviction rates have historically been used as a Scottish Government National Indicator on Scotland Performs and they are also included in the Scottish Policing Performance Framework. As such, they are commonly used to rank performance across different jurisdictions, such as Community Justice Authorities (CJAs), local authorities or police force areas. However, there is an inherent problem in using this approach since it implicitly assumes that a difference in reconviction rates reflects a 'real' difference between organisations. In reality, all systems within which these organisations operate, no matter how stable, will produce variable outcomes in the normal run of events. The questions we need to answer are therefore: Is the observed variation more or less than we would normally expect? What are the possible explanations for outcomes which show significant deviations from the norm?

6.2 In this respect, it is better to use a method of comparison that takes account of inherent variability⁸ . The funnel plot is a technique used in statistical process control and provides a simple way of determining whether differences are likely to be due to spurious or common-cause variation, rather than some special cause⁹ . Common-cause variation is the variation inherent within any system, for instance, variations in healthcare outcomes due to differences in case-mix and client characteristics, and can never be completely eliminated. Special-cause variation cannot be attributed to the inherent variability within a system and requires further explanation to identify its cause.

6.3 In effect, a process control chart allows organisations to be split into three groups: those with outcomes which are as expected (the majority of organisations in a stable system), and those with outcomes which are significantly higher or lower compared to the norm. Such differences can provide a useful start in terms of thinking about potential differences in the context within which these organisations operate, such as population composition or differences in practice, which may explain some of the difference in outcomes.

6.4 Table 10 shows the unadjusted reconviction frequency rates and reconviction rates for each CJA area and Chart 6 and Chart 7 show these rates against the number of offenders. The plot takes into account the increased variability of the smaller authorities, where a small increase in the number of reconvictions may lead to a large percentage change in the reconviction rates. Rates for CJAs which lie inside the funnel are not significantly different from the national rate, and we can then usefully focus on possible explanations for rates which deviate significantly from the national figure. In this case, the cut-off level for statistical significance is 95% (or two standard deviations from the mean): if there were no difference between CJAs apart from that which could reasonably be attributed to random variation, we would expect that 5 per cent of the authorities (i.e. only 1 of them) would lie outside the funnel.

Chart 6 One year reconviction frequency rate by CJA: 2010-11 cohort

6.5 Chart 6 shows that Tayside and Glasgow have higher reconviction frequency rates than one might expect, while North Strathclyde, Lanarkshire and Lothian & Borders have lower rates than expected. Whilst this is useful for highlighting that there are practical differences in reconviction frequency rates between each CJA, it does not allow us to identify if this disparity is due to a variation in case-mix or a variation in practice. Case-mix in this scenario refers to the differences in offender characteristics (e.g. age, gender, crime, disposal, ethnicity, deprivation, etc.).

Chart 7 One year reconviction rate by CJA: 2010-11 cohort

6.6 It is interesting to see that Chart 7, which shows a funnel plot of reconviction rates, provides similar results to those in Chart 6, which shows a funnel plot of reconviction frequency rates.

6.7 Chart 8 provides the standardised reconviction rates¹⁰ against the observed number of offenders minus expected number of offenders. This plot takes account of differences attributable to the case-mix. Since all CJAs are within the funnel it suggests that the apparent differences in reconviction rates in Chart 7 are primarily attributable to either the variation in the characteristics of the offenders, the type of crime they committed, or the sentence they received, rather than differences in 'performance'. This overall conclusion for all authorities on the 2010-11 cohort is consistent with:

(a) the findings provided in the 2011 reconvictions publication (which provided funnel plots on the two year reconviction rates for the 2007-08 cohort); and

(b) the findings provided in the 2012 reconvictions publication (which provided funnel plots on the one year reconviction rates for the 2009-10 cohort).

Chart 8 Standardised one year reconviction rate by CJA: 2010-11 cohort