Reconviction Rates in Scotland: 2009-10 Offender Cohort

6 Comparing reconviction rates across administrative areas

6.1 Reconviction rates have historically been used as a Scottish Government National Indicator and they are also included in the Scottish Policing Performance Framework. As such, they are commonly used to rank performance across different jurisdictions, such as Community Justice Authorities (CJAs), local authorities or police force areas. However, there is an inherent problem in using this approach since it implicitly assumes that a difference in reconviction rates reflects a ‘real’ difference between organisations. In reality, all systems within which these organisations operate, no matter how stable, will produce variable outcomes in the normal run of events. The questions we need to answer are therefore: Is the observed variation more or less than we would normally expect? What are the possible explanations for outcomes which show significant deviations from the norm?

6.2 In this respect, it is better to use a method of comparison that takes account of inherent variability^[5]. The funnel plot is a technique used in statistical process control and provides a simple way of determining whether differences are likely to be due to spurious or common-cause variation, rather than some special cause^[6]. Common-cause variation is the variation inherent within any system, for instance, variations in healthcare outcomes due to differences in case-mix and client characteristics, and can never be completely eliminated. Special-cause variation cannot be attributed to the inherent variability within a system and requires further explanation to identify its cause.

6.3 In effect, a process control chart allows organisations to be split into three groups: those with outcomes which are as expected (the majority of organisations in a stable system), and those with outcomes which are significantly higher or lower compared to the norm. Such differences can provide a useful start in terms of thinking about potential differences in the context within which these organisations operate, such as population composition or differences in practice, which may explain some of the difference in outcomes.

6.4 Table 13 provides the reconviction frequency rates and reconviction rates for each CJA area and Chart 7 and Chart 8 shows these rates against the number of offenders. The plot takes into account the increased variability of the smaller authorities, where a small increase in the number of reconvictions may lead to a large percentage change in the reconviction rates. Rates for CJAs which lie inside the funnel are not significantly different from the national rate, and we can then usefully focus on possible explanations for rates which deviate significantly from the national figure. In this case, the cut-off level for statistical significance is 95% (or two standard deviations from the mean): if there were no difference between CJAs apart from what could reasonably attribute to random variation, we would expect that 5 per cent of the authorities would lie outside the funnel.

Chart 7 One year reconviction frequency rate by CJA: 2009-10 cohort

6.5 Chart 7 shows that Tayside and Glasgow have higher reconviction frequency rates than one might expect, while North Strathclyde, South West Scotland and Lothian & Borders have lower rates than expected. Whilst this is useful for highlighting that there are practical differences in reconviction frequency rate between each CJA, it does not allow us to identify if this disparity is due to a variation in case-mix or a variation in practice. Case-mix in this scenario refers to the differences in offender characteristics (e.g. age, gender, crime, disposal, ethnicity, deprivation, etc).

Chart 8 One year reconviction rate by CJA: 2009-10 cohort

6.6 It is interesting to see that Chart 8, which shows a funnel plot of reconviction rates, provides similar results to those in Chart 7. Tayside and Fife & Forth Valley have higher reconviction rates and higher reconviction frequency rates than one might expect. However, in comparing the two charts it can be seen that only Lothian & Borders has lower reconviction rates than expected, whereas North Strathclyde, South West Scotland and Lothian & Borders have lower reconviction frequency rates than expected.

6.7 Chart 9 provides the standardised reconviction rates^[7] against the observed number of offenders minus expected number of offenders. This plot takes account of differences attributable to the case-mix. Since all CJAs are within the funnel it suggests that the apparent differences in reconviction rates in Chart 7 are primarily attributable to variation in the characteristics of the offenders between authorities, rather than differences in practice or performance. This conclusion on the 2009-10 cohort is consistent with the findings provided in the 2011 reconvictions publication which provided funnel plots on the 2007-08 cohort.

Chart 9 Standardised one year reconviction rate by CJA: 2009-10 cohort