SUMMARY JUSTICE REFORM: UNDERTAKINGS EVALUATION

Appendix A: Methodology

Core Approach

The evaluation comprised a mixed methods approach that combined analysis of secondary data, collection of primary qualitative data from interviews and questionnaires, and a parallel cost-benefit analysis exercise. A staged approach was taken so that findings from early stages could inform the design and content of the later stages.

Secondary Data Analysis

Secondary data analysis focussed mainly on Key Performance Indicator (KPI) data from the Scottish Government's Criminal Justice Board Management Information System (CJBMIS) and the Monitoring Workbook. This included data from all partners agencies involved in the administration of undertakings including ACPOS, the SCS and COPFS. The KPI data analysed included:

• Number of undertakings issued;
• Number of standard prosecution reports submitted to COPFS;
• Alleged offences processed by use of undertakings;
• Use of undertakings conditions;
• Number of warrants granted for failure to appear at pleading diet or continued pleading diet while on an undertaking;
• Number of convictions for breach of undertaking conditions;
• Main penalty given for breach of undertaking;
• Percentage of accused appearing by undertaking on first calling in court within 28 days of caution and charge;
• Average number of diets per case;
• Percentage of summary criminal cases dealt with within 26 weeks; and
• Average time from caution and charge to verdict.
• Following initial analysis of the KPI data and discussions with key stakeholders, four case study areas were selected in which to concentrate the qualitative research. These were based upon Local Criminal Justice Board (LCJB) areas, and were Ayrshire, Central, Lothian and Borders, and Glasgow and Strathkelvin.

Interviews with Key Stakeholders

In-depth interviews were conducted with a range of key stakeholders. This included 10 interviews with Sheriffs, six interviews with Justices of the Peace, nine interviews with Procurators Fiscal, nine interviews with Defence Agents, and 20 interviews with police operational supervisors (Inspectors and Sergeants). A total of eight group interviews were also conducted with police Constables with operational experience of undertakings.

As part of the evaluation, questionnaires were also issued to a small number of accused arrested and detained by the police where they were known by police to have previously been released on an undertaking. These participants were asked to comment on previous cases where they had been given an undertaking (not on the current incident for which they had been arrested), and their responses were provided confidentially to the evaluation. Using this approach, a total of 11 responses from accused released on an undertaking were achieved.

Cost-benefit Analysis

A limited cost-benefit exercise was also attempted to assess whether the benefits i.e. savings generated by the reforms to undertakings, were sufficient to outweigh the corresponding burdens arising from the reforms. This encountered several challenges, not least being a lack of available data to inform its execution rendering a full economic analysis not possible. Instead, the evaluation considered the likely impact of the reforms on the workloads of the main criminal justice agencies involved in the administration of undertakings, as well as the impacts on failure to appear and repeat rescheduling of cases which may all have associated costs to the system.