Review of the Aberdeen Problem Solving Approach: report

2 Methods

This chapter details the methods used to address the aims of the Review.

2.1 Summary

A mixed method approach was adopted which combined primary qualitative data with secondary analysis of routinely collected monitoring data. The primary research undertaken is summarised in Table 2.1. It provided rich data on participants' experiences, and the perspectives of participants, professionals and wider stakeholders on: how the PSA is being delivered; what distinguishes it from other sentencing procedures; what is working well; what could be improved; and lessons for other areas. The court observations enabled further comparisons with other sentencing procedures. The analysis of the monitoring data provided some quantitative data on the profile of participants (including risk/needs assessment), engagement, compliance and sentencing outcome.

Table 2.1: Summary of methods

Audience/court proceeding	Method	Number	Length of interview/ group/ observation
PSA participants	Face-to-face individual in depth interviews	11 (2 men in the community, 3 men in custody, 4 women in the community, 2 women in custody)	Maximum of 45 minutes
Sheriff Clerk Depute and Procurator Fiscal	Paired depth interview	1 (2 participants)	1 hour, 50 minutes
CJSW social workers and support workers	Mini group	1 (4 participants)	1 hour, 20 minutes
CJSW manager	Face-to-face individual in depth interview	1	50 minutes
Sheriffs	Face-to-face individual in depth interviews	3	40 minutes – 1 hour, 50 minutes
Defence agents	Mini group	1 (4 participants)	45 minutes
Wider stakeholders	Face to face individual in depth interviews	4 (2 x Aberdeen Women's Centre, 1 x Police Scotland and 1 x Venture Trust)	Maximum of 45 minutes
PSA review/sentencing hearings	Observation	7 (includes 2 where the PSA participant failed to attend)	5 – 15 minutes
Community Payback Order ( CPO) / Drug Treatment and Testing Order ( DTTO) review/sentencing hearings	Observation	13 (6 CPO sentencing hearings, 5 CPO review hearings, 2 DTTO review hearings)	5 – 12 minutes

2.2 Qualitative research

A total of 11 PSA participants and 18 professionals were consulted in this research, and 20 court observations were undertaken. All qualitative research was conducted face-to-face by members of the research team between October 2017 and January 2018. The interviews and focus groups were structured around discussion guides (see Appendix 1), designed by the research team in consultation with the Scottish Government. Interviews were audio-recorded (with participants' permission). The transcripts of recordings and interviewer notes were then systematically analysed to address the Review's aims.

2.2.1 PSA participants

The total number of PSA participants is still relatively small: at the time of the Review fieldwork, 35 cases had been closed and there were 16 live cases. Eleven PSA participants were recruited to the research via members of the CJSW team. The research team liaised with CJSW to outline the desired number and profile of PSA participants. The aim was to conduct 14 interviews and to cover a spread in terms of:

• gender
• outcomes (including those who had successfully completed and those who had not)
• whether they had exited the PSA or were still on it.

As anticipated, the recruitment of PSA participants was challenging due to the difficult circumstances many of them faced. Social workers were unable to contact some of the potential participants while others did not consent to take part. By the end of the fieldwork period, 11 interviews had been conducted. A spread was achieved in terms of the above factors. The Scottish Prison Service assisted in arranging the five interviews with PSA participants who were in custody. These interviews took place in four prisons/ young offender institutions across Scotland.

All potential research participants were provided with an information sheet ( Appendix 2) in advance of the research which explained: the purpose of the research, what taking part would involve and that everything they said would be confidential and anonymous. They all also signed consent forms ( Appendix 3) immediately before the interviews.

The anonymised stories of four participants have been used to form case studies to illustrate points made throughout the report (see Jillian's Story, Craig’s Story, Kelly’s Story and Angela’s Story).

2.2.2 Professionals involved with the PSA

Court and CJSW staff assisted the research team in the identification and recruitment of key professionals. All professionals who were invited agreed to participate ( Table 2.1). Professionals consented to participate, understanding that, given the limited number of professionals involved with the PSA, complete anonymity could not be guaranteed in reporting their views, although names have not been used.

2.3 Court observations

The research team observed court proceedings in Aberdeen Sheriff Court on six separate days between 11 October and 8 November 2017.

Seven PSA review hearings were observed. This allowed for both of the current PSA sheriffs, as well as both male and female PSA participants, to be observed.

As a comparison, 13 Community Payback Orders ( CPOs), or Drug Testing and Treatment Orders ( DTTOs) were also observed ( Table 2.1). The CPO hearings took place in the mainstream court while the DTTO and PSA hearings took place in a smaller court.

The aim of the observations was to collect information on:

• the processes, timings and physical context of the hearings
• the topics discussed
• the communication styles used
• the level and type of participation of those in attendance.

To guide the observations a court observation tool was developed by the researchers in collaboration with the Scottish Government.

2.4 Analysis of monitoring data

Data routinely collected by the CJSW team was analysed as part of the Review. An anonymised dataset containing all monitoring data collected from the inception of the PSA until September 2017 was provided to the research team. The data was cleaned and analysed using SPSS software. The analysis was descriptive in nature (due to the small numbers involved).

2.5 Interpreting findings

The research took the form of a review. This approach was considered the most appropriate means of meeting the research aims i.e. to provide feedback on the Aberdeen PSA to policy makers and the Aberdeen Court Service and to inform Community Justice Partners about the potential use of problem solving approaches in other summary courts in Scotland. The Review was conducted as planned. However, the following are possible limitations of the research design and the available data:

• the Review was restricted to reporting on perceived short term outcomes. This means that analysis was informed by qualitative data on PSA participants' and professionals' perceptions of the outcomes. Other than the routine monitoring data (which included social workers' assessments on factors such as engagement, compliance and final outcome), there was no quantitative data available on outcomes.
• the extent to which the Review can draw firm conclusions about the approach's impact is also limited by the small number of participants who have taken part in the PSA. This is particularly true for men as the men's programme has been running for a shorter time. The women's PSA started in November 2015 and the men's in August 2016. By September 2017, 30 women and 18 men had participated.

In relation to the monitoring data, it is worth noting that:

• unique IDs for each individual were deliberately not included in the monitoring data for data protection reasons; given the small size of the sample, they may have allowed the identification of individuals. The absence of unique IDs meant that it was not possible to identify how many people had been screened for the PSA on how many occasions though this was otherwise not a problem for this Review.

APSA area for improvement
Tracking longer-term outcomes, including reoffending, would require a means of linking the relevant data. It may be possible to access Scottish Offenders Index data anonymously with the relevant URN once the numbers of cases grow to the point that an individual possibly becoming identifiable is no longer an issue.

• there may also have been data entry errors that could not easily be identified. We undertook a number of cross-checks on the data and made a very small number of adjustments when inconsistencies emerged. In a few cases, we went back to CJSW for further information or to check that the assumptions we made were correct.