Evaluation of the Community Reintegration Project

Annex A: Research Methods

1 This was a mixed-method evaluation, with an emphasis on qualitative data collection. The general approach adopted was based on the importance of understanding how the CRP process was working in practice, in order to consider the implications for the development of throughcare for short-term offenders.

2 The evaluation involved an analysis of available monitoring data and interviews with different groups involved in the CRP process. The project also included a validation event allowing stakeholders to reflect on the project findings prior to the drafting of the report. Further details of the evaluation methods are presented below.

Monitoring data

3 A data collection system was established by SPS as part of the original CRP pilot process. The system involved recording progress through the system for each eligible offender on an Excel spreadsheet. Information gathered included the following:

• Initial engagement/disengagement with the process; and subsequent disengagement at later stages of the process.
• Completion of key milestones in the process (e.g., Comprehensive Screen, Standard Reviews, pre-release meeting).

4 Information was gathered by SPS from each participating site and made available to the evaluation team.

5 There were, though, a number of issues which needed to be addressed with the data initially gathered by SPS. Completion of the spreadsheet by individual establishments was patchy and there were inconsistencies in the way the information was recorded at different establishments (variation in fields and information entered), and SPS liaised with the individual establishments to improve the data quality over the last six months of the pilot period. The research team worked with SPS to clean the data and achieve a more reliable, consistent data set for each prison before merging the data into a single spreadsheet containing key information for analysis purposes. Basic descriptive analysis of the data allowed an overview of the operation of the pilot to be presented, with a particular focus on the period October 2013 to March 2014.

6 As well as the prison based-data, SPS also collected information from participating sheriff courts on the transfer of CJSW reports, again with a focus on the period October 2013 to March 2014. This information was also made available to the research team.

Qualitative interviews

7 The main focus of the research was qualitative interviews undertaken with different groups involved in the CRP process. This approach was adopted as it represented the best method for exploring the range of views and experiences of those with an interest in the process from a range of different perspectives. A total of 69 interviews was carried out (including one group interview involving five interviewees and two joint interviews) over the course of the project, as summarised in Table A.1 below.^[18] The approach taken in relation to each group of interviewees is outlined below.

Prison staff

8 A total of 20 face-to-face interviews was carried out across the four participating prisons. Interviewees were nominated by a key CRP contact at each establishment who was asked to identify around four individuals who were involved in different aspects of delivering the CRP process. Interviewees included residential staff and managers (14), non-residential staff (Link Centre, offender outcomes etc) (5), and administrative staff (1). Interviews lasted around an hour. In addition a single interview was carried out with a member of staff at Castle Huntly given that there was the potential for CRP participants from Perth to be transferred to the Open Estate during the course of their sentence.^[19]

Criminal justice social work staff

9 Face-to-face interviews (including one joint interview) were carried out with 11 staff members in criminal justice social work teams in the two relevant local authority areas. The lead social worker dealing with CRP in each area was invited to nominate appropriate members of staff, again with a view to covering all CRP-related activities, both during the custody stages and following the return of offenders back to the community. Interviewees included social workers (4), social work support staff (6) and administrative workers (2). Interviews ranged from around 20 minutes for administrative staff to around an hour and a half for those involved in the day-to-day work of dealing with CRP clients.

Court staff

10 The sheriff clerk and/or the appropriate sheriff clerk depute were interviewed at each of the participating sheriff courts. In all six court staff contributed to the evaluation via one individual interview and one group interview. These interviews focused on the role of the courts in transmitting electronic copies of the CJSW report to the appropriate prison for CRP-eligible offenders.

Community-based agencies

11 Interviews were carried out with representatives of eight community-based agencies. In all, nine interviews were carried out involving 10 members of staff - separate interviews were carried out with two different members of one organisation giving the benefit of a frontline and strategic view, while one interview at another agency involved two members of staff both of whom were involved in frontline service delivery). All but one of the interviews was carried out in person; the remaining interview was carried out by phone. The interviewees were selected on the basis of recommendations from the local CJSW teams with a view to reflecting the key offender issues of housing, addictions and benefits and, in the case of women offenders, family relationships, and sought information on the experience of providing services and support to CRP participants. In many cases specific knowledge and awareness of the CRP process was limited and interviews often represented a more general exploration of dealing with the needs of offenders following release from prison.

CRP participants

12 Twenty-two interviews were carried out with CRP participants in custody: eight male offenders and 14 female offenders. Assistance was sought from a key CRP contact at each prison in identifying and recruiting appropriate CRP participants to take part in the study. The key contact was provided with criteria to use in selecting potential interviewees, with a view to covering a range of characteristics including age, sentence length, number of times in prison, and stage reached in the CRP process.

13 Five interviews were undertaken with participants in the community: three women and two men. Similar to the process adopted in the prison setting, CJSW staff were asked to identify and recruit potential interviewees. For women offenders, appointments were set up by the CJSW team; for male offenders the research team were given contact details to allow interviews to be arranged once consent was obtained from the individual - these were then carried out by phone.

14 Securing interviews with CRP participants in the community proved challenging, and the original target of eight interviews was not reached. This was always anticipated to be a difficult client group to involve in the research; however, the fact that CRP participants - men in particular - disengage from the process very quickly following their return to the community increased the challenge for the research. Elsewhere, however, the research reached - and exceeded - its target number of interviews.

Table A.1: Summary of achieved fieldwork interviews

	Per prison (x4)	Open Estate (x1)	Per LA area (X 2)	Per court (x4)	Total
SPS staff at pilot prisons (including personal officers, residential managers, Link Centre staff and others)	Perth: 4 CV: 6 E: 4 G'nck: 6				20 Target 16
SPS staff (manager) at OE		1			1
LA CJSW staff (including support workers, social workers and admin staff)			Lanarkshire 4 Dundee 7		11 (including one joint interview) Target 8
SCS staff (Sheriff Clerk or nominee)				Dundee 1 Lanarkshire courts (3) 1	2 (one individual / one group interview) Target 4
Offenders pre-release	Perth: 8 CV: 6 E: 3 G'nck: 5				22 Target 17*
Offenders post-release			Lanarkshire 3 Dundee 2		5 Target 8
Other practitioner groups in community-based agencies			Lanarkshire 4 Dundee 5		9 (including one joint interview) Target 6
TOTAL (Target 58)					70

*The overall target number of interviews for offenders in custody was 17 based on achieving three interviews in each prison running the CRP for female offenders (CV, Edinburgh and Greenock) and eight in Perth, running the CRP for male offenders.

15 The rationale behind the approach to sampling and recruitment of CRP participants - i.e., enlisting the help of prison and community staff to recruit in line with a set of agreed criteria - is worth commenting on. Although the potential for bias was recognised, the number of potential interviewees and the tight timescale for the project made this approach an efficient option. For women in custody the available numbers of offenders eligible for the CRP process in each establishment was small, and reaching the target number of interviews was likely to involve close to100% of those eligible and so there was limited scope for bias. For men in custody, this approach offered flexibility, limiting the scope for non-availability of interviewees on the day of the fieldwork visit. For CRP participants in the community, the approach was informed by advice from CJSW staff who indicated that in a high proportion of cases people disengaged from the process fairly quickly following release from prison. There was, thus, only a small number of 'live' cases at any one time. CJSW teams were, however, asked to consider if there were disengaged ex-clients who they felt could be usefully approached, and the research included one such interview.

16 Interviews with CRP participants sought to explore views and experiences of the CRP process and the impact of the process on the individual. However, it should be noted that explicit awareness of the process amongst participants was limited. Individual interviewees were often unable to recall specific CRP stages or were unable to distinguish CRP activities from other prison interactions and activities. The information gained from these interviews provided valuable insight into the general experience of needs assessment and receiving support in custody but did not in all cases provide feedback on identifiable aspects of the CRP process specifically.

17 Interviews with CRP participants in custody lasted around 30 to 40 minutes. Interviews with participants in the community were somewhat longer, reflecting the fact that the participant had reached a later stage in the throughcare process.

Validation event

18 A research validation event was held during the early stages of the analysis and attended by representatives of key stakeholder groups (i.e., SG, SPS, individual prisons and local authority CJSW teams, SCS and third sector groups). This provided valuable input to help ensure that the findings and conclusions of the evaluation resonated with both policy makers and practitioners, were well attuned to the realities of the policy and practice environment, and could fully support policy development work in this area.

The analysis

19 Analysis of the monitoring data was carried out using SPSS to provide basic information about the operation of the CRP in relation to the numbers and characteristics of participants and activity at different stages of the process. The interviews carried out across the various groups gathered a great deal of qualitative information about the working of the CRP process. All the interviews were transcribed with thematic analysis then undertaken, allowing the research team to see the range of views offered on each theme. The results of both the quantitative and qualitative are drawn on in addressing the research questions as presented in the remaining chapters of this report.