The 5 Step Approach to Evaluation: Designing and Evaluating Interventions to Reduce Reoffending
Updated guidance on how to use the 5 Step approach to design and evaluate criminal justice interventions.
Appendices
Appendix 1: Example proposal for an evidence-based intervention
Why is this service needed?
There are 2 clear reasons for providing this service
1. There is a gap in current provision. The Scottish Prison Service have identified a gap in services aimed at improving prisoners' social, communication and emotion-management skills. There is currently no prison-based service available to address these needs in prisoners.
2. There is strong and consistent evidence that this intervention should help contribute to reducing reoffending.
We have drawn on robust research evidence to demonstrate how our planned activities should lead to prisoners developing better communication and emotional skills which should help them deal more effectively with people, achieve positive goals (such as finding employment) and desist from crime after liberation. We have drawn from reviews of the evidence rather than single studies which can be biased or unreliable. We have drawn on the following sources:-
- Systematic and literature reviews on 'what works' to reduce reoffending to provide justification for throughcare and the needs the service will address .
- A range of qualitative research studies which suggest how the service should be implemented to achieve outcomes and data provided by the Scottish Government on prisoners' reconviction rates.
What outcomes do we want to achieve?
Evidence - what factors contribute to reducing reoffending?
The findings from international systematic reviews of 'what works' to reduce reoffending provides strong and consistent evidence services need to target criminogenic needs to reduce reoffending.
A systematic review published by the Ministry of found a clear association between poor social and emotion management skills with reoffending, therefore they are both justifiable targets for interventions aimed at reducing reoffending. Other dynamic criminogenic needs have been found to be criminal peers, drug use and criminal attitudes.
There is fairly strong evidence that that better outcomes are achieved if services are holistic, well-structured and continue after release (reference).
What outcomes will we be trying to achieve?
This intervention will focus mainly on improving offenders social and emotion management skills. To ensure that this intervention forms part of a wider holistic service which will address multiple needs, this intervention will link into existing throughcare processes as one of the range of services that prisoners can be referred to in prison and in the community.
The community-based part of the intervention will also encourage liberated prisoners to join sports and leisure groups which should help ex-prisoners build pro-social networks.
Therefore this intervention will focus on addressing 3 needs that have been found to be associated with reoffending - improve social skills, improve emotion management and develop social networks
Which activities will achieve outcomes?
Evidence for linking this intervention with existing throughcare processes
Although there is a lack of controlled experimental studies which show that throughcare reduces reoffending, there is a growing body of evidence that throughcare plays an important role in helping short term prisoners reintegrate back into the community and that throughcare can enhance the effectiveness of prison-based interventions (add reference). With particular relevance to social skills and emotion management, some research studies have shown that throughcare can enable prisoners to practice skills they have learned once they return back to their communities which will help them deal with real life situations, seek work, control aggression and engage positively with non-offending peers (add reference).
As there are existing throughcare processes, we have designed this intervention to align with the processes and with the main tenets of effective throughcare which are listed below. According to an international review of throughcare published in 2013 by SCCJR,
- Throughcare should start as early as possible following sentencing
- Pre-release planning is important and prisoners should be involved in devising these plans
- There should be continuity of provision through the gate
- Supporting prisoners to practice their skills in the community embeds more positive behaviour.
How will our service link with throughcare?
In accordance with this evidence, this service will engage prisoners who are referred to us as early as possible and we will also offer one-to-one support available to prisoners on an appointment basis. Once liberated, prisoners will be able continue to improve and practice their social and emotional management skills in the community through the provision of a community-based structured programme.
As there is strong evidence that criminal peers place offenders at a high risk of reoffending, once released the users will be encouraged to find work, join local sports or social clubs of interest to the users so they can forge relationships with non-offending peers. Where possible, links to clubs and employers will be made prior to release to smooth the transition. The community-based service will support prisoners for at least 6 months after release or until a time they are confident in using positive social skills without further support.
Evidence on effective techniques to improve social skills and emotional management
There is strong and consistent evidence from systematic reviews that show CBT is the most effective technique at improving social skills, managing aggression and reducing reoffending (add reference).
What approach will we use in our sessions?
Specialist workers who are trained and experienced in motivating offenders using CBT techniques will run structured interventions within the prison and in the community. These will improve communication and emotion management skills though using CBT approaches, participatory role play within peer groups.
Within the prison, peer groups will be used to practice holding conversations, allowing prisoners to learn from each other how to express ideas and develop effective techniques which help them deal with anger and frustration. As stated above, it is important that once released, prisoners also start to build relationships with pro-social peers.
Evidence on required intensity of support
Risk, Needs and Responsivity principals show that the intensity or 'dosage' of supervisions needs to be relative to the risk of reoffending (add reference).
Our approach to intensity of support
As this group are at a high risk of reoffending (see participants section), this project will provide intensive support. This will take the form of one-to-one support on an on-going appointment basis and two, one hour sessions per week designed to improve social and emotion management skills. This frequency will continue in the community as appropriate to individual users.
Evidence on what increases motivation
Research also shows that targeting this group may present problems with lack of motivation and engagement as the target group may be extremely resistant to receiving support. However research has found there may be rational reasons for a lack of motivation (e.g. poor experiences of interventions in the past, chaotic life-styles which act as a barrier to participation, being forced to own up to their offending, guilt, fear of ridicule from others and a lack of self-efficacy etc).
Our approach to motivating prisoners
Therefore our staff are trained in evidence-based methods to motivate reluctant offenders by developing trusting relationships, showing willingness to focus on and discuss life goals and discuss how they could achieved, agreeing shorter term treatment goals, explaining processes and activities that will be fully explained and by using enjoyable tasks. This should help prisoners to feel motivated and sustain engagement with the programme.
Who will we reach?
Evidence on selecting appropriate target groups
RNR principals suggest that offenders with a medium-high risk of reoffending are most likely to benefit from structured interventions compared with those who are low-risk or very high risk.
Offenders who at a high risk of reoffending tend to have the following characteristics,
- They have a number of prior convictions
- They are serving short term sentences
- They have multiple criminogenic needs
Data on the Scottish prison population provided by the Scottish Government confirms that prisoners serving short term sentences (6 months or less) who have multiple prior convictions have the highest reconviction rate out of all prisoner groups. Approximately 70% of prisoners under 30 serving under 6 months with more than 10 prior reconvictions were reconvicted within a year compared with 50% of all prisoners serving short term sentences.
Which target group will we reach?
This project will therefore target the most prolific male prisoners who are
- Serving sentences of under 6 months
- Have over 10 prior convictions
The total number of prisoners across the whole estate who should be eligible for the service per annum is based on figures for 2010-11 cohort which suggests 1264 prisoners would fall into this category. As this is an intensive service, which requires considerable time spent with prisoners we aim to target 30 offenders per annum.
Appendix 2: An example data collection framework
Example data collection framework for a criminal justice intervention
Part of logic model analysed | Indicators | Data collected from (data source) | Data recorded in… | Data entered into database as… |
---|---|---|---|---|
INPUTS |
||||
Were there sufficient resources to run the intervention and how were they deployed?
|
|
Manager and staff Annual accounts |
Intervention level database |
Costs, values and views Costs can be reviewed periodically (e.g. annually) |
Part of logic model analysed | Indicators | Data collected from (data source) | Data recorded in… | Data entered into database as… |
---|---|---|---|---|
PARTICIPANTS Collect information on your users to check that you reach your intended target group.
|
User ID number |
N/A |
User level database |
Entered as 01, 02, 03 etc |
Name |
User |
User level database |
Name |
|
Date of birth |
User |
User level database |
Date of birth |
|
Age at start of programme |
User |
User level database |
Age |
|
S number (if possible) |
Police/CJSW |
User level database |
S number |
|
Gender |
User |
User level database |
Column - Gender |
|
Local Authority |
User |
User level database |
Assign code for each LA area |
|
Current index offence |
SOI |
User level database |
Code crime types e.g. |
|
Needs e.g.:
Risk
|
LSC/MI |
User level database |
LSC/MI scores and record of needs for each user |
Part of logic model analysed | Indicators | Data collected from (data source) | Data recorded in… | Data entered into database as… |
---|---|---|---|---|
ACTIVITES |
||||
What did users experience?
|
Date of intervention start/completion |
Case record |
User level database |
Enter dates in xx/xx/xxxx format |
How many users had their needs assessed? |
Case record |
User level database |
For each user |
|
|
Intervention supervisors |
User level database |
For each user, create columns for each module or activity that was supposed to be undertaken by the individual. For each activity record intended number of sessions, session duration, content etc. Create a column for if they completed the intervention as a whole e.g. |
|
|
Case record |
User level database |
For each user, assign a code for whether each activity was completed or not. For example: |
|
How did users experience the intervention?
|
|
User |
User level database |
Assign codes to closed responses For example, user views on session length - or relationship quality - Poor = 1 or usefulness Very useful = 1 Analyse qualitative open questions (not entered into data base) |
Part of logic model analysed | Indicators | Data collected from | Data recorded in… | Data entered into database as… |
---|---|---|---|---|
SHORT- and MEDIUM-TERM OUTCOMES |
||||
Did change happen? Quantitative measures of change
|
For example, record the score on psychometrics tests (if appropriate). Record where users started from on key outcome variables such as attitudes, needs, skills, knowledge, awareness, status, views , feelings, behaviours or competencies. |
User and intervention supervisor |
User level database |
Create two columns-one for the outcome variable before and one column for after the intervention . For example Communication skills and 'able to express ideas' (after): Or HousingHas stable accommodation (before): Yes = 1 No= 2 Has stable accommodation (after): |
Qualitative measures of change
|
|
Users friends and family Intervention Supervisor User and family |
User level database |
Could code answers e.g. Supervisor views of change 'Made progress' And/or transcribe interviews for more depth information |
LONG-TERM OUTCOMES |
||||
Did the intervention reduce reoffending? |
Statistical differences in outcomes between a control group and the treatment (user) group |
Reconviction data |
Separate analysis conducted |
Contact
There is a problem
Thanks for your feedback