We use cookies to collect anonymous data to help us improve your site browsing experience.

Click 'Accept all cookies' to agree to all cookies that collect anonymous data. To only allow the cookies that make the site work, click 'Use essential cookies only.' Visit 'Set cookie preferences' to control specific cookies.

Set cookie preferences

Your cookie preferences have been saved. You can change your cookie settings at any time.

Publication - Advice and guidance

The 5 Step Approach to Evaluation: Designing and Evaluating Interventions to Reduce Reoffending

Published
31 May 2016
ISBN
9781786522368

Updated guidance on how to use the 5 Step approach to design and evaluate criminal justice interventions.

View supporting documents Supporting documents

Judging the worth of an intervention

How can the 5-Step Approach help funders to make their decisions?

Assessing an evaluation report
The extent to which an organisation has undertaken these 5-steps could be judged using an objective scoring scale which would standardise the way interventions are judged

A standardised, objective and transparent scoring system could be developed to assess the extent to which these 4 elements have been addressed in the report, namely,

  • To what extent is the intervention based on strong and consistent evidence drawn from the results of sound research studies?
  • Is there is logic model that shows clear, evidence-based and logical links between each activity and the outcomes?
  • Has an independent and robust evaluation been carried out?
  • To what extent did the evaluation show a) that the resources (inputs) and been spent on evidence-based activities, that b) the target group were obtained c) that most completed the intervention and d) that the anticipated outcomes for users were achieved?
Example judging criteria matrix for reducing reoffending interventions Yes, No or to some extent
(Comments)		 Weighting Score (1-3)

Is there a clear rationale for this intervention? Why was it needed?

Did the target group pose a medium to high risk of reoffending?

Is intervention content (what they are going to do) described in detail?

Is there a comprehensive assessment of published robust studies?
Is this evidence clearly embedded into the design of the programme? For example, does the intervention develop individual motivation, skills and competencies?

Are there also evidence-based or at least logical links between inputs (costs), content/activities and short-, medium- and long-term outcomes?

Has an independent evaluation been carried out?

Does the evaluation collect appropriate data to test the logic model as described?

Impact - was there a control or a comparison group e.g. matched pairs? If so, what was the sample size?

Did the evaluation show that resources were spent appropriately on activities with users?

Is there evidence that activities were carried out and to a high standard?

How many were eligible? What was the throughput?

Were outcomes achieved? Was impact measured and has the intervention made a real difference?

Features and Advantages of a scoring system

  • A scoring system could be developed in collaboration with Justice Analytical Services, funders and interventions.
  • Judging criteria could be weighted according to the importance of each criteria
  • A total score could be worked out for each intervention and assessed - it even provides a basis for making objective and transparent comparisons between interventions.
  • There is a precedent for this type of scoring system - 'formal' criminal justice programmes seeking accreditation are assessed using a similar scoring system and Analytical Services use a similar system of criteria to assess bids for research projects
  • Advantages and disadvantages of the 5-step approach

Advantages and disadvantages of the 5-step approach

Advantages Disadvantages
  • Inclusive - all interventions of any size should be able to conduct this type of evaluation.
  • Giving credit for evidence-based approach and a sound model of change can offset problems with conducting 'gold standard' impact evaluations.
  • Funders could rate the quality of evaluations on a scale which allows weighting to help compare programmes.
  • A transparent and consistent scoring system would support and enable a process of 'certification' (similar to accreditation of formal programmes) which could raise the quality of interventions which in turn should reduce reoffending in the longer-term.
  • The approach is already endorsed and used to commission change fund projects.
  • Not everyone is familiar with logic models, how to embed the evidence or evaluations so evaluators and funders might need support.
  • It falls short of a quantitative and objectively verifiable measure of impact on long term outcomes.
  • In order for interventions to conduct a robust logic model evaluation, they must have sufficient time for medium term outcomes to materialise. Short funding cycles may act against this although this approach does allow other aspects of the process to be evidenced sooner, for example evidence-based practice, a clear logic model, sound implementation of activities and short-term outcomes.

Contact

Back to top