The 5 Step Approach to Evaluation: Designing and Evaluating Interventions to Reduce Reoffending

The 5-Step Approach

The 5-step approach to evaluation

	Identify the problem		If your ultimate aim is to change people's attitudes, emotions or behaviour, you need to be clear and explicit what it is you are trying to change and why there is currently a need for this to happen.
	Review the evidence		Interventions should be clearly structured and designed using robust evidence so it is important to be familiar with the results from relevant 'what works' and desistance evidence-base. If the aim of the intervention is more specific, for example to promote recovery from drug addiction or to improve parenting skills then also track down the relevant evidence-base and embed the findings into how the service works.
	Draw a logic model		A logic model is a simplified diagram which shows, step-by-step, why the activities you plan should achieve your aims. The logic model forms the basis for evaluating the whole project - you are going to test whether these steps happened as you predicted.
	Identify Indicators and monitor your model		Use the logic model to identify indicators (i.e. measurements or observations) that things actually happen as you predicted. You will need to collect data about your project FROM THE START on inputs, activities, users, short, medium and long-term outcomes.
	Evaluate logic model		Analyse the data you've collected on your various indictors to evaluate how well your project worked for your various users. Report on whether your data suggests the logic model worked as planned. Be honest about any areas which were less effective. Use this to improve your service.

The 5-step approach: A summary

1. Identify the problem
It is essential that you are clear from the start about the problem you are aiming to address. What exactly are you trying to change and why is this is needed at this particular time and place? Perhaps there are local gaps in service provision or recent events which suggest intervention would be timely.

2. Review the evidence
The most effective projects and services build on strong and consistent evidence about what works to reduce crime or what helps offenders desist from crime and they also learn from previous experiences. Therefore, the 5-step approach puts a deliberate emphasis on using existing evidence and the evaluation should measure the extent to which each component of your service is based on good evidence. The first step is therefore to understand the results of 'what works', effective practice for staff and desistance studies. If your service is focused on achieving a particular intermediate outcome it is worth reviewing specific evidence on relevant areas such as recovery from addiction, effective parenting, emotion management, mentoring and throughcare in order to plan your service. You should also learn from previous experiences to continuously improve the service.

3. Draw a logic model of how your service should work
The logic model is a step-by-step diagram which shows the ultimate outcomes (change or results) you are aiming for and step-by-step how you intend to achieve them. It details inputs (e.g. money, staff, resources) needed to deliver your activities and how they should lead to short, medium and long-term outcomes and ultimately meet your aims. It is useful to think of longer term outcomes as wider social change that you are contributing to and that only collaboration will produce long lasting social change. In this sense, logic model outcomes vary in terms of how much influence your project has over them and in turn, how accountable your project is for achieving them.

It should describe how evidence, funds and staff will be used to design and deliver activities and how exactly, based on your review of the existing evidence, these activities are expected to lead to short, medium and long term outcomes. Your project won't operate in a vacuum so don't forget to identify external factors which could help or hinder the achievement of outcomes. These could be policy changes, the economic climate or the level of support for your project receives from your organisation .

A template and excellent guide can be found here: http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html

4. Identify indictors and collect monitoring data
Using your logic model as a guide, identify a) priority evaluation questions and b) indicators that will test whether the project actually worked as the logic model predicted. You should collect data on what activities were delivered to whom, as well as evidence that they led (or didn't lead) to the short-term and longer-term changes you anticipated. Collecting and analysing data can be resource intensive so agree what is most important to know from the start and why you and/or your stakeholders need to know it and be realistic about what questions can and can't be answered.

NB. It is important that you collect 'base-line' (pre-project) information about your users to compare with information you later collect during and after the intervention.

5. Evaluate logic model
You now need to analyse the data you've collected in order to test whether the project worked in accordance with your logic model. You should assess how well activities were delivered, levels of user engagement and whether users' needs were met or their attitudes changed. Case studies can be used to illustrate examples of who the service worked for and did not work for and why that might be.

WARNING!

Do not leave planning your evaluation until the end of your project

• Steps 1-3 should be carried out before the project begins
• Step 4 (monitoring) should continue from the very start to the end of your project (and, ideally, beyond).
• Step 5 (analysis) should not be left to the end either. Interim and on-going evaluations will enable you to make improvements to your project or service.