Designing and Evaluating Behaviour Change Interventions

Guidance for service providers, funders and commissioners

The 5 Step Approach

The 5 step approach to evaluation

	Identify the problem		If your ultimate aim is to change people’s behaviour, you need to be clear what it is you are trying to change and why there is currently a need for this to happen.
	Review the evidence		What you intend to do should be grounded in the evidence of ‘what works’ and why. Service providers should review the available evidence in order to plan activities which can be expected to achieve the intended behaviour change. The evidence should guide what you do and help you to understand the process through which it should work.
	Draw a logic model		A logic model is a diagram which shows, step-by-step, why the activities you plan should achieve your aims. The logic model forms the basis for evaluating the whole project – you are going to test whether these steps happened as you predicted.
	Identify Indicators and monitor your model		Use the logic model to identify indicators (i.e. measurements or observations) that things actually happen as you predicted. You will need to collect data about your project FROM THE START on inputs, activities, users, short, medium and long-term outcomes.
	Evaluate logic model		Analyse the data you’ve collected on your various indictors to evaluate how well your project worked for your various users. Report on whether your data suggests the logic model worked as planned. Be honest about any areas which were less effective. Use this to improve your service.

The 5 step approach: A summary

It is essential that you are clear from the start about the problem you are aiming to address. What kind of behaviours are you aiming to change and why is this is needed at this particular time and place? Perhaps there are local gaps in service provision or recent events which suggest intervention would be timely.

2. Review the evidence

The most effective projects and services build from existing evidence about what works – they learn from previous experiences. Therefore, the 5 step approach puts a deliberate emphasis on using existing evidence and the evaluation should measure the extent to which your service is based on evidence. The first step is therefore to gain an understanding of the existing evidence base in order to plan your service.

3. Draw a logic model of how your service should work

The logic model is a step-by-step diagram which shows the ultimate outcomes you are aiming for and step-by-step how you intend to achieve them. It details inputs (e.g. money, staff, resources) needed to deliver your activities and how they should lead to short, medium and long-term outcomes and ultimately meet your aims.

It should describe how evidence, funds and staff will be used to design and deliver activities and how exactly, based on your review of the existing evidence, these activities are expected to lead to short, medium and long term outcomes.

Examples of logic models can be found on pages 16 - 20  and a template and excellent guidance can be found here:
http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html

4. Identify indictors and collect monitoring data

Using your logic model as a guide, identify indicators  that will test whether the project actually worked as the logic model predicted. You should collect data on what activities were delivered to whom, as well as evidence that they led (or didn’t lead) to the short-term and longer-term changes you anticipated.

Nb. It is important that you collect ‘base-line’ (pre-project) information about your users to compare with information you later collect during and after the intervention.

5. Evaluate logic model

You now need to analyse the data you’ve collected in order to test whether the project worked in accordance with your logic model. You should assess how well activities were delivered, levels of user engagement and whether users’ needs were met or their attitudes changed. Case studies can be used to illustrate examples of who the service worked for and did not work for and why that might be.

WARNING!

Do not leave planning your evaluation until the end of your project

• Steps 1-3 should be carried out before the project begins.
• Step 4 (monitoring) should continue from the very start to the end of your project (and, ideally, beyond).
• Step 5 (analysis) should not be left to the end either. Interim and on-going evaluations will enable you to make improvements to your project or service.