Designing and Evaluating Behaviour Change Interventions Summary

Background: The tricky business of assessing impact in the real world

A Scottish approach to evaluation

Co-production
Our approach to evaluation enables funders and service providers to work together in pursuit of their shared aims – to improve outcomes for service users and communities. The 5 step approach also engages with service users’ views as a resource for evaluation rather than seeing users solely as an object to be measured.

Asset-based
The 5-step approach focuses on ways in which evaluation is possible for services of any size, rather than expecting all services to use an experimental evaluation method which may not be appropriate or possible for smaller, community-based organisations. The 5 step approach allows even the smallest service to demonstrate the contribution they are making to change.

An improvement culture
Evaluation enables improvement and even the most successful service can always be developed further. Furthermore, with the 5 step approach, evaluation is an on-going process, not something to be saved for last. This means that services can be continually improved in order to best meet the needs of their users.

How do you know if you are making a real difference to users?

It’s not easy to find out if you’re making a real difference to people, especially in the chaotic real world. There are hundreds of variables which can effect people’s attitudes, motivations and behaviour. So how can you tell if your project is making any difference?

Researchers and scientists generally agree that best way to determine if your project or service has made a difference is to use a randomised control trial (RCT), sometimes referred to as an 'impact evaluation' but these are not easy to do in practice, especially in a complex social setting.

An alternative to RCTs

A 'middle ground' approach

Rather than carrying out a small RCT which might be impractical and would only deliver meaningless results, we recommend that small-scale project organisers carry out a 5 step approach to evaluation. This is summarised in the following slides and detailed in the remainder of this pack.

This approach to evaluation is practical for projects of any size but does rely on providers having a clear sense of what they’re hoping to achieve and how they’re going to get there – a theory of change. For this reason, using the 5 step approach, must begin at the planning stage.