The challenge of population balance: mapping Scotland's institutional and intervention landscape

Annex 1: The Theory of Change Approach

Policy development is often weakened by implicit (and frequently unrealistic) assumptions. The ToC approach aims to “surface and articulate” (Connell and Kubisch 1998) the chains of logic, and associated contextual assumptions, which occupy the space between inputs/interventions and final goals. ToC belongs to a wider family of methods of studying intervention logic, which includes:

• Intervention/Implementation/Mechanism Mapping (Corbie-Smith et al. 2010, Kok et al. 2011, Cherrington et al. 2012, Oconnor et al. 2021, Geng et al. 2022, Fernandez et al. 2023)
• Logic Modelling and Logic Frameworks (MacLeod 2016)
• Results Based Management (Wauters, 2012)
• Realistic Evaluation (Tilley and Pawson 2000)

The functions which can be carried out by ToC, are not confined to policy design, but also include monitoring/management, and evaluation. Since it is essentially collaborative, it can also play a powerful role in establishing consensus and building community capacity. ToC methods can be used at any point in the policy process, from ex ante, right through to ex post.

The term Theory of Change was first used by Carol Hirschon Weiss in 1995, in a paper which set out to improve the effectiveness of “Comprehensive Community Initiatives” carried out by third sector and philanthropic organisations, aimed at improving the lives of individuals and households in the most deprived communities of the US.

The original motivation for the ToC approach was the pursuit of clarity of purpose and more effective orchestration of multiple activities by a plurality of actors towards a common (but often difficult to specify) long-term goal. Such initiatives often tackled complex social problems, characterised by intricate and recursive networks of cause and effect, which were also very much affected by place-based contextual constraints. Simple linear models of impact (involving policy inputs, stand-alone processes, and outputs) were considered unrealistic. The solution suggested by Weiss and her colleagues was a collaborative exercise, involving all relevant actors, beginning with a careful articulation of the ultimate goal, and then working backwards through a causal chain from outcomes, and intermediate outcomes, taking account of constraints and feedback effects, in order to “reverse engineer” the intervention(s) required.

In recent years ToC has been promoted by the Centre for Theory of Change, a notfor-profit organisation based at the City University of New York. A recent EU Horizon project (RELOCAL) devised a graphical format adapted from the work of the Centre for ToC (Copus et al. 2019, Piras et al 2022) as a basis for systematic comparative analyses of 33 European interventions promoting different forms of spatial justice.

Reverse engineering begins from the final product and works backwards. The ToC approach emphasises the need to always keep in view the final goal. In terms of RUSTIK, this means a better handling of one or more of the three transitions.

For example, (Figure 2) the rural demographic challenge is often driven by net outmigration and associated ageing. Realistically, mitigation of this process can only be achieved through discouraging outmigration of young people, and encouraging the in-migration of younger families. So, working backwards from the nature of the challenge, local policy makers might identify a set of policy actions which might include (i) providing more opportunities for local youngsters, (ii) increasing the attractiveness of the area as a destination for young families from elsewhere, and (iii) supporting in-migrant families to help them settle in. Within each of these actions there would be a set of “stepping-stone” intermediate outcomes, which could be monitored to provide an assessment of their effectiveness. The impact of each of these policy actions could be enhanced if more was known about the motivation of in and out migrants. So, a good initial link in the chain of intermediate outcomes would be some kind of data experiment to collect such information.

All of this makes sense in theory. However, in some rural areas demographic shrinking is so strongly established that, realistically, any policy can only hope to slow it down. Absolute growth is not likely to be achievable. In ToC terms this is represented by the concept of an “accountability ceiling”.

A final aspect of the ToC perspective is transferability between contexts. A logical chain of intermediate outcomes may, or may not, hold good if applied to a different region, a different policy context, with different administrative structures, institutional history and welfare system. All of these contextual “assumptions” need to be clarified if “external validity” (Williams 2017, 2018) is to be achieved.

References:

Cherrington, A., Martin, M., Hayes, M., Halanych, J., Wright, M., Appel, S. Intervention mapping as a guide for the development of a diabetes peer support intervention in rural Alabama. Prev Chronic Dis 2012;9:110053. DOI: http://dx.doi.org/10.5888/pcd9.110053

Copus, A. Piras, S., Tobiasz-Lis, P., Dmochowska-Dudek, K., Wójcik, M., Napierała, T., (2019) Synthesis Report: Towards an Operational Concept of Spatial Justice, Deliverable 8.2, RELOCAL project. Available at: https://relocal.eu/wp- content/uploads/2020/01/D8.2-Synthesis-Report-Towards-an-Operational- Concept170120.pdf accessed 25th January 2024

Corbie-Smith, G., Akers, A., Blumenthal, C., Council, B., Wynn, M., Muhammad, M. and Stith, D., 2010. Intervention mapping as a participatory approach to developing an HIV prevention intervention in rural African American communities. AIDS Education and Prevention, 22(3), pp.184-202.

Connell, J., and Kubisch, A., 1998. Applying a theory of change approach to the evaluation of comprehensive community initiatives: progress, prospects, and problems. New approaches to evaluating community initiatives, 2(15-44), pp.1-16.

Fernandez, M.E., Powell, B.J. and Ten Hoor, G.A., 2023. Implementation Mapping for selecting, adapting and developing implementation strategies. Frontiers in Public Health, 11.

Geng EH, Baumann AA, Powell BJ (2022) Mechanism mapping to advance research on implementation strategies. PLoS Med 19(2): e1003918. https://doi.org/10.1371/journal.pmed.1003918

Macleod, C.J.A., 2016. How can logic modelling improve the planning, monitoring and evaluation of policy measures and wider interventions for multiple benefits?. Final Report to Scottish Government.

Piras, S., Tobiasz-Lis, P., Currie, M., Dmochowska-Dudek, K., Duckett, D. and Copus, A., 2022. Spatial justice on the horizon? A combined Theory of Change scenario tool to assess place-based interventions. European Planning Studies, 30(5), pp.952-973.

Wauters, B., 2012. Sourcebook on Results Based Management in the European Structural Funds. Community of Practice on Results Based Management. https://www.betterevaluation.org/sites/default/files/sourcebook_tusseninres.pdf

Weiss, C.H., 1995. Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families. New approaches to evaluating community initiatives: Concepts, methods, and contexts, 1, pp.65-92.

Williams, M. (2017) External validity and policy adaptation: a fivestep guide to mechanism mapping. Policy Memo. Blavatnik School of Government. University of Oxford. [https://www.bsg.ox.ac.uk/sites/www.bsg.ox.ac.uk/files/documents/BSG-WP- 2017019_0.pdf Accessed 10^th July 2017]

Williams M. (2018) External Validity and Policy Adaptation: From Impact Evaluation to Policy Design. Working paper. [ https://www.semanticscholar.org/paper/External-Validity-and-Policy-Adaptation-%3A-From-to-Williams/797e2d96dc9467ce94ea1e54e03a2e34467c36ce accessed 4th April 2019]