Community-led regeneration approach: review

6. Theory of Change and What Lies at the Heart of PCF Implementation

Introduction

Chapter 6 presents findings from two ToC workshops and the Realist analysis of the mini interview data which were captured at the project workshops.

In Realist Evaluation, 'mechanisms' contribute to, or cause, outcomes. Contexts (including intervention activities and strategies) are believed to 'fire' mechanisms. As such, contexts do not influence outcomes directly and independently, but through the firing of 'mechanisms'. The Glossary at the end of the report provides further detail. The aim of Realist Analysis is therefore to identify and refine specific Context, Mechanisms and Outcome Configurations ( CMO-Cs).

CMO-C example: participating in a bespoke training course (context), meant the beneficiary felt the course content and delivery methods were relevant (mechanism) to their goals, so motivating them (mechanism) to complete all of the course and achieve a qualification (Outcome).

This chapter deals predominantly with attempts at refining learning about what were important underlying mechanisms linked to the most frequently reported outcomes. On this basis, a small number of CMO-Cs are developed to demonstrate some of the core mechanisms and their relationships to outcomes and contexts of the PCF. As such, this Chapter should not be seen as the main learning about the impact of the overall PCF approach but as providing a partial explanation as to why the outcomes described in Chapter 5 have occurred, reinforcing these findings and informing the conclusions and recommendations.

An outline of the Programme's ToC and Realist approaches is detailed in Chapter 2 and Appendix A . These approaches were requested by the Scottish Government to identify and refine the theories and assumptions underpinning the PCF.

The theory-driven approaches included the following research and design activity:

• A Research Advisory Group workshop took place before the fieldwork which clarified the original PCF ToC, agreed underlying assumptions and identified the key contexts, mechanisms and outcomes of interest to the review;
• These key contexts, mechanisms and outcomes informed the design of the overall review, coding framework and analysis;
• The learning from the workshop focused the research on early (rather than longer-term) outcomes achieved for anchor organisations and staff as well as beneficiaries;
• The workshop process identified co-production as the most important context - this led to the design and use of specific research tools (the development ladders and the mini interviews); and
• A second workshop was conducted after the fieldwork to present findings and consider how these should be incorporated into a revised ToC.

Figure 6.1: Original Theory of Change

People and Communities Fund Theory of Change

Inputs	Outputs		Change
	Activities	Participation	Learning	Action	Leaning and actions contribute to positive and sustainable 'outcomes'
People in the community's knowledge, time and reflections on lived experiences Community anchor organisation forms and structures (for community member involvement e.g. panels, forums, community board members etc.) Community anchor organisation resources Partners and professionals' expertise and time Partners' Strategies for community engagement that makes clear the aims, objectives and outcomes deriving from activity PCF Funding (accessed directly by community anchors)	Community anchor works with people in the community to identify the issues in need of addressing using: surveys, focus groups, public meetings, informal methods and so on. (Engagement) Community anchor works with local and national delivery partners and people in the community to decide what should be done about the issues; using workshops, meetings, study visits and so on. (Co-production) Community anchor takes responsibility to deliver the action identified; using management of initiatives, ongoing community feedback and so on (Co-delivery).	A cross section of people from the 'community' Community anchor organisation People in the community who are beneficiaries of services (don't necessarily participate until 'action' phase) Local and national delivery partners, ideally including community planning partner involvement. Professionals	People in the community gain understanding of: local area other community members ways to participate professional culture potential influence People in the community are motivated to: think about local issues and solutions engage participate communicate aspire People in the community gain skills including: confidence communication skills engagement skills Community Anchors, CPPs and Professionals gain improved understanding of: community issues local context community priorities the value of community-led work potential solutions what works and what doesn't	People in the community: participate make social connections develop their skills access services support each other respect and trust each other and professionals co-operate with professionals influence decisions Community Anchors, Partners and professionals: co-operate with community members respect and trust community members share or give away power to community members enable/facilitate community members' participation develop their skills respond to community members' needs and priorities change their working practices People in the community benefit from: increased range of accessible services increased co-ordination of services improved access to services improved management of services potentially increased ownership and management of community assets	Tackling Poverty People have improved life chances People have improved wellbeing Social Inclusion People in the community: are empowered are resilient have social capital have sense of belonging Barriers to participation are removed Community is cohesive Community is inclusive People live in healthy environments People live in safe environments People have access to quality services People have access to quality facilities People have access to sustainable services and facilities for the long-term

ToC Assumptions and Realist Contexts Mechanisms and Outcomes

The initial PCF ToC was the focus of the first workshop and is presented in Figure 6.1 (above).

The key underlying ToC assumptions and learning from the workshop were:

• An acknowledgement that the population/community needs assessments conducted by PCF projects were most likely completed prior to or as part of the PCF application process rather than post funding;
• Re-affirming that co-production was a necessary programme ingredient/activity and a 'unique selling point' of PCF (and indeed a pre-defined condition for funding);
• The process of co-production would bring benefits for all stakeholders, i.e. Community Anchors, staff, partner organisations, and beneficiaries;
• Tailoring support /services on the basis of beneficiaries' needs was expected as part of the co-production process to develop bespoke /personalised support or services;
• The PCF approach requires a long timeframe to achieve significant target population impact within and across funded projects, given the focus on our most disadvantaged communities. As such the research focused on short-term outcomes for Community Anchor organisations and beneficiaries who had sustained engagement with the PCF project;
• PCF is primarily a catalyst for improvements in Community Anchor organisations, their co-production processes and the development of bespoke/tailored and ideally sustainable services;
• The services enabled by PCF may in time be scaled-up and/or support the partners such as the Community Planning Partnership ( e.g. via Service Level Agreements);
• The initial ToC model had not separated anticipated programme mechanisms from outcomes. This was done following the first workshop to help the study address the realist questions of 'why' change had /had not occurred; and
• PCF underpins the ethos of the Community Empowerment (Scotland) Act 2015.

Table 6.1 highlights examples of the priority contexts, mechanisms and outcomes identified at the first ToC workshop. These were generated as independent lists rather than linked CMO-Cs.

Table 6.1: Examples of PCF ToC Priority Contexts, Mechanisms and Outcomes

Contexts	Mechanisms	Outcomes	Negative/unintended outcomes
Target groups Whole community, BME communities, young people, rural communities	Beneficiary Confidence, self-worth, perception of control, pride	Beneficiary Knowledge and skills, employability, improved finances, social/networks capital, wellbeing	Beneficiary Isolation, dependent
Theme Training, advice & support	Staff/Partner/ organisation Trust, legitimacy, reciprocity	Staff/Partner organisation Skills, improved: partnerships, motivation	Staff/Partner organisation Overstretched, demotivated, cynical
Type of Community Anchor. Development Trust, Registered Social Landlord, Social Enterprise, Voluntary Sector	Volunteer Achievement, fulfilment	Volunteer As for beneficiaries	Volunteer De-motivated
Community Anchor characteristics Experience level, co-production process/experience, volunteer support, extent of service tailoring	Community Anchor Reputation	Community Anchor Improved: engagement, collaboration, sustainability	Community Anchor Overstretched, conflicted

The coding framework and all research tools were, therefore, developed to uncover learning about changes in a range of anticipated short-term outcomes - those in Community Anchors, partner organisations and engaged beneficiaries, and to investigate the mechanisms and linked contexts most likely to have contributed to the achievement of these outcomes.

Evolving Realist theories / CMO-C's

Realist Evaluation is interested in why, as well as whether, an intervention works, and in why it works in some contexts and not in others ( e.g. for some people in certain situations and not others). In order to understand why an intervention does or doesn't work realist analysis follows a particular sequence of steps:

• first, outcomes are established;
• then, mechanisms that are present when outcomes are or are not achieved are identified; and
• finally, the contexts associated with those mechanisms that have contributed to the desired outcomes are explored.
• This learning is then used where feasible to establish/refine more specific CMO-Cs.

Frequently Reported Outcomes and Mechanisms

The most frequent outcomes reported across all participants (n=120) - those reported by more than half (n=60) - are shown in blue on the right-hand side of Figure 6.2 (below). Those outcomes bordered by red were also the most commonly reported by beneficiaries (n=65). Those with green text were the most commonly reported by Community Anchor staff (n=39).

Figure 6.2 also presents the most commonly reported mechanisms (independent of outcomes) across all participants - those reported by more than half of participants (n=60) are shown in in yellow. Again, red borders and green text indicate mechanisms most commonly reported by sub groups - beneficiaries and anchor staff respectively.

The other mechanisms shown in Figure 6.2 represent those relatively less commonly reported overall - these have no fill or border colour. It should be noted that these were still recorded by many participants (although less than half) and some were more strongly linked to specific outcomes than those mechanisms most commonly reported independent of outcomes. They have, therefore, still contributed to outcomes.

Interpreting Figure 6.2

The outcome of 'Improved skills, knowledge and undertsanding' was reported by more than half of all participants (coloured blue). Broken down by participant group, this outcome was also reported by more than half of the beneficiaries (bordered in red) and more than half of the Community Anchor staff (green text).

By comparison the outcome of 'Improved aspiration' was reported by more than half of overall participants. Broken down by participant group, the colour codes show that more than half of the beneficiaries experienced this outcome, but a smaller proportion of Community Anchor staff.

Similarly the mechanism of 'pride' was reported by more than half of all participants. Broken down by participant group, this was also reported by more than half of the beneficiaries but by less Community Anchor staff.

However, the mechansim of 'belonging/connectedness' was reported by more than half of all participants and more than half of each of the sub groups - beneficiaries as well as Community Anchor staff.

Figure 6.2: Frequently Reported Outcomes and Mechanisms

Mechanisms associated with Commonly Reported Short/Interim Outcomes

Further analysis focused on the mechanisms most strongly associated with the commonly reported short/intermediate outcomes (from all participants). Figure 6.2 shows via arrows the mechanisms most often reported when specific common outcomes were recorded as 'present' compared to 'not present' ^[8] .

By way of an example, when participants reported the outcome of 'improved skills, knowledge and undertsanding' they were more likely to report feeling pride, confidence, self esteem and trust than any other mechansims.

Subsequent analysis (where possible) identified the mechanisms most strongly linked to the commonly reported outcomes by the sub groups (beneficiaries and Community Anchor Staff) as well as overall participants - using the same method. Figures illustrating these associations are contained in Appendix D . Again, other mechanisms on all of the models may contribute to the short-term outcomes but not as strongly as those linked.

Contexts Most Frequently Reported in Relation to Common Mechanisms

Context data were recorded from the mini interviews. These interviews were used to uncover participants' personal journeys, their feelings and experiences about aspects of PCF project processes. An analysis similar to that conducted for the links between mechanisms and outcomes was undertaken to assess the frequency of the project contexts that were present when the four most common mechanisms (belonging, pride, confidence and trust) were.

Table D.3 in Appendix D shows the results of the above analysis. The areas highlighted show where the highest levels of reporting for each of the measures conducted was found.

The key findings of the analysis presented in the Appendix D (Table D.3) are summarised below in Table 6.2 (below):

Table 6.2: Most frequently reported contexts of the four common mechanisms

Common Mechanism	Most frequently reported Contexts
Trust	'Trust' was experienced where a detailed assessment of needs was conducted, where people felt welcomed, had time to talk and where an opportunity to network and socialise was given.
Belonging	'Belonging' was experienced where a detailed assessment of the community member's needs was conducted and more opportunities were offered or encouraged. The ability to socialise/network and the provision of feedback may also be important for 'belonging'.
Pride	'Pride' was experienced when people felt welcomed and had been given time to talk as well as having a needs assessment and the ability to socialise/network.
Confidence	The contexts associated with 'confidence' are a needs assessment and having/being encouraged to take up more opportunities as well as the diverse services being offered and taken up.

The most consistently associated contexts across the range of mechanisms were:

• The range of 'opportunities on offer/encouraged';
• A 'needs assessment offered';
• Being 'made welcome/having time to talk'; and
• 'Opportunities to network and socialise'.

Provision of feedback was associated with belonging but not consistently with the other frequently reported mechanisms.

The context of 'co-delivery and volunteering' was reported as a frequent context in relation to belonging and pride based on simple frequencies but not consistently across all the measures or across other common mechanisms on which we focused. However, the responses to the development ladders reported in Chapter 5 clearly indicate that many Community Anchors were already experienced across several areas of engagement and co-production whilst many beneficiaries reported progression upwards on the ladders, thereby demonstrating the positive contribution of the community-led approach.

The contexts most frequently associated with the common mechanisms illustrate that the strengths of PCF projects are their bespoke/personalised support leading to tailored opportunities from a wide range of services. The findings from the group work in the workshops reinforce this learning.

Validation and Revision of the Theory of Change

The Realist Analysis and ToC workshops reinforced findings detailed in Chapter 5 . Together the findings have confirmed or refined several aspects of the initial PCF ToC in relation to activities and outcomes as well as associated mechanisms and contexts.

The timing of community needs assessments

Assessments of population/target group needs for developing key services were conducted prior to PCF funding. However, many projects provided ongoing assessment of the needs of their beneficiaries as part of delivering person-focused services. This was delivered throughout the PCF implementation period.

Anticipated PCF outcomes

All participant groups (Community Anchor staff, beneficiaries and partner organisations) reported many positive outcomes from PCF. The most commonly reported outcomes were:

• Improved skills, knowledge and understanding (overall and for Community Anchor staff and beneficiaries);
• Increased engagement and participation, influence in co-design, delivery and greater uptake of services (overall and for anchor staff and beneficiaries);
• Improved networks/local knowledge and connectedness (overall and for anchor staff);
• Improved aspiration (overall and for beneficiaries);
• These outcomes were all anticipated as part of the learning, action changes or sustainable actions in the original ToC; and
• Anticipated long-term outcomes such as improved well-being & social capital were also commonly reported overall, and improved life chances reported by beneficiaries specifically.

The above adds support for these 'claims' in the initial theory.

Underlying mechanisms

The initial ToC also mentioned anticipated changes in confidence, trust and inclusion/belonging. These were confirmed by the research. They are shown in the revised ToC as commonly reported mechanisms associated with the key outcomes listed above.

Key contexts

The initial workshop re-affirmed that co-production was a necessary programme ingredient/activity and a 'unique selling point' of PCF. Co-delivery/volunteering was not one of the most consistent or common contexts highlighted by the Realist Analysis. It was, however, reported by more than half of participants and the evidence from the development ladders and project level data suggest that co-production (or elements thereof) were present in all projects studied. Many participants had strengthened their skills/experience in co-production through involvement in PCF funded initiatives. A number of models are presented in Chapter 3 to show the various approaches used.

The most frequent/consistently reported contexts along with reported outcomes such as 'improved social capital', 'developing networks and connectedness' all align with the wider elements of co-production ( e.g. building on assets and doing this for and with people, and tailoring support) detailed in the community engagement standards/continuum of co-production shown earlier in this report.

The revised Theory of Change

The revised ToC has incorporated the above learning and is shown in Figure 6.3 (below).

The revised version has separated out mechanisms of change (illustrated in the initial ToC as learning, or behaviour changes) and highlights the most common mechanism operating within PCF as well as the key outcomes achieved.

The key activities have also been revised in line with research evidence.

Figure 6.3: Revised Theory of Change

Arrows have been added showing specific links between activities (contexts) and mechanisms that were evidenced by the research. The arrow for activities are grey/dashed to differentiate where they overlap. The arrows linking outcomes to influencing mechanism are coloured according to the outcomes to which they relate.

Lessons for Future Support Programmes

The range of positive outcomes reported indicates that engagement and co-production are associated with benefits from PCF implementation for Community Anchors, partner organisations and beneficiaries.

The Realist Analysis highlighted some of the strongest relationships between service provision (aspects of context), changes that were experienced by most PCF stakeholders and outcomes achieved suggesting that if certain contexts can fire certain mechanisms, then positive outcomes can be achieved.

The following tables highlight where the Realist Analysis has contributed to developing more specific CMO-Cs rather than links between mechanisms and outcomes or context and mechanisms independently as detailed above. CMO-Cs are where theories are described in more detail in terms of a context triggering a mechanism which in turn contributes to an outcome. Ideally the more specific these are the better in terms of programme improvement and transferring learning. Such specificity, however, often requires multiple related research studies.

Table 6.3 illustrates that there was evidence that outcomes related to improved knowledge, skills and understanding are influenced by the mechanisms shown. These mechanisms in turn seem to be influenced by several similar contexts.

Table 6.3: CMO-Cs for Improved Knowledge, Skills and Understanding

Key for Tables
Code	Context
1	A wide range of opportunities and encouragement to take them up
2	Needs assessment
3	Welcome and time to talk	The presence of Contexts	May help trigger Mechanisms	To achieve Outcomes
4	A diversity of services offered /taken up	1, 2, 3 & 4	Confidence	Improved knowledge and skills
5	An ability to socialise and network	1, 2, 5 & 6	Belonging/connectedness
6	The provision of feedback	1, 2, 3 & 5	Pride
7	Co-delivery	7	Ownership and commitment

Table 6.4 illustrates the mechanisms and contexts associated with improvements in engagement, and participation, influencing co-design/co-delivery and greater uptake of services. The mechanism of 'belonging' in turn may be influenced by the provision of personal needs assessment/awareness of the issues raised and being welcomed and provided with time to talk.

Table 6.4: CMO-Cs for Increased Engagement and Participation

The presence of Contexts	May help trigger Mechanisms	To achieve Outcomes
1, 2, 5, & 6	Belonging/connectedness	Increased engagement & participation, influence in co-design/delivery and greater uptake of services
Unable to differentiate	Reputation
Unable to differentiate	Reciprocity

The above emerging CMO-Cs and the other patterns in the data reported in this chapter are a guide to which mechanisms and their associated contexts may have played the strongest role in achieving PCF reported outcomes. This supports the refinement of general theories contained in the ToC towards more specific CMO-Cs. These refined theories could be further elaborated or tested in future research to further clarify which contexts and mechanisms are not simply associated with outcomes but which are necessary and sufficient to cause, or contribute with others, to anticipated PCF outcomes.