Community-led regeneration approach: review

2. Study Methodology

Introduction

This Chapter provides an overview of the study approach and the methods that have been applied by this research.

As some of the methods and terms used are highly technical, a glossary is provided at the end of the report. In addition, a more detailed explanation of the study methods is presented in Appendix A .

Study Approach

The study integrated a number of elements from the following research approaches:

• Realist Evaluation;
• Theory of Change ( ToC) Review;
• Qualitative Comparative Assessment (at programme level - across all projects researched); and
• Contribution Analysis.

As indicated above, the design of the study incorporated a Realist Evaluation approach, which is a theory-based method with a focus on how and why participants of a particular initiative benefit.

While more conventional evaluation or impact assessment studies are often content with establishing that certain benefits have been achieved, the Realist Evaluation approach goes further by exploring the reasons why and how the benefits have been achieved. Through this deeper approach, it is hoped that more learning can be extracted to inform future initiatives.

The study centered on the ToC of the PCF Programme as devised by the Scottish Government. The ToC represented the underlying assumptions and aspirations of why and how the Fund would be helpful in addressing a set of identified needs.

The existence of a ToC represents good practice by enhancing the clarity of purpose (why funding is made available), specifying the approach for Programme intervention (how PCF is supposed to be used), and making the objectives and anticipated outcomes of funding transparent (what is expected to be achieved).

Reviewing the ToC and its assumptions for implementation and anticipated achievements was a key aim of the study to test if the PCF Programme is working as intended.

Study Methods

Based on the research aims, the study sought to understand how and why certain approaches and models operate within PCF and how outcomes have been achieved.

The study applied a mix of desk-based and primary research methods. All study tools were designed in a bespoke manner in line with the existing PCF ToC. The study tools focused on exploring what changes have been reported by participants, how these changes have developed in practice and what mechanisms contributed to these outcomes in varying contexts.

Primary Research Participants

As the promotion of co-production is one of the key aims of the PCF, our research needed to engage with a range of stakeholders involved in the implementation of the programme to cover all angles of co-production.

Therefore, our fieldwork engaged with the following participant groups:

• Community Anchors - these were the community-based organisations who were responsible for the delivery of the PCF funded initiative The study engaged with staff of the Community Anchors directly involved in delivering PCF funded services;
• Partner Organisations - those organisations that supported the Community Anchors through joint working, signposting or referrals of service users/people in need; offering additional services to project beneficiaries, or being directly involved in the delivery of the PCF project;
• Volunteers -community members who volunteered because of a desire to help the Community Anchor and a motivation 'to do good' in the community, or local people with the same motivations, but previously (or concurrently) themselves service users. Here, volunteering is often part of gaining employability and other transferable skills; and
• Beneficiaries - this group includes those community members in need of support, receiving and/or participating in the services funded by PCF.

How projects were selected

The sample frame for the research was based on a Scottish Government randomly selected cross-section of 25 projects which received PCF funding in 2015/16. Of the 25 pre-selected projects, the study was resourced to select 12 of these projects for the research.

We have applied the following rationale to capture and compare what were seen as important programme characteristics within the ToC:

• In order to compare results between the projects and between different participant groups, the study method needed to focus on thematically similar projects. The following three themes were chosen in this context:
  • Training and Upskilling, and Employability Training (five projects);
  • Advice/Support Service (Benefits) & (Combination) (four projects); and
  • Volunteering and Peer Mentoring (three projects).
• Within each theme, we sought a good spread across the sectors of applicants/Community Anchors to explore if a successful implementation of PCF depends on a particular sector:
  • Voluntary Sector (four projects);
  • Community Development Trust (three projects);
  • Registered Social Landlord/Housing Associations ( HA) (three projects); and
  • Social Enterprise (two projects).
• The sample of 12 projects also allowed for a range of target groups to be presented:
  • whole community (three projects)
  • young people (a frequent target audience in PCF) (three projects);
  • rural community (two projects); and
  • ethnic minorities (one), women (one), those at risk of poverty (one), and other vulnerable groups (one).
• There was a mixture of community organisations with dates of establishment ranging from 1968 to 2009, with more than half (seven) established since 2000.
• The average project size reflected that of the PCF total population. The 12 selected projects had an average (mean) PCF award of £85,470, ranging from £20,000 to £185,000.
• Geographically, the project sample spread across Scotland, including the Western Isles, the Highlands, Argyll and Bute, and across Lowland Scotland, Figure 2.1 (over).

Figure 2.1: Map of Project Locations

Detailed Research Tools

The research was carried out primarily by conducting workshops ^[2] with Community Anchor staff, volunteers, project beneficiaries and partner organisations, supplemented by telephone interviews where appropriate.

The workshops consisted of a number of interactive activities, partly engaging the whole group including sharing of experiences, reflection and self-reporting of achievements and partly involving more individual work (with assistance where required) to assess:

• Co-production levels (Development Ladders);
• Outcomes from the support (group work with a 'Ketso' workshop toolkit); and
• Detailed learning journeys (Mini Interviews).

Full details of the research tools used can be found in Appendix B .

Study Points to Note

• Although only a sample of 12 projects was selected from the 197 projects funded in 2015/16, the views of 136 individuals were captured by the study. The research findings present examples of how the PCF approach has operated in these 12 cases rather than the Fund as a whole. Therefore, the findings and conclusions should be understood in relation to the 12 projects alone.
• Delivery of the intensive fieldwork, write up and analysis was managed within a tight schedule, together with the set-up of workshops with stakeholders and beneficiaries. Thanks to the commitment of the participating projects, the study was completed timeously, although more time would have allowed for wider, and greater participation.

Note should be taken with regard to the following aspects of the sample and the fieldwork findings:

• Although drawn from a random sample of 25 PCF funded organisations, the 12 projects involved in the study were selected in line with the sample frame, but essentially volunteered their participation and identified their own partner organisations and beneficiaries to attend the workshop ( self-selection at project as well as beneficiary level). Although open and honest feedback from all participants involved in the study was encouraged, this could indicate why the study findings are overwhelmingly positive.
• The study methods used in the primary research were all based on self-reporting techniques ( i.e. individuals were asked how they felt, how they experienced their learning journey, what impacts this had on their lives, etc.). Baselines or validated pre and post measurement scales were not available or feasible to use given the timescales, engagement opportunities with the projects and available research funding.

Appendix A outlines further limitations specifically relating to Realist Evaluation (as an approach).