Regeneration Capital Grant Fund: evaluation

7. Conclusions

7.1 Introduction

This chapter sets out key findings from the research, and key learning points. The key findings are set out under each of the four core research questions for this study.

The key learning points develop ideas for helping to ensure that future funding rounds deliver as much value as possible, achieve desired outcomes and avoid undesired outcomes.

7.2 Key findings

7.2.1 What have projects achieved in terms of community involvement?

Across the 14 focus projects, community involvement in assessing need was strong. In most projects, community was a driving force in identifying need. In almost all projects communities had the opportunity to explore ideas in more depth, through community-led design events, options appraisals and feasibility studies – sometimes led by community groups – and had the chance to get involved in planning or designing facilities.

RCGF projects used a wide range of innovative ways of involving communities, and most felt involved, listened to and respected. In some projects, communities had influenced the final design, the level of community use, the pricing structure and the management arrangements. In most of the projects, communities had been involved in influencing service provision in some way. In some projects, community organisations or social enterprises were managing the facility, while in others, communities had budget responsibility for deciding on community priorities. However, some reported that although they had enjoyed the process, they did not feel their ideas were taken into account.

Community members and partners recognised that community involvement was challenging. It worked best when the idea came from the community and was led by community organisations showed strongest ongoing support. Within the 14 focus projects, the strongest community driven projects were in small rural areas with strong community structures.

When projects were initiated by public sector organisations, these were more successful when partners proactively reached out to communities, and there was the time and resource available to invest in capacity building. Community involvement worked particularly well where there were passionate community activists and strong community organisations.

7.2.2 What have projects achieved in terms of social outcomes?

Most RCGF projects involved in this research had a significant impact on:

• community identity and aspirations – most projects had a significant impact on how communities felt about their area. The projects helped to improve the image and perception communities had about their area, built a positive identity and encouraged people to visit.

• community spaces – most projects provided local people with new or improved places to meet and connect, in some instances free of charge. This provided people with opportunities to socialise, develop skills, use services and get out of the house. Facilities were largely well used.

• safety – communities felt safe using new facilities. Most facilities were described as relaxing, open and inclusive.

• skills development – most projects provided opportunities for community members to develop their skills – either through focused employability support; wider activities to build skills for life, learning and work; or volunteering opportunities.

Some RCGF projects have had an impact on:

• community networks – the projects helped some community organisations to become stronger and more sustainable - with organisations reporting increased service use, new opportunities to generate revenue, increasing community membership, and success accessing further funding sources for future activity.

• public services – in some projects, facilities provided opportunities for public services to co-locate, offering services like health, housing, education, social work and employability from a single building. Community members and public sector organisations found this useful.
• supported mental health and wellbeing – some projects have supported mental health by giving people a place to go, an opportunity to be part of the community, reducing isolation, providing volunteering opportunities and enabling access to services.

• built vibrant places – some RCGF projects helped to improve the appearance of towns and high streets, provided a central focal point, increased visitors and encouraged visitors to the area. Some reported greater use of local cafes, shops and public transport. Some projects have also supported the local economy by providing business space and supporting tourism.

One of the focus projects had an impact on physical health, with new sports facilities providing more people with more opportunities to be active.

Projects also helped to deliver a range of employment and employability outcomes for people in the community. Local people were employed during the construction phase in some projects. Volunteering and some employment opportunities also came from ongoing service delivery. Both during the construction phase and in ongoing delivery, there were opportunities for skills development. Few negative or undesired outcomes were reported.

7.2.3 What difference has RCGF made to projects?

RCGF funding was a key success factor in delivering projects. In most projects, it was felt that the project may not have happened without RCGF. RCGF funding was important in:

• providing large amounts of capital funding not available from many other funders
• unlocking other funds – for the project, or for wider works such as streetscaping, town centre regeneration, public realm improvement and artwork
• reducing the risk for other funders
• strengthening and clarifying local authority commitments

7.2.4 What are the key factors affecting successful delivery of projects?

The main success factors were:

• working in partnership
• a clear vision
• clear decision making processes
• community involvement
• achieving RCGF funding – and through this, other funding sources

The main challenges were:

• the nature of the sites and buildings
• securing and managing a package of funding
• timescales – with some feeling RCGF timescales were tight, and that more flexibility was required to effectively engage communities, enable decisions to be made in partnership and fit with wider regeneration activity
• sustainability – balancing community use with business focused decision making

7.2.5 What are the key factors affecting the quality of project monitoring and reporting?

Overall, the RCGF processes were broadly felt to be sensible, reasonable and proportionate. Some felt that the monitoring requirements were fine, but a few felt they were over complicated and not focused enough on impact and outcomes. Some said they found the discussions about progress with the regeneration team more useful than filling out the monitoring forms.

A few projects had systems in place for measuring social outcomes, and a few were thinking about evaluation arrangements for the future.

The main challenges around evaluation were:

• different requirements from different funders
• challenges demonstrating social and community outcomes
• the timescales required to achieve social outcomes
• attributing social outcomes to the project
• the time, money and staff resource required to monitor and evaluate

7.3 Key learning points

7.3.1 Community involvement

Community involvement in identifying need for RCGF funded projects was strong. This was clearly demonstrated at application stage. The requirement for projects to involve communities from an early stage is helping to enhance the focus on community involvement. However, there is often less focus on demonstrating community involvement as RCGF projects progress.

This research identified potential for more to be done to help some RCGF supported projects to be led or driven strongly by communities, beyond the stage of assessing need. Scottish Government could clarify whether ongoing community involvement is a priority – in planning, design and service delivery. If it is, RCGF processes could be adapted accordingly such as:

• asking projects to highlight whether community involvement and empowerment is a key intended outcome of their work
• asking for more evidence of planned ongoing community involvement if successful, at application stage – and explicitly incorporating this in the guidance and assessment process
• providing funding or linking to other resources which can support community capacity building
• providing more time for funds to be spent, to allow effective and meaningful community engagement in project design and delivery as the project develops after a successful funding application

7.3.2 Describing intended outcomes

The logic of funds like RCGF is that physical regeneration helps to bring about wider, longer term economic and social outcomes.

This is well articulated in the Scottish Government's exploration of how town centre regeneration works, which includes a draft logic model for regeneration^[19], provided as Appendix 4. This logic model shows that by achieving physical outcomes, the logic is that economic and social outcomes follow – in the longer term. It also shows that some social outcomes are more likely to be achieved in the shorter term than others. For example, outcomes like providing community access to facilities and improving the image and perception of an area can be reasonably short term outcomes, while outcomes around economic, physical and mental health for residents and enhanced social capital may take longer.

Within the current RCGF application process, the weighting of outcomes between physical, economic and social may reduce the priority that funded projects and applicants give to these longer-term outcomes. It may be useful to be clearer that social (and economic) outcomes are the intended result of physical regeneration – the reason for the physical works being done.

Scottish Government may wish to consider using a logic model to describe the logic of key RCGF activities and outputs, the intended outcomes, and the anticipated time frames for these. A working logic model (for further development) is provided at Fig. 7.1. This model draws on Scotland's National Performance Framework, the Place Principle and the supporting outcomes within the regeneration strategy. RCGF applicants would then be asked to demonstrate how their project fits with the logic model for the Fund, and identify a small number of outcomes (say two to four) that their project is focusing on.

Figure 7.1: Draft working logic model

7.3.3 Funding decisions and processes

Most involved in this research were happy with the RCGF processes and felt that RCGF was an extremely useful source of funding. However, the research did identify some areas that may benefit from further consideration, including:

• timescales for spending funding – the timescales for spending funding were felt by some to be restrictive. Scottish Government should consider introducing more flexibility on when funding is spent by the project, to ensure adequate time for community involvement, working in partnership and effective project planning and delivery.

• connections between funders – the RCGF often supports projects alongside other national funding sources. There is potential to consider developing stronger links with other funders as part of the decision making process, to enable joined up decisions about which projects are of strategic importance.

• exceptional projects – the RCGF is very flexible, and enabled most projects to tackle large scale, complex regeneration. However, some sites and buildings are particularly complex and expensive, and require an experienced team of funders and deliverers from different organisations to work jointly over a number of years. It is worth considering whether in these exceptional projects there should be a strategic approach to bringing funders together around the project over a longer time period.

7.3.4 Monitoring and reporting

This evaluation highlighted that the current monitoring system provides little information about social outcomes. Scottish Government analysis of the monitoring information on physical and economic outcomes also raised questions such as:

• how useful is the submitted information?
• who uses it, why and when?
• how easy is it to collect?
• how reliable are the figures?

From a review of submitted monitoring, final and completion forms (which only includes very few completion forms) a number of issues with the current monitoring and reporting system can be identified. These are explored below.

Table 7.1: Monitoring and reporting issues identified

Issue	Solution	Implications
Projects are selecting most outcomes – to try to fit with as many priorities as possible in application stage – which means reporting is across a wide range of areas in little detail	Develop a clear logic model for RCGF Ask projects to fit with a maximum of 3 or 4 outcomes and commit to reporting on these in depth Provide guidance on whether any outcomes are key priorities for Scottish Government Adapt assessment process accordingly	Funded projects may feel this is more directive than the current approach. It would need to be made clear that local organisations can prioritise whichever outcomes are most relevant to their local area – or clear guidance would need to be provided on why certain outcomes are a key priority for the fund.
The same form is submitted for quarterly monitoring, final reporting and completion reporting – despite the focus on what the Scottish Government wants from each stage shifting	Develop a simple quarterly reporting form focused on activities, finance, physical outputs and community engagement. Develop a separate completion form focused on outcomes (including empowerment). Encourage funded projects to take responsibility for self-evaluation over the longer term and to communicate findings with Scottish Government.	Evaluating outcomes is challenging. Support and resources may be needed to enable this shift. Some funded projects may not have the resources and skills to undertake self-evaluation.
There is little incentive to submit a completion form a year after the project is completed, when all financial claims have been made.	Introduce incentives – potentially including publicity / profile for projects to raise awareness of their successes; Be flexible – such as linking to other funder's requirements and being flexible about the format of completion evidence to fit with what is being submitted to other funders; Continue to be proactive in requesting forms – emphasise the importance of this evidence from the outset and send reminders.	It still may be challenging to encourage funded projects to submit a completion form a year after the project is completed.
There is limited evidence of the submitted information being used	Be clear why projects are being asked for this information, how it will be used by Scottish Government and partners nationally, and how it could be used by partners locally	A clear framework for how information requested will actually be used and shared would need to be developed.
There is little guidance on how projects might measure and report on social outcomes – which are complex	Develop written guidance and associated support with evaluation; and/ or Include a provision for evaluation / evaluation support within each award.	Resources and skills needed to develop guidance and provide support.

Over the time that RCGF has operated, there has been a strong shift in the approaches of many funders, to focus much more strongly on outcomes than processes. As RCGF forms have remained largely the same, it is understandable that these issues have begun to emerge.

There may also be opportunities to align or rationalise the range of government and government agency procedures around application, approval, monitoring and evaluation for this type of project.

There is potential to make a few simple adaptations to the monitoring system to ensure that it adequately captures outcomes as well as processes. However, this would also require clear guidance and some capacity building work to ensure that projects have the skills and support required to report effectively on outcomes. Evaluation of social outcomes is complex, and this may require capacity building, resources and support for projects and decision makers.