Regeneration Capital Grant Fund: evaluation

6. Project monitoring and evaluation

6.1 Introduction

This chapter explores feedback on RCGF processes in general, and then focuses specifically on monitoring and evaluation. It is largely based on feedback from project leads and partners with experience of RCGF processes.

6.2 RCGF processes

Overall, project leads and partners felt that the RCGF processes were sensible, reasonable and proportionate. Most felt that the two stage process worked well, and that the application process was not overly burdensome.

A few would welcome more feedback about why some applications are successful and others are not. A few felt that it would be useful for the RCGF to have more clarity and focus on the social outcomes, with the main priority currently appearing to be the building itself rather than the people who would benefit from it.

A few project leads indicated that they felt the timescales for hearing the outcome of applications was getting longer, and that this created a lot of uncertainty and could be hard to manage.

Most felt that it was a very useful grant, offering significant amounts of money to deliver large projects – which few other funds were able to do. Project leads and partners were pleased that RCGF could fund a significant proportion of the costs for a project, and felt it was often a lever to unlock money from other funders.

"Overall, it was a very positive experience. It is a good source of funding."

(Project lead)

However, a few felt that major funders (particularly Scottish Government and other government agencies) could do a lot more to synchronise their application and reporting procedures, to reduce duplication. A few funders interviewed as part of this research (as project partners) indicated that they were aware that some felt that application and monitoring processes of funders should be better aligned.

One project lead felt it could be hard for local authorities to ring fence funding for a project which may be dependent on a grant application which may or may not be successful.

One project lead had concerns about the role of the national panel in decision making and would like to see a mechanism through which local authorities could strategically prioritise their RCGF applications. It was felt that this would reduce the planning and consultation time spent on projects which were not successful and allow the national funding to better connect to local strategic priorities.

A few project leads had concerns about the way the fund was allocated between local authority areas. While one felt there was a focus on central belt projects, another felt the awards were not made proportionately to the level of deprivation in the local authority^[17].

6.3 The RCGF monitoring process

RCGF funded projects are required to submit regular monitoring forms, in line with an agreed schedule – at least every four months. At the end of the work, funded projects are also required to submit a project completion form.

The monitoring forms cover:

• financial information – grant claimed, project costs, wider funding
• progress – key activities, delays and changes
• outputs and outcomes – progress towards outcomes (a brief commentary, and a table for numerical outputs and outcomes)
• project publicity

There is a final monitoring report and claim that should be submitted one month after the project is completed. All grant should be drawn down at this stage.

The project completion report should be submitted 12 months after the project end date (when the project received its certificate of building works).

Some project leads and partners thought that the monitoring arrangements were fine, proportionate and easy to understand. Most were well used to completing similar forms. However, some felt that the financial reporting requirements were over complicated or that the outputs focused information was complex.

Some felt that the monitoring forms were not focused enough on impact and outcomes. A few felt the monitoring forms were not very meaningful, particularly on social outcomes. A few project leads indicated that it was difficult to measure social outcomes using hard data, and that there should be a stronger focus on collecting people's stories and encouraging a qualitative approach.

"It can be hard to demonstrate outcomes, like community benefits."

(Partner)

Funds such as The Big Lottery^[18] and Aspiring Communities Fund were highlighted as having more outcomes focused monitoring systems.

6.4 Project approaches to monitoring and evaluation

As projects were developing, project leads and partners largely focused on completing the forms required by their funders. There were a range of strong project management approaches in place to track delivery. For example, some used PRINCE 2 approaches to project management. A few projects were able to tap into specialist expertise within the council to help with financial monitoring and completing the monitoring forms.

As projects were completed, projects began thinking about measuring usage, footfall and outcomes.

Example: Measuring usage In one project, the local authority used a project management approach which it developed from PRINCE 2. It provided inputs for the council's internal financial and accountability process. It also fed information into the different monitoring arrangements for each funder. The project itself uses a simple system for measuring the use of the facility – by adults and young people and men and women.

Some projects had systems in place for measuring social outcomes. A few used anecdotal evidence from speaking with service users. A few had started to ask service users to write down their stories, so that social outcomes could be recorded. A few were just beginning to think about evaluation. For example, in one project there is a plan to pull together a multi-disciplinary team to evaluate the project when it is complete.

In a few projects, work had begun evaluating the activities and services delivered within the new facility. For example:

• in one project, 900 people who had attended a community event at the facility were interviewed. This meant they got detailed feedback from a large number of local people, which was very useful for planning future activities.
• in one project the local authority paid for digital optimisation software to measure the impact of the town centre changes on people's quality of life.
• in one project, the local authority is about to introduce a monitoring and evaluation system asking the project to report on its impact, outcomes and finances. It asks standard key questions, such as footfall and number of classes. It is linked to locality plans, and the community learning and development plan. This system has been tried out in other facilities and has worked well.

Example: Measuring health and wellbeing impacts One project which was completed in 2015 undertook an evaluation once the building was complete and the local health improvement team is monitoring the health and wellbeing impacts of the project. This is done through NHS monitoring processes. They monitor participation levels and gather feedback pre and post groups and projects. They produce an annual report that is shared with the facility manager and produce evaluation reports at the end of every project. They use a Plan, Do, Study, Act (PDSA) methodology.

6.5 Factors affecting monitoring and evaluation

Project leads and partners were asked what helped and hindered in terms of monitoring and evaluation.

Project leads and partners felt that it was important to:

• involve people experienced in monitoring and evaluation
• be clear about your budget – and include contingencies
• set clear milestones at the outset and stick to these
• hear people's stories to gather evidence of social outcomes

"You need to have as much visibility over your programme and budget as possible. There is no wiggle room. There is an expectation to deliver the milestones on time."

(Partner)

Some challenges with monitoring and evaluation were also identified:

• different partners can be responsible for different funds
• different funds require different reporting arrangements, at different times – this can result in duplication of effort
• it requires all partners to be open and share information
• some partners may have limited control of the budget, for example if they are administering it for third parties (such as social enterprises and community groups)
• it can be hard to demonstrate social and community outcomes.
• social outcomes can take longer to achieve than physical and economic outcomes
• social outcomes can be affected by much wider factors in society
• evaluation takes time, money and staff resource – all of which are limited
• there is no development fee to cover the cost of support, monitoring and evaluation
• third sector organisations and community groups can need a lot of support around monitoring and evaluation

Project leads and partners felt that it was important to recognise that monitoring and evaluation does require specialist skills. Some project leads received support from funding officers in local authorities to fulfil RCGF monitoring and evaluation requirements. A few said it was sometimes taken for granted that people have the skills to monitor and evaluate their project, but often they need support.

"A lot of support is needed to do monitoring and evaluation properly.

This is often underestimated."

(Project lead)