Review of the Climate Challenge Fund

6. Project Delivery: Challenges and changes

6.1 This section of the report describes the key challenges that case study groups identified in the running of their projects and the actions they would do differently if running their project again. This is described to highlight the potential areas for support that any future CCF or successor scheme may need to consider. As these factors are covered elsewhere, this section does not include comments from respondents on the carbon calculations, CCF support and the CCF itself.

Biggest challenges

Communicating and engaging with community and stakeholders

6.2 Some projects experienced problems applying the services developed in previous CCF projects to new communities. These problems included establishing a reputation and earning the trust of community members, engaging with new local authorities and their agencies, and understanding where service overlaps and/or synergies existed with other community groups and charitable organisations.

6.3 Moreover, existing communication issues persisted, such as difficulty in encouraging the uptake of energy efficiency measures and behaviour change through climate change messaging (explaining potential fuel cost savings was more effective). Indeed, one interviewee noted that "the carbon thing does not work with young people … it is no way to engage" [CS01 P1]. The selection of appropriate modes of communication (e.g. leaflets, drop-in sessions and social media) was important.

6.4 It was noted that there was noticeably less interest among community members in project interventions designed to reduce domestic heating fuel bills when the weather was mild. It was also noted that fixed term times and busy teaching schedules presented challenges when attempting to involve teachers and schoolchildren (and, indirectly, their parents) in project activities. Perceptions of what is 'cool' (particularly in relation to cycling) were a barrier to engaging with children of secondary school age.

Staffing and volunteering issues

6.5 Many projects reported issues with staff and volunteer retention - "it's generally known that working with volunteers is difficult, providing training, and building a relationship" [CS19 P1]. Other projects found staff recruitment and management challenging and high staff turnover was identified as an issue in at least one project. Particular staff and volunteer issues included insufficient time for proper recruitment, role definition difficulties, skills deficits and training requirements, and maintaining enthusiasm where volunteers were drawn from outwith the immediate community. One interviewee noted: "It's difficult to keep people [volunteers] on board; to get them to stay involved for a three year period." [CS03 P1]. One project saw employing staff as a solution to the challenges of managing large numbers of volunteers that could only offer small amounts of time.

Engaging with professional services

6.6 Groups that sought to refurbish community buildings for energy efficiency and improved utility often struggled to understand the demands and requirements of the construction procurement process. This was made more challenging where the building in question had listed historic status: "...getting planning permission ... with us being a listed building, what was required by the Council to get the permission to go ahead with changing windows and things. That was a side of things that we could have had big delays" [CS09 P1].

6.7 However, in the case of one project, this barrier was alleviated through utilising the skills and professional experience of a community member employed as an architect.

Data collection

6.8 Energy efficiency data collection and interpretation emerged as a particular problem for some groups. Volunteers were reported to be reluctant or unable (due to accessibility constraints) to collect data from the energy meters or energy monitors of targeted community members. For example, on one project the operation of energy monitoring hardware by homeowners was found to be difficult, leading to user apathy: "[Energy] monitors, that was difficult. Well, we haven't really been able to measure that ... we'd set ourselves a target ... 20 households trying to reduce 10% of their energy. We haven't been able to show that we've been able to do that" [CS09 P1].

6.9 Furthermore, relating changes in energy consumption resulting from the installation of energy efficiency measures and/or behaviour change to actual CO2 reductions based on energy meter readings (which could not be disaggregated) was challenging or impossible. In addition to this, those working on transport projects also found behaviour change monitoring challenging as it was difficult for projects to gather travel behaviour data (via travel diaries) after providing advice and services.

Project administration/ management

6.10 Many project groups had limited experience of project management (which had to be developed during project delivery and was often described as a steep learning curve) and the practical demands of executing a project (e.g. home office, IT setup and maintenance, developing promotional materials). A belief was expressed by a stakeholder that "running an energy project is quite challenging ... a pain" [SH14] due to the challenges of training, staff management and the need to consider personal safety of advisors.

Doing things differently?

6.11 Case study respondents were also asked to indicate if there was anything that they would do differently if they were to run the project again.

6.12 A small number of respondents (each from different project teams) remarked that they would not have done anything differently on their projects and were very satisfied with how their projects had progressed. One stated "I wouldn't change anything" [CS11]. Another stated "I can't see any weaknesses …[we] didn't have any problems at all" [CS42]. However there were many comments about changes that would be made. These are described in the following sections.

Planning

6.13 Many respondents stated that they would revisit how they had planned their projects if the occasion arose to do them again. Gaining a better understanding of interest levels in the project among members of the target audience, establishing stronger relationships with potential project partners (including businesses and local authorities) earlier in the project process, and allowing more time to develop the project application and implement the project (generally, one year was viewed as an insufficient duration to effect change) all emerged as significant themes. Also, greater attention would be given to the definition of project scope: "If I was to go back and change the application, I would probably say that we needed more of a target audience and I would have probably said schools and businesses would have been the way to go" [CS07 P2].

Implementation

6.14 In terms of project implementation, most groups reported that they would have done at least one project task differently, generally focused on making them more accessible and engaging to encourage low-carbon behaviours. Two groups, each undertaking an energy advice project, concluded that retaining multiple volunteers who were only committed to a few hours work per week was not as effective as employing a smaller number of paid staff: "We would not deliver [the project] with volunteers ... but if we had used the funding we received ... to have staff delivering the service we would have been more effective I think" [CS06 P1].

6.15 Indeed, other groups also mentioned that they would seek to employ more staff in a variety of functions (e.g. project management, book-keeping).

Monitoring and evaluation

6.16 Consideration of project scope is closely linked to the monitoring and evaluation of project outputs and outcomes. Improvements to data gathering processes were raised including standardising paperwork requirements, conducting audits at more appropriate intervals (sensitive to community restrictions), and involving community members to a greater degree in data gathering process definition and execution. For example one group suggested that: "... the school catering staff should be kept more in the loop regards audit arrangements and feedback which currently goes through the teaching staff" [CS03 R].

6.17 Also, it was noted that project outputs and outcomes should be more clearly defined or more focused to make it easier to measure from the beginning. In addition, some respondents indicated that they would be more specific and realistic (particularly with regard to setting CO2 reduction targets) while others stated in hindsight that they had not been ambitious enough.

Relationship with KSB and support

6.18 Many projects felt that they should have been more proactive and sought more support and clarification from the CCF team at KSB. This related in particular to CO2 reduction targets and methodologies, clarification over implications if missed, taking greater advantage of training and to have pushed for more support in general.

Summary: Project Delivery: Challenges and changes

6.19 Although many groups undertook their projects successfully, there were key themes arising in terms of the possible support that any successor fund may need to consider to ensure that projects have the best chance of achieving success. In particular this relates to enhanced training and support on:

• Community engagement, audience targeting and understanding
• Volunteer (and staff) management
• Project and group management and administration
• Data collection, monitoring and evaluation
• Best practice in project delivery (by type)

6.20 There is clear scope to share more 'best practice' and guidance in relation to all these elements in any future programme. The CCF in its current form is very much focused on enabling communities to develop their own solutions, however it would appear that some more guidance and support in relation to these areas may be beneficial, including for example 'toolboxes' in relation to, for example, project types, resources and monitoring plans that can be adapted by groups.

6.21 In addition there is also scope for a future scheme to foster more effective partnerships and collaborations to meet wider Scottish Government aims. This would help to ensure a more sustainable legacy as well as identify potential follow-on, community-led projects in other related areas (this is discussed in further depth in chapter 10).