Review of the Climate Challenge Fund

8. Delivering Carbon Savings and Changing Behaviours

8.1 Central to the aims of the CCF were achieving measurable carbon savings and fostering behaviour change amongst the target community. These aims and challenges are discussed in this section. Please note that the in-depth case studies, described in Appendix C go into more detail on these issues at the individual project level. This section draws together the key lessons and insights from across the projects.

Carbon Calculations: Context

8.2 The CCF is focused on delivering behaviour change and, as a result, emissions reductions at a local level as part of the wider Scottish Government approach to tackling climate change. Hence the CCF asks projects to collect data on carbon emissions baselines and reductions over time for two key purposes: a) to help groups think through the logic of how they are addressing climate change; and b) to focus projects on the link between their activities and outcomes.

Setting targets

8.3 The CCF provided guidance to groups applying for, and awarded, funding to help them estimate their projects' CO2 emissions baselines, set targets to reduce emissions and measure performance. Guidance used by case study projects included the CCF's Low Carbon Route Maps^[10] and worked examples provided in CCF/ CASP training workshops, along with information from the Department of Energy and Climate Change (DECC), EST and HES^[11]. Some projects used expertise built from previous applications to the CCF or gained help from other CCF-funded groups. Although helpful to many, some mentioned that they found the route maps confusing, "[we] trawled through them to get figures you could use" [CS01], or not broad enough (i.e. insufficient examples were present that could be applied to particular activities).

8.4 Some groups mentioned that the KSB DOs were heavily involved in shaping their applications towards activities that were more focused on behaviour change and reducing CO2 emissions (and would therefore better address the CCF criteria). This involved changing, adding or altering activities and topics to their projects. For many this was felt to be a positive process, prompting groups to "explore the softer side" of carbon reduction [CS24], or to "encourage the inclusion of other factors such as a workshop [with the community]" [CS23 Int2] in applications. However, for a small number of others this influence was perceived to have had a negative effect, with applicants feeling they were being forced to be too ambitious with their applications: "We kept adding things on. And that's why we've got these six outcomes. Which I feel is a bit much" [CS23]. It was also suggested by one group that DOs advised some projects to grow food in order to amass comparatively easy-to-achieve CO2 savings.

8.5 Conversely, the difficulties of setting targets and potential challenges of measuring changes had the effect of restricting proposed activities for some groups: "[we] made a conscious decision in application to stick to gardening [not transport] because we would not know how to work out CO2 reduction estimates" [CS12]. Other projects experienced similar difficulties, with one respondent struggling in an (ultimately rejected) application to account for the CO2 savings that could result from behaviour changes arising from a series of radio broadcasts.

8.6 The challenge of gathering baseline data was acknowledged by some groups, with one project reporting that not enough baseline data was gathered in the first instance, leading to accounting problems later in the execution of the project. Some project teams noted that CO2 reductions associated with food production were difficult to baseline, with one group indicating that they found it difficult to find carbon figures for food (to set the baseline) and another suggesting that savings achieved were underestimated (as calculations could not fully take into account all carbon associated with food-growing, e.g. sufficient food miles, fertilizer, transport emissions). Similarly, carbon savings associated with travel projects were also perceived as being difficult to quantify (both to baseline and to measure).

8.7 In one case, estimated CO2 reductions went beyond simply accounting for 'operational' carbon savings (i.e. those occurring directly from project activities), with 'embodied' CO2 (including production and shipping carbon emissions associated with products and materials) forming part of the group's CO2 reduction targets (as they felt this was necessary). This (and the concerns in the previous paragraph) highlights the difficulties and potential ambiguities that groups may have had in defining the scope of their targets and in how detailed the calculations needed to be. DOs were reported as encouraging community groups to use whichever data sources they deemed to be appropriate to their projects (with assumptions explained). The Low Carbon Route Maps highlight that there is "no one best way" to complete applications as each project and group are different, and asks groups to "think about any extra emission sources" when they plan their work. Although this flexible guidance is helpful in one sense, it led some respondents to become concerned and confused about the process.

8.8 Reviewing and (potentially) incorporating multiple sources of data into baselines was noted to be a difficult and time-consuming exercise. Others commented that this involved considerable data estimations and guesswork (highlighting the need for baselines), thus making targets inaccurate and difficult to meet, as well as, according to one respondent, "an impossible task" [CS03 P2]. Another noted that "most would find it difficult to engage with this level of complexity" [CS18].

8.9 It should be acknowledged that one respondent with previous CCF project carbon accounting experience perceived that the carbon reduction calculation methodology and guidance applicable to the CCF3 funding period was an improvement over that provided previously.

8.10 In a few cases there was evidence that applicants felt pressure to set higher savings targets in their applications than they necessarily felt comfortable with achieving. This was related both to a need to submit a strong application (particularly as the application process became more competitive), but also to a belief that groups needed to commit to achieving a certain level of carbon saving based on the scale of funding requested.

Recording savings

8.11 For the purpose of grant applications, when projects were at the planning stage, the CCF's Low Carbon Route Maps indicated that potential carbon savings could be estimated using secondary data (e.g. typical amounts of energy used by a household for space heating, anticipated numbers of households making changes). However the Route Maps also indicated that "all these pieces of information should be checked or updated with better information once the project is implemented"^[12] (for example through the collection of primary data via a survey of participants).

8.12 In line with the concerns expressed in relation to setting targets, it emerged that many community groups had substantial difficulty determining CO2 reductions associated with project activities, although this challenge tended to diminish somewhat with experience of performing the calculations over the course of their projects. Some groups identified innovative practice in relation to calculating savings, with one project reporting that secondary school pupils in a third year mathematics class were performing the CO2 saving calculations under the direction and supervision of teaching staff. Some projects also created bespoke spreadsheets to facilitate carbon accounting calculations, with one project successfully sharing its spreadsheet with other community groups.

8.13 Data gathering to verify carbon reduction targets was described as difficult for some, especially in relation to domestic energy consumption where there was homeowner suspicion or apathy (even where encouraged by face-to-face contact or free energy efficiency gifts, such as remote-controlled wireless sockets). Also, access constraints and energy monitoring technology presented barriers. It was noted that obtaining energy consumption data from fuel bills reduced the risk of double-counting from changed behaviours and installed energy efficiency measures.

8.14 One group realised that they were using incorrect carbon calculations when sending metal to landfill appeared to save more CO2 than recycling it. Working with their DO, they were able to work out where they had gone wrong and obtain good carbon savings estimates for their calculations. Echoing these types of challenges, others commented that more information on, and greater standardisation of, carbon accounting methods and data by CCF would have been helpful.

8.15 Some community groups were markedly anxious about meeting CO2 reduction targets. Advice from DOs (e.g. guidance about what emissions to include or exclude) seemed to allay such fears. However relationships with DOs varied and many groups experienced changes to their DO throughout their projects' lifetime, meaning that some groups were not able to gain certainty on this process as quickly as they would have liked.

8.16 Some case study projects found it difficult to meet their CO2 reduction targets. Reasons provided for this included errors in interpretation of data to create baselines and the overestimation of achievable targets (with groups reflecting that greater realism was required at the application stage).

8.17 Many case study projects felt that the advice or approach on calculating carbon savings had changed during the course of their CCF3 projects (although KSB have indicated to the research team that there was in fact no change in advice over the course of the CCF3 period) For some groups this resulted in significant efforts being expended on recalculations and was a point of considerable annoyance for the project teams affected. For example one group highlighted the requirement, towards the end of its project, to change their data collection methods and collect additional primary data. These findings suggest that more clarity or emphasis in highlighting the process that groups were required to undertake once their project was 'live' may have been beneficial (i.e. the extent to which groups had to generate primary data and what type of information had to be replaced with primary data; see also the discussion in 8.11 above).

Overall Impressions

8.18 The reliability and utility of CO2 reduction estimates generally was questioned by some groups, in particular by those who were aware that the carbon saving figures identified were not used in the calculations for the Scottish Government's progress against targets. Indeed many were not aware of this, with some groups assuming that the figures were used directly in the emission calculations^[13]. Others were unclear about how the detailed carbon savings identified and reported to the CCF were used.

8.19 Moreover, it was mentioned by some that projects risked becoming meaningless if their primary focus was on realising CO2 reductions rather than achieving behaviour change and community benefit, especially on short duration projects. Interestingly in one case, a group was explicitly told by its DO that less focus should be given to CO2 reduction targets than to meeting community targets. It is of course important to note that some groups saw achieving CO2 reductions as a project success.

Stakeholder perspectives

8.20 Diverse views emerged from stakeholders regarding the utility of community group carbon reduction estimates. It was suggested that removing the CO2 measurements (which in some contexts, such as food-growing projects, were viewed as of limited use) would make funding more accessible, and that the narrow measurement of carbon penalised projects and inhibited long-term sustainability. An opposite proposition was to focus on supporting projects to become embedded in their communities to foster long-term behaviour change. Moreover, one respondent felt the real value of the CCF scheme was in encouraging new ways of thinking and other 'softer' benefits (such as community empowerment) rather than carbon legacy. As such, the merit of the carbon evaluation requirement (and the efforts taken to undertake calculations) was queried.

8.21 However, measurement was seen as a means to continue to raise awareness of CO2 emissions and climate change within communities. This was despite difficulties in comparing carbon reduction figures between projects and actual CO2 savings often being very much less than estimated savings.

Behaviours

8.22 This section describes key insights arising from the case studies into facilitating and encouraging behaviour change amongst community audiences.

Audience Targeting and Messaging

8.23 It was clear that for many case study projects there was very little specific targeting in terms of the audiences for their project. Many projects had not considered particular audience segments at project outset and were targeting their projects at any possible participant: "[we were targeting] anybody and everybody - but [it] changed throughout experience of project…to those that could change" [CS02 Int2.2]. There were however some projects that were designed with a specific audience (e.g. those that were based around schools, particular ethnic groups, small geographic communities, housing association tenants). However, even within these more targeted projects, groups typically did not take a strategic approach to developing engagement routes to different audiences, for example by identifying the best communications channel to reach these audiences, or tailoring messages at project outset. There was however, significant evidence of groups learning as their projects developed, through focusing in on particular target audiences or groups (e.g. schools or businesses) or realising the impact of new communications channels (such as social media). Many indicated that having this learning at project outset could have assisted their project delivery.

8.24 Most groups found that their target audiences typically did not want to engage on climate change and that other elements of their projects had more 'traction', as the following quotes describe: "Some people are keen on their own individual footprint and will make lifestyle changes to reduce it - but they are few and far between." [CS05 Int 1.2]; "[the] most important thing is climate change - but this is not the most important thing within the community - if we talked about climate change to start with it would not start an easy conversation. Usually start with bills and then link back to climate change - highlighting that it is beneficial for their pockets but also for the environment" [CS10 Int1].

8.25 Despite the lack of strategic audience-targeting at project outset, many groups developed their messaging throughout their projects to more effectively engage and meet the needs and drivers of their audiences. Examples of this included, energy-saving projects focusing on bill-saving, energy-saving or the environment depending on the type of client engaged, or the following food waste project shifting the focus onto community (rather than individual) benefits and success: "Some people aren't that interested in putting their food waste in a caddy but when they hear collectively what [the area] has achieved they get a sense of pride" [CS05 Int 1.2].

8.26 Other groups began to shift their efforts onto particular institutions or 'gatekeepers' for engagement, such as working through schools or seeking to work with businesses to access, and maintain contact with, groups of potential participants to both engage and monitor in the longer term.

Measuring Behaviour Change

8.27 Measurement of behaviour change was a very real challenge for many groups. Reporting and data collection of changes to individuals' behaviour was explored through a variety of routes across different project types, including the use of questionnaires and surveys, waste audits, travel surveys, use of online cycling recording software to track travel behaviour and energy questionnaires. Where these were undertaken, for many groups, the level of responses received was typically lower than had been hoped for, resulting in many groups being less confident than they had hoped to be in their results: "[we] weren't privy to some of the changes made [this was] very frustrating as we can't record it as a project success" [CS03 Int2.2, B4].

8.28 For others, development of the processes to record 'before' and 'after' behaviour, or to understand specifically what behavioural outcomes they could achieve, was commenced too late in the process. This was initially less of a focus for some than getting the project delivery aspects underway. Indeed one respondent commented that they attended training on this towards the end of their project, when they felt it was too late.

8.29 That said, there was some clear evidence of good practice and adaptive thinking from groups in terms of ways to measure change. For example, one energy advice project developed their interventions with recipients to ensure that baseline energy data was gathered directly from bills, or requested from suppliers, during the first contact with their recipient. This was followed up with further monitoring via meters and energy monitors. Other projects sought to work with other organisations (such as schools or other groups) to encourage and organise the collection and return of information on changes made (e.g. through reminding and encouraging their members to undertake surveys). There was also evidence of groups using incentives such as competitions or prize draws on surveys to encourage feedback on behaviour changes.

8.30 Alongside these challenges there was also an acknowledgement by many case study respondents that behaviour change is a long-term process and, as a result measurement, is difficult: "one single thing doesn't change behaviour overnight it is the community changing gradually over time" [CS13 Int1 B3]. In line with this, many felt that their projects were contributing to change over time, rather than delivering instant change: "Behaviour change can't be achieved in such a short period of time" [CS19 Int1.2 B3].

Summary: Carbon calculations and behaviour change

8.31 Calculating carbon savings was perceived as a challenging, complex and confusing process for many groups and, as a result, setting, recording progress against and meeting targets became a significant concern and required substantial efforts to undertake. There was also some concern that this was acting as a barrier to progressive activity within community groups as too much focus was being placed on this element.

8.32 The inclusion of carbon targets in applications was shown to impact on projects both positively and negatively: adding elements to some projects to increase potential carbon savings, but restricting activities for other groups where it was felt that measurement of changes would be difficult. There was a lack of clarity over the purpose for which the carbon calculations were undertaken, with some groups mistakenly assuming that the figures are directly used by the Scottish Government in their calculations of performance against climate change targets, and others unsure of the purpose of the calculations and how they are used.

8.33 These findings would suggest that clarity over the purpose and the routes used for projects to set carbon emissions savings targets, and measure progress against them, would be of benefit to any future fund. Consideration should also be given to the extent to which this element should be focused on, given the efforts required and difficulties faced by groups to deliver on carbon reporting.

8.34 In terms of delivering behaviour change to their audiences, there was limited evidence of groups undertaking audience targeting and tailoring at project outset, and there may be scope for any successor fund to support and encourage this through provision of resources, guides and training. Measurement of behaviour change was a real challenge and this, along with insufficient consideration of behavioural outcomes at project outset, made this process difficult.

8.35 These findings highlight the scope for additional support and enhanced sharing of best practice in any future or successor scheme, to prevent projects repeating the mistakes of other projects in terms of planning, monitoring and delivery.