Draft climate change plan: draft third report on policies and proposals 2017-2032

6. Monitoring and evaluation

6.1.1 Monitoring the implementation of the Climate Change Plan is necessary to ensure effective implementation and communication of progress under the Climate Change (Scotland) Act 2009. Our proposed monitoring framework builds on approaches to assess the previous reports on policies and proposals and on the method applied by the Committee on Climate Change to monitor progress against the UK carbon budgets. This monitoring framework marks a new approach to understanding the implementation of the Climate Change Plan. We will continue to develop the approach over the course of its application.

We have used the following principles to design and develop the framework:

PRINCIPLE 1: The monitoring framework will focus on both policy implementation and policy outputs. This is necessary to describe progress and also to support learning and continuous improvement in policy delivery.

PRINCIPLE 2: Development of proposals will also be monitored to ensure the plan is on track for its longer-term goals.

PRINCIPLE 3: Over time we will develop policies that are SMART (Specific, Measurable, Attainable, Realistic and Time limited) to allow them to be monitored.

PRINCIPLE 4: The monitoring framework will be maintained online and a summary report published annually from 2018 onwards.

PRINCIPLE 5: While the monitoring framework will ensure a consistent approach across the plan it will be proportional in its data and indicator demands to reflect the range of different policies within the Climate Change Plan.

PRINCIPLE 6: Improvement and learning underpins the framework. Policy design and implementation will evolve and respond to the learning from monitoring to support improvement in delivery.

6.1.2 The monitoring framework will provide the information required to allow for ongoing policy development and improvement in implementation. Policy implementation will, therefore, be dynamic and responsive to the learning from monitoring data. The indicators within the monitoring framework provide the direction of travel that is expected to occur as a result of a particular policy. The collection of data ensures evidence-based decisions can be taken to amend the policy approach in response if required.

6.1.3 The monitoring framework applies a suite of indicators to present an accurate description of progress and a full explanation of policy implementation. Indicators apply the SMART criteria. This means that they must be Specific, Measurable, Attainable, Repeatable and Time relevant. Importantly the data behind each indicator will be of sufficient quality to allow confident interpretation of its meaning. We are developing the suite of indicators in the monitoring framework to answer the following questions:

• Is implementation of the policies and proposals in the Climate Change Plan on track?
  This will provide a description of progress against the Climate Change Plan timelines.
• How is the policy implementation process progressing?
  This will allow for learning and improvement in policy delivery.

6.1.4 We are using the following indicators to monitor the implementation of the Plan:

• Annual greenhouse gas emissions statistics. These present the highest level outcome indicator. The presentation of sector greenhouse gas emissions provides the raw data that, when combined with the indicators below, allows for the verification of sector projections and policy development.
• Policy outcomes. Policy outcomes are the measurable consequence of a particular package of policies and proposals. For example a suite of policies and proposals are intended to help increase the sale of electric vehicles. Progress against each of the policy outcomes will be monitored on an annual basis.
• Policy implementation indicators. These indicators will address the question: How is the policy implementation process progressing? In order to capture the learning needed to allow for quality improvement in policy delivery, and to offer an explanation of outputs, we will identify and apply implementation indicators for a selection of policies. For each indicator a projected trend will be described using quantitative measures, this is necessary to understand the implementation process.
• Policy output indicators. These indicators will address the question: Is implementation of the Climate Change Plan on track? We will use policy output indicators to determine if implementation of the Climate Change Plan is on track and will flag any policy area that needs particular attention. In some cases it will be possible to measure the policy output on an annual basis, in others it will be necessary to use an indicator to describe progress. For each indicator a projected trend will be described using quantitative measures. This is necessary to understand the implementation process.
• External drivers. Policy outputs will be achieved as a result of policies working in combination with external drivers. It is important to track the key drivers as they are likely to provide a significant explanation for progress. External drivers will include contextual factors (such as demographics, oil price etc) and drivers that we expect to see a positive trend in as a result of collective efforts to build a low carbon future (e.g. reductions in technology costs).
• Long-term trend indicators. In certain sectors there may be policies that are responsible for contributing to long-term trends. Where these are significant we will identify and monitor them. Examples of long-term indicators might include attitudinal surveys, uptake of vocational courses or graduation from particular degree courses.

6.1.5 The draft Climate Change Plan describes proposals as making a significant contribution in future years when their development allows them to be converted to policies. To ensure the implementation of the Climate Change Plan remains on track, the monitoring framework will track the development of proposals. We will ask the following questions of each proposal:

• What action has been taken to develop the proposal?
• What are the next steps to progress the proposal to policy?
• Will the proposal still deliver the projected outcome on time?
• Is the proposal still relevant in the context of changing circumstances?
• What are the risks to developing the proposal further and how will these be addressed?

6.1.6 We will set up a governance body to oversee and respond to the monitoring and implementation of the Climate Change Plan. It will have an important role in ensuring the Scottish Government and its partners are able to respond to the results of the monitoring framework, applying the learning from the suite of indicators to ensure quality improvement in policy action and rigorous scrutiny of implementation. This body will provide advice to the Scottish Cabinet on a regular basis.

6.1.7 We will continue to develop the monitoring framework and intend to publish an update alongside the final Climate Change Plan later in 2017. The final version will be published in 2018, from when we plan to publish annual summary monitoring reports.

Monitoring reserved climate policies

6.1.8 Implementation of the Climate Change Plan is reliant on a combination of EU, UK and Scottish policy. The monitoring framework has been designed to understand and improve the delivery of Scottish Government climate policy. However, it must also capture the progress made under UK and EU policy direction. To this end the required policy outputs from UK Government and EU policy will be assessed at regular intervals to see if they are consistent with the assumptions made in the Plan.

6.1.9 The following tables present examples of how the monitoring framework will be applied to policies from different sectors.

Table 6-1: Peatland monitoring example

Table 6-2: Forestry monitoring example

Figure 3: The monitoring framework