Climate change - Scottish National Adaptation Plan 2024-2029: monitoring and evaluation framework

Monitoring Indicators

Background

As outlined in the Plan, success will be measured by tracking progress made towards the 5 outcomes and 23 objectives set out in SNAP3. A set of indicators has been developed to monitor progress towards these outcomes and objectives.

Objective indicators will be reported on annually. These are based on indicators of progress at the objective level of SNAP3. They represent measures which we can expect to see change at an annual level^[3]. These will, therefore, be reported on alongside the Plan's annual progress reports.

Outcome indicators represent the higher-level trends in resilience we expect to see to meet the 4 domestic outcomes set out in SNAP3. If progress is being made towards the policies and activities set out in the Plan, we would expect to see some positive changes in these outcomes at the end of the SNAP3 period. Therefore, these outcome level indicators are being measured at beginning of the Plan to establish a baseline and will be reported on again at the end of the five-year period to allow and assessment of progress.

It is important to note that trends in Scotland's overall resilience will take place over a longer timescale than the 5-year SNAP3 period. While we hope to see annual changes in our objective indicators and changes in our outcome indicators at the end of the Plan period, many of the changes in resilience we want to see will take longer than the SNAP3 period to be fully realised, particularly for outcome-level trends. This monitoring represents a first step towards greater understanding of trends in Scotland's resilience (which is also measured by the more in-depth CCRA). In the longer term, monitoring will continue past the SNAP3 period to understand whether sustained progress towards outcomes is being achieved and leading to Scotland becoming more resilient. This longer-term monitoring will make use of developments and improvement in data quality to help capture the best possible picture of Scotland's resilience at the time.

The indicators presented in this framework cover the four domestic level outcomes and accompanying objectives in the Plan. At the outcome level these are the areas where we are able to gather data which can capture progress towards Scotland becoming more resilient. The international outcome (Outcome 5) and its objectives will be reported on through a narrative summary of progress in the annual progress reports. Improving data collection and availability around international objectives will be one of the aims of future data development work.

The following pages present the monitoring framework and its indicators at the outcome and objective level. There is, in some places, overlap between reporting at an outcome and objective level. Indicators and data availability will be reviewed over the SNAP3 period as data development aims to improve the availability of data which can be used to track those longer term trends at the outcome level. More detailed information about each indicator is included in Appendix 1.

Nature Connects Indicators

Outcome indicators (baseline and 5 year follow up reporting)

Outcome: Nature connects across our land, settlements, coasts and seas.

Objective indicators (annual reporting)

Communities Indicators

Outcome indicators (baseline and 5 year follow up reporting)

Outcome: Communities are creating climate-resilient, healthy and equitable places.

Objective indicators (annual reporting)

Public Services Indicators

Outcome indicators (baseline and 5 year follow up reporting)

Outcome: Public services are collaborating in effective and inclusive adaptation action.

Objective indicators (annual reporting)

Economy and Industry Indicators

Outcome indicators (baseline and 5 year follow up reporting)

Outcome: Economies and industries are adapting and realising opportunities in Scotland's Just Transition.

Objective indicators (annual reporting)

Development of indicators

The outcome and objective indicators have been developed collaboratively with input from analysts, policy leads and external stakeholders from across key policy areas in the Plan. This collaborative approach has been key to informing the identification and agreement of indicators to be included in the framework.

A long list of potential objective indicators and data sources was developed following an initial data mapping workshop and commission with stakeholders. This was supported by a desk based research exercise, including a review of international climate change adaptation monitoring and evaluation frameworks. Data from the longlist of potential indicators was then assessed through further stakeholder engagement to create an initial shortlist of draft indicators.

Each of the proposed objective indicators from this shortlist were then assessed against a set of criteria to identify those for inclusion in the final monitoring framework. These criteria are:

• Is the indicator relevant? – Does the indicator relate to key elements of climate adaptation, including vulnerability, risk, exposure and adaptive capacity?
• Is the data representative? – Is the indicator as representative as possible of the key characteristics of the objective area within the adaptation plan that it fits under?
• Is the data available? – Is data readily available and accessible for use by wide range of stakeholders?
• Is the indicator sensitive? – Is the indicator should be sensitive enough to detect changes over time, particularly in relation to the lifespan of the adaptation plan?
• Is the indicator understandable? – The indicator should be easily understandable by a wide range of stakeholders, including non-experts, to ensure effective communication
• Is the indicator practical? – Is the indicator cost-effective to use and does it have low resource requirements for data collection and analysis?
• Is indicator baseline data available? – Is it possible to set clear, quantifiable baseline for the indicator to track progress^[4]

A similar process was followed for the development of the draft outcome indicators. An initial desk-based data mapping exercise informed a series of stakeholder workshops. These workshops focused on each of the 4 outcome areas to be included in the framework and involved a wide range of stakeholders. They were used to narrow down the long list of indicators and highlight data which could fill gaps which had been identified in the data mapping exercise. The proposed indicators were then assessed against the same criteria as the objective indicators – with one additional criterion focusing on the availability of timely baseline data. Detailed information on the development of these outcome level indicators can be found in the published baseline data report.

Key data gaps and priorities for further indicator and data development

The indicators presented in this monitoring framework have been chosen as they fit the criteria set out above. This includes practicality of accessing and availability of data. This is important as it means the indicators used in the monitoring framework can be readily reported on throughout the SNAP3 period. The indicator development process has, however, identified a number of areas where additional data could strengthen our monitoring evidence base and fill identified gaps. Input from stakeholders has been key in informing our understanding of these data gaps.

While the indicator set presented here is an important step towards monitoring adaptation in Scotland, over the lifetime of the Plan we will identify opportunities for further data development and improvement. This will involve working closely with stakeholders to identify priority areas where additional data will improve our understanding of progress towards adaptation as well as opportunities where this data can practically be developed. For example, more data may be required for a robust coastal monitoring programme on coastal adaptation. This may include developing new datasets or improving existing ones so that they provide information more relevant to adaptation outcomes and objectives. Any new data collection will be proportionate and focus on evidence most relevant to the objectives and outcomes in the Plan. Where possible, efforts will be made to ensure any new data development and updated indicators align with existing monitoring and reporting requirements for other climate and environmental policies and plans. As well as reducing administrative burden, this approach will support coordinated policy and data development across policies within the climate and environment policy system.

Reporting

The timeline below outlines current plans for publishing monitoring and evaluation updates during the lifetime of the Plan. It focuses on statutory annual reporting to Parliament and independent assessment of SNAP3 by the CCC, with other research and evaluation evidence being produced throughout the SNAP3 period. Plans for reporting on evaluation evidence will be developed separately.

• September 2024 – this M&E framework is published, including a 'baseline' dataset for the SNAP3 outcome indicators.
• May 2025 – annual report to Parliament with first round of reporting on objective indicators.
• May 2026 – annual report to Parliament with second round of reporting on objective indicators and wider updates on policy progress.
• Summer-2026 – CCC's independent assessment of UK climate risk published.
• Late-2026 – first independent assessment of SNAP3 progress from the CCC.
• Early-2027 – statutory CCRA4 laid in UK Parliament (tbc).
• May 2027 – annual report to Parliament with third round of reporting on objective indicators.
• May 2028 – annual report to Parliament with fourth round of reporting on objective indicators and wider updates on policy progress.
• Late-2028 – second independent assessment of SNAP3 progress from the CCC.
• May 2029 – final annual report to Parliament with a focus on reporting on progress to objectives and update on progress made towards outcome indicators over the 5-year SNAP3 period.