Annex 3: Monitoring and Evaluation
Foundation and Principles
Establishing a comprehensive monitoring and evaluation (M&E) framework for the Adaptation Programme is vital to ensure the effectiveness of Scotland’s efforts to adapt to the impacts of climate change, identify whether resilience is increasing and opportunities are being realised, and ensure that reporting on progress and implementation is evidence-based.
The first Adaptation Programme was accompanied by significant developments in adaptation monitoring in Scotland and evaluation of the first Programme was able to draw on:
- Annual Public Sector Climate Change Reporting
- Adaptation Indicators published in 2016 by ClimateXChange (Scotland’s centre of expertise on climate change).
The monitoring and evaluation framework for the second Adaptation Programme builds on this monitoring foundation and has been developed in response to specific recommendations from both the Adaptation Sub-Committee (of the UK Climate Change Committee) and ClimateXChange to ensure that we can effectively monitor implementation of the Adaptation Programme and track progress towards the outcomes.
The following principles guided the development of the framework:
Principle 1: Indicators to measure progress will be considered at the same time that planned outcomes are identified. This will encourage the development of measurable objectives to enable:
- external evaluation of progress
- internal evaluation of delivery and progress
- timely changes to the programme in response
Principle 2: The adaptation process will be monitored to assess whether the programme actions are taking place and that policies and interventions are on track. Process indicators support accountability in the short term, but also monitor the implementation of actions which are aimed at achieving long-term outcomes out-with the usual programme timeframes.
Principle 3: The framework will link the adaptation process to adaptation outcomes and aims to discourage the listing of policies and actions without considering their potential effectiveness.
Principle 4: The identification of outcome and process milestones will be encouraged to assess interim progress. Routinely identifying milestones and targets, specifying a timetable and considering potential effectiveness, will aid the reporting process and enable the delivery of flexible adaptation strategies.
Principle 5: Existing indicators and monitoring frameworks will be utilised where appropriate. This will facilitate integration of adaptation across other policy areas, help to align M&E mechanisms and minimise duplication of reporting effort.
Principle 6: Improvement and learning underpins the framework- by identifying what we need to measure not just what we know we can, the framework can be used as a tool to highlight monitoring gaps which could be filled by future adaptation measures.
The Use of Themes to Structure Monitoring and Evaluation
Outcome and process monitoring themes will structure the quantitative and qualitative evidence for evaluating progress. The following tables for Outcome 5 and 6 provide examples of how this structure will:
- Draw upon existing indicators where possible and appropriate,
- Identify potential indicators for future development,
- Provide case studies where more appropriate or where metrics are unavailable, and
- Help to highlight monitoring gaps.
| Outcome 5: Monitoring and Evaluation Indicators |
| Process Monitoring (What are we doing?) |
| Themes |
Indicators |
| Reducing non-climate pressures |
Existing indicators:
- Area of woodland with active, approved deer management plans (Scottish Forestry Strategy; CXC Adaptation indicator NF14)
- Freshwater bodies affected by diffuse pollution due to agriculture (SEPA River Basin Management Plans (RBMPs) pressure data; CXC Adaptation indicator NA14)
- Freshwater bodies with less than good morphological status (SEPA RBMPs classification data)
- Soil sealing (Ecosystem Health Indicator 13)
Potential indicators:
- Management of Invasive Non-Native Species (Ecosystem Health Indicator 11- currently utilises presence data for a number of key species but ‘In the future, absence records will show where these species have been effectively removed through management’)
|
| Habitat restoration/ creation with co-benefits |
Existing indicators:
- Peatland restoration (Climate Change Plan indicators; Ecosystem Health Indicator 10)
- Number of hectares of restored peatland per year
- Number of hectares on the road to recovery
- Number of projects approved for funding from the Peatland Action restoration project funding
- Native woodland and forest creation (Climate Change Plan indicators)
- Number of hectares of woodland created
- Area of new woodland created with grant scheme support
- Area of new woodland created on the national forest estate
- Extent of urban greenspace (State of Scotland’s Greenspace)
- Number of planning authorities with current Forest and Woodland Strategies (Climate Change Plan indicator)
Potential indicators:
- Extent of Natural Flood Management schemes (NFM network)
Case studies:
- Creation of pollinator friendly habitats (Pollinator Strategy progress reports)
|
| Increasing collaboration & flexibility |
Potential indicators:
- Area of land under landscape scale conservation (CXC Adaptation Indicator NB7- based on 2014 data gathered by Scottish Forestry)
- Progress towards a National Ecological Network (SNH)
Case studies:
- Place-based partnerships for sustainable land use
|
| Managing resources sustainably |
Existing indicators:
- Sustainability Certification Schemes (Aichi Target 7 monitoring)
- High Nature Value farming and forestry (Aichi Target 7 monitoring; Ecosystem Health Indicators; CXC Adaptation Indicators NA9 and NF3)
- Use of the Ecological Site Classification (ESC) decision support tool (Forest Research; CXC Adaptation Indicator NF6)
Case studies:
- Sustainable land management projects which protect and improve water quality in catchments.
Sustainable management of natural resources is also considered under sub-outcome 3.1 in the Economy outcome. |
| Maximising health and wellbeing benefits |
Potential indicators:
- Prescriptions for Green Exercise (NHS Greenspace/ Our Natural Health Service Programme)
- Extent/ creation of greenspace in Air Quality Management Areas
Case studies:
The health benefits of the natural environment are also considered under sub-outcome 2.2 in the Climate Justice outcome. |
| Increasing knowledge and understanding |
Existing indicators:
- Number of land managers/ consultants trained through the Peatland Action programme (Climate Change Plan indicators)
Case studies:
- Citizen science monitoring programmes (e.g. Pollinator Monitoring Scheme)
- Natural capital approach on National Nature Reserves (SNH to pilot this to better communicate the socio-economic values of nature)
|
| Improving access to the natural environment |
Potential indicators:
- Green Infrastructure funding (Green Infrastructure Strategic Intervention)
- Number of greenspaces improved and regularly used for outdoor learning (Outdoor Learning in Nature Fund)
- Extent and connectivity of green corridors for active travel (Sustrans)
Case studies:
- Green Infrastructure projects (e.g. funded through the Green Infrastructure Strategic Intervention)
|
| Sub-Outcome Monitoring (Is it working?) |
| Habitat extent and connectivity is protected and enhanced |
Existing indicators:
- Extent of deep peat habitat (James Hutton Institute soil data; CXC Adaptation Indicator NB11)
- Extent of native woodland (CXC Adaptation Indicator NB10a based on the Native Woodland Survey Scotland)
- Functional habitat connectivity (Ecosystem Health Indicator 8)
|
| Habitat condition is protected and enhanced |
Existing indicators:
- Condition of native woodland (Ecosystem Health Indicator 3)
- Condition of freshwater bodies (Ecosystem Health Indicator 6; CXC Adaptation Indicator NB24)
- Invasive non-native species (Ecosystem Health Indicator 11; CXC Adaptation Indicators NB37 & NB39 specific for native woodland and freshwater INNS)
Potential indicators:
- Condition of peatland (CXC Adaptation Indicators NB13- draws upon evidence from various sources, update sources to be determined)
Case studies:
- Impact of extreme events on protected sites or key habitats
The condition of protected sites is an National Performance Framework indicator considered at outcome level. |
| The diverse natural environment is protected and enhanced |
Existing indicators:
- Abundance of wintering water birds (Scotland Biodiversity Indicator; CXC Adaptation Indicator NB6b/NB17b)
- Abundance of specialist and generalist butterfly species (Scotland Biodiversity Indicator; CXC Adaptation Indicator NB16b)
Potential indicators:
- Changes in species suite (under-development as an Ecosystem Health Indicator)
- Measurement of genetic diversity (under-development to enable assessment of Aichi Target 13)
|
| Regulating services are maintained |
Existing indicators:
- Pollinator monitoring (Indicators of Ecosystem Services in Scotland; Pollinator Strategy)
- Soil organic carbon stocks (Indicators of Ecosystem Services in Scotland; Ecosystem Health Indicator 7)
- Carbon sequestration (Scottish natural capital: ecosystem service accounts)
- Air pollutant removal by vegetation (Scottish natural capital: ecosystem service accounts)
|
| Supporting services are maintained |
Monitoring will draw on the cross-cutting themes above on habitat extent, condition and diversity. |
| Provisioning services are maintained |
Existing indicators:
- Area of class 1 agricultural land available (James Hutton Institute Land Capability for Agriculture; CXC Adaptation Indicator NA2)
- Abstraction of water for irrigation (SEPA Water Resources Data Returns System; CXC Adaptation Indicator NA13)
- Natural regeneration in native woodland (Native Woodland Survey Scotland; CXC Adaptation Indicator NB23)
Potential indicators:
- Contribution of woodlands, forests and the forest sector to the Scottish economy
- Volume of available wood fibre
Provisioning services are also considered under sub-outcome 3.1 in the economy outcome. |
| Cultural services are maintained |
Existing indicators:
- Outdoor recreation visits (Scotland’s People and Nature Survey, SNH; Scottish Household Survey)
Potential indicators:
- Combined health and ecosystem indicator (under development by NHS Scotland, SNH and SEPA)
- Numbers of visits to forests and woodlands (Scotland’s People and Nature Survey)
|
| Understanding and recognition are increased |
Existing indicators:
- Attitudes towards the natural environment/ Identification of benefits gained from visits to the outdoors (Scotland’s People and Nature Survey/ Scottish Nature Omnibus, SNH)
- Numbers of people taking active steps to improve the natural environment (Scottish Nature Omnibus, SNH)
Potential indicators:
- Evidence of public understanding of ecosystem services
Case studies:
- Examples of businesses considering the environment (and its value) in decision-making
|
| Outcome 6: Monitoring and Evaluation Indicators |
| Process Monitoring (What are we doing?) |
| Themes |
Indicators |
| Reducing non-climate pressures |
Potential indicators:
- Proportion of Marine Protected Areas (MPAs) conservation objectives with measures in place (MPA reporting)
- Proportion of vulnerable areas identified in the National Coastal Change Assessment covered by Shoreline Management Plans
Case studies:
- Evidence of good practice in the prevention of biofouling
|
| Habitat restoration/ creation with co-benefits |
Potential indicators:
- Extent of restoration of coastal habitats (saltmarsh, dunes)
- Creation/ restoration of coastal habitats for flood prevention
Case studies:
- Managed realignment projects
|
| Increasing collaboration & flexibility |
Potential indicators:
- Use of the Place Standard Tool in coastal community planning
- Development of regional Marine Planning Partnerships
|
| Managing resources sustainably |
Existing indicators:
- Proportion of fish stocks exploited below FMSY (fishing mortality consistent with achieving maximum sustainable yield) (Marine Plan; UK Marine Strategy indicator)
Potential indicators:
- Vessel monitoring (Outcome 2 of Inshore Fisheries Strategy: ‘Marine Scotland will implement an appropriate form of vessel monitoring to provide good quality information on the footprint of inshore fishing’)
- Sustainable development of offshore renewables (Scottish Marine Energy Research (ScotMER))
- Proportion of coastal planning decisions that consider climate change impacts (e.g. anticipated coastal erosion)
Sustainable management of natural resources is also considered under sub-outcome 3.1 in the Economy outcome. |
| Maximising health and wellbeing benefits |
The health benefits of the natural environment are also considered under sub-outcome 2.2 in the Climate Justice outcome. |
| Increasing knowledge and understanding |
Potential indicators:
- % MSS (Marine Scotland Science) survey programme completion
- % MSS data submission
- Marine Scotland research spend (internal projects and external commissions)
- Extent of marine invasive non-native species (INNS) monitoring (Scottish Marine INNS Working Group)
- Number of beaches routinely surveyed for litter (Marine Conservation Society Beachwatch programme)
Case studies:
- Citizen science monitoring programmes (e.g. CoCoast, Bioblitz)
|
| Improving access to the natural environment |
Potential indicators:
- Survey of connectivity between the public and marine systems
|
| Sub-Outcome Monitoring (Is it working?) |
| Habitat extent and connectivity is protected and enhanced |
Existing indicators:
- Extent of key coastal habitats e.g. dunes, machair, saltmarsh (CXC Adaptation Indicator NB10b)
- Extent of key marine habitats e.g. sea grass, kelp-beds, cold-water coral (National Marine Plan, Marine Scotland)
Potential indicators:
- Total area of protected sea (MPAs, SACs)
|
| Habitat condition is protected and enhanced |
Existing indicators:
- Prevalence of marine/ coastal litter (OSPAR; Marine Conservation Scotland; MSS Pilot Scottish Beach Litter Performance Indicators)
- Condition of Marine Protected Areas (SNH/ Marine Scotland)
Potential indicators:
- Prevalence of key marine/ coastal non-native invasive species (Scottish Marine INNS Working Group)
- Seafloor integrity assessment (Marine Strategy Framework Directive (MSFD) descriptor 6)
Good Environmental Status in Scottish waters is an MSFD indicator considered at outcome level. |
| The diverse natural environment is protected and enhanced |
Existing indicators:
- The numbers and breeding success of seabirds (Biodiversity Strategy indicator; MSFD indicator)
- Composition and relative proportions of ecosystem components (MSFD indicator)
- Abundance trends of functionally important selected groups/species (MSFD indicator)
Potential indicators:
- Relative abundance of warm water species (Good Environmental Status indicator- ratio between abundance of warm and cold water species; Community Temperature index - MarClim)
|
| Regulating services are maintained |
Existing indicators:
- Blue carbon resources in Scotland’s MPA network (Marine Scotland)
Potential indicators:
- Evidence of coastal protection afforded by coastal habitats (Dynamic Coast)
- Evidence of saline intrusion (SEPA)
- Marine regulating services (e.g. waste remediation, coastal protection, nursery habitats) (SNH feasibility study for a marine natural capital asset index; National Marine Plan)
|
| Supporting services are maintained |
Monitoring will draw on the cross-cutting themes above on habitat extent, condition and diversity. |
| Provisioning services are maintained |
Existing indicators:
- Fish and shellfish stocks/landings (Marine Scotland)
- Aquaculture fish and shellfish production (Marine Scotland)
Provisioning services are also considered under sub-outcome 3.1 in the Economy outcome. |
| Cultural services are maintained |
Existing indicators:
- Number of harmful algal blooms (Food Standards Scotland; MCCIP; CXC Adaptation Indicator NM7)
- Bathing water quality (SEPA)
- Numbers of visits to beaches (Scotland’s People and Nature Survey)
Potential indicators:
- Public perceived value of coastal sites
|
| Understanding and recognition are increased |
Existing indicators:
- Awareness of MPAs and their role (Scottish Nature Omnibus, SNH)
Potential indicators:
- Proportion of Crown Estate Scotland assessed for social, economic and environmental benefits and impacts of future decision-making (Marine Scotland and Crown Estate Scotland- under development)
|
