Development of a combined marine and terrestrial biodiversity indicator: research

5. Further detail on producing a headline metric

5.1 Combining species data from disparate sources

92. As already discussed, most terrestrial biodiversity indicators measure trends in the status of species using abundance data. The standard approach is to calculate the geometric mean abundance each year (Buckland et al. 2005), and to scale to some baseline value in the reference year (e.g. set to 100 in the year 1970). Indicators constructed in this way are effectively tracking changes in the abundance of some notional average species. Thus, it is relatively straightforward to combine data from separate sources, so long as they can be converted into a common currency. For example, the Priority Species Indicator of species abundance (Eaton et al. 2015b) is made up of a mix of count data and modelled outputs, but all are easily converted into a metric of species' abundance.

93. Combining data into a headline metric could be regarded as more problematic when the data represent different currencies. The headline metrics in the 2016 State of Nature report (Hayhow et al. 2016) and the Dutch Living Planet Index (van Strien et al. 2016) both contain a mixture of data on species abundance and distribution (occupancy). Combining them is technically straightforward if one assumes that a given change (e.g. 10%) is comparable in both occupancy and abundance. However, there are reasons to believe that occupancy and abundance are not directly comparable, and it's likely that changes in species' occupancy underestimate changes in abundance (Bart and Klosiewski 1989; Buckley and Freckleton 2010). A recent study of moths in Scotland (Dennis et al. 2019) showed no clear correlation between abundance and occupancy trends; indeed, species with negative population trends showed varied occupancy responses. It is therefore harder to interpret changes in an indicator that combines occupancy and abundance data. For these reasons, a decision has been made to retain two separate indicators for Priority Species with the UK Biodiversity Indicators, one based on distribution data and the other based on abundance data, rather than combining them as a single indicator. Furthermore, the most recent State of Nature report (Hayhow et al. 2019, and Walton et al. 2019 for Scotland) reverted to reporting trends in abundance and distribution in separate metrics.

94. The creation of composite indicators is further complicated when the currencies are more heterogeneous, for example an abundance trend and a red list index. More extreme would be to combine an index based on abundance with an index in which the data represent something other than individual species, such as the marine trophic index. However, there are indicator approaches which enable the combination of such disparate sources, providing they can be presented as quantitative measures. The Natural Capital Asset Index for Scotland, one of the National indicators used to measure progress towards the National Performance Framework 'Economy' outcome, combines measures of natural capital across a broad range of services and habitats, using weightings to address variation in the importance of these to human wellbeing, and thus combines a total of 38 separate indicators (SNH 2019).

5.2 Disaggregation of headline indicators

95. In addition to the headline aggregated metric, many indicators are presented in a form disaggregated by taxonomic group, geography or habitat. These disaggregated indicators are often treated as supplementary information but are usually essential to inform an understanding of the reasons behind underlying trends in any high-level metric – this is particularly true when that metric summarises data across a broad suite of species that show a wide variation in trends, and if there is any systematic pattern to this variation, in other words if some subsets of species have a different trend to others. This can be seen, for example, in the UK Wild Bird Indicator (C5), in which farmland birds (C5a) show a very markedly greater decline than other species, and within the farmland birds a subset of species defined as farmland specialists show a greater average decline than those defined as farmland generalists. There is currently considerable variation in how the various disaggregations are presented. The UK Wintering Waterbirds indicator (C5e) presents separate lines for wildfowl and waders on the headline indicator plot, while for the UK priority species indicators (C4a, C4b) and pollinating insects (D1c) taxonomic disaggregations are only presented in technical annexes. By contrast, the Living Planet Indicator (WWF 2018) has many disaggregations, including into five biogeographic realms and ecosystem (marine vs terrestrial vs freshwater). In some cases, 'sub-indicators' might be necessary steps to enable the creation of the headline metric (e.g. metrics for the state of nature, or natural capital, for individual habitats/ecosystems as used in the creation of the Norwegian 'nature index', or the Scottish NCAI); in other instances they are backwards disaggregations of the headline measure.

5.3 Uncertainty and assessment

96. There are a variety of techniques for assessing change in biodiversity indicators. A common theme is to use some kind of statistical procedure to calculate the uncertainty around this headline value (e.g. the 95% confidence intervals). The magnitude of this uncertainty is then used to determine whether change in the headline indicator has been statistically significant.

97. Uncertainty in biodiversity indicators tends to be measured relative to the index value in some baseline year, which is nearly always the first year in the series. The index value in the baseline year is typically shown without error, so the uncertainty in subsequent years measures change relative to the baseline. Treating the data in this way is potentially misleading and has a number of undesirable consequences. First, the confidence intervals are constrained to become progressively wider over time, making it harder to detect statistically significant change, not easier. Second, the presentation shows only whether long-term change has been significant: it's not possible to determine the statistical significance of changes over recent years (e.g. the last decade). Third, treating the baseline as if it were known without error is counter-intuitive, because the early years of the time series are typically based on far fewer data than later years (e.g. the UK Butterfly Monitoring Scheme consisted of approximately 30 sites in the mid-1970s, but now has over 1000).

98. A variety of alternative statistical treatments exist, each of which has different implications for how uncertainty in the headline indicator can be presented and interpreted. Two (non-exclusive) options are 1) to allow the baseline to be expressed with uncertainty, and 2) to use a year other than the first year as the baseline. This could be, for example, a year with particular relevance with regards to drivers of change in biodiversity, such as when new governmental policies are initiated.

99. As noted above, there are many techniques currently employed for calculating uncertainty in the headline indicator. These approaches differ fundamentally in what the uncertainty represents, making it impossible to compare across methods. Bootstrapping across the raw data propagates uncertainty from the raw observations into the headline indicator, but it is time-consuming and not appropriate for all datasets. An alternative approach is to separate the production of species indices from assembling them into a composite indicator. Until recently, such two-stage approaches tended to ignore uncertainty in the species' indices, but new methods are now available for this purpose (Soldaat et al. 2017; Freeman et al. 2020).

5.4 Presentation of indicators

100. The most common format for presentation is as a line showing how the indicator has changed over time, with a ribbon to delimit the confidence intervals. This is a standard presentation format that is straightforward to produce when input data permit a single interpretable metric to be calculated on an annual basis (notwithstanding the issues raised in section 5.3).

101. More complex forms of presentation are necessary when the data depart from this format. For example, if the indicator were to contain a mixture of species metrics (section 4.1) and non-species data (section 4.5) then it may be impractical to combine them into a single headline metric that could be interpretable. In this instance, some kind of vote-counting approach would be required, whereby the number or proportion of separate indicators showing positive or negative trends, or no change, could be summarised.

102. In addition to the line graph, the UK species indicators, for example C5ai: Breeding farmland birds in the UK, also display stacked bars that display how species are distributed into five categories of change (2 increasing categories, 2 declining categories and one 'no change' category). These bar charts are based on the same data as the headline indicator but display a different type of information: the bar charts are primarily qualitative in that they display the balance of species into categories, whereas the line graph is quantitative in that it represents the average trend across all species. We do not propose the use of such bar charts for the new Scottish indicator, because it is unclear whether the threshold rates of change are equally appropriate across the diverse range of taxa and data types available (for example, is a 10% decrease in abundance for a fish equivalent to a 10% decline in occupancy for an insect?).

5.5 Summary

103. In this chapter we have focused in more detail on the creation of a biodiversity indicator through combining species' trend data. This includes how data from disparate sources can be combined in a single indicator, although there are good reasons for holding reservations about the combination of trends in the different currencies of abundance and occupancy (distribution).

104. We stress the value of presenting disaggregated indicators showing trends underlying headline indicators, which may hide important detail, and so aid understanding of patterns of biodiversity change.