Seabirds: strategic ornithological compensatory measures: review

3 Methods

3.1 Ecological feasibility (WP1)

3.1.1 Refining the scope of the review for each compensatory measure

We began by identifying the conservation actions relating to each compensatory measure listed in Table 4. Subsequently, we sought feedback from the project steering group. This led to a finalised list of review topics, and their associated review type and research questions (steps illustrated in Figure 3).

Figure 3. The steps used to move from the initial table of compensatory measures to a finalised list of review topics

, associated research questions, and corresponding review types following consideration and discussion with the steering group.

1. Initial table of compensatory measures (Table 4):

• Strategic Compensation Workshop held by the Marine Directorate
• Pearce-Higgins et al., 2021 MarPAMM report

2. Define corresponding conservation actions:

• RSPB consideration
• Steering group discussion
• Measures identifed following ECOWINGS Expert Ellicitation

3. Finalised list of review topics and associated research questions:

• Finalised list of reviews
• Types of reviews identified (systematic, targeted, or scoping)
• Research questions identifed

The initial table of compensatory measures (Table 4) was developed outwith this project and provided by the Marine Directorate. This provided 12 short-listed compensatory measures along with a list of the seabird species most likely to benefit from each one. This table was derived from compensatory measures identified in a workshop held by the Marine Directorate (then Marine Scotland) in 2022. The workshop, attended by representatives from different parts of Marine Scotland (now either within the Marine or Offshore Wind Directorates; including Science, Licensing, and Planning functions), NatureScot, and RSPB, focussed on identifying compensatory measures for seabird species in Scotland that could potentially be progressed in a strategic way.

Additionally, conservation actions identified by Pearce-Higgins et al. (2021) were also considered. This study considered two types of actions, those directly targeting the same impact mechanism, (termed ‘counteracting adaptation responses’), e.g. creating new inter-tidal habitat to offset that same type of habitat lost, or indirectly (termed ‘compensatory adaptation responses’), by benefitting a species but not addressing the same impact mechanism. Note, while that report uses the term ‘compensatory adaptation response’ the focus was on compensating (i.e. offsetting) the impacts of climate change on seabird populations, so it was not in the context of how compensation is considered in this study (i.e. compensation under the Habitats Directive). However, as most of the actions identified in that report were general conservation actions (i.e. not specific to climate change), many also have the potential to be considered as options for compensating for OWF impacts.

We then sought to map each of the compensatory measures (Table 4) to conservation actions. Some compensatory measures were more easily allocated to conservation actions than others and, therefore, scored the difficulty of allocating actions to measures on a scale of 1–5 (Table A 1). This was first assessed internally by the project team and was subsequently discussed with the project steering group (meeting held 20^th March 2023). Key points arising in this discussion are summarised in Table A 1. We also considered the compensatory measures and associated conservation actions identified by an Expert Elicitation Exercise conducted within the ECOWINGs project (K Searle, personal communication). After identifying the relevant conservation actions for each compensatory measure, and incorporating feedback from the steering group, we designated the type of review to be applied to each measure (described in the ‘outcome' column of Table A 1).

Reviews are a ubiquitous evidence synthesis approach undertaken to identify and collate relevant literature, aiming to draw conclusions on specific research questions and/or to summarise the current state of knowledge within a particular topic or research field. There are two main types of reviews, scoping reviews and systematic reviews (Munn et al., 2018), and another less-common type of review, targeted reviews. Here we used the three types of review approaches (summarised in Table 5) depending on how easily the initial list of compensatory measures could be mapped onto conservation actions (for additional information on the review types and rationale behind these see Appendix 2: Description of the different type of reviews considered within the project).

Table 5. Summary of the types of review used in this report with a description of how these are used and how their approaches differ.

This table can be viewed in Excel format.

Based on the initial 12 compensatory measures, we identified 12 potential reviews to be conducted (Table 6); while the number of reviews remains consistent with the initial count (Table 4), there is not a direct one-to-one correspondence between the original compensatory measures and the final 12 reviews. For example, ‘biosecurity’ merged with ‘predator eradication’ while ‘predator eradication’ itself was split into ‘mammalian predator eradication/management’ and ‘avian predator management’. For each review, the conservation action or review topic were defined along with associated research questions. Of the 12 reviews, seven are systematic reviews, four scoping reviews, and one a targeted review.

Table 6. Finalised scopes for reviews and types of reviews to be used.

This table can be viewed in Excel format.

3.1.2 Systematic review – general methods

We conducted systematic literature reviews where possible (Table 6), to gather available published evidence regarding the impact of the conservation actions on seabirds. At this stage, our main objective was to undertake a comprehensive and unbiased overview of all published and available studies that have, directly or indirectly, tested the effect of the identified conservation actions on the focal seabird species, or, when not possible, proxy species. If performed correctly, a systematic review guarantees to find the most relevant and current knowledge in a thorough, objective, unbiased, and reproducible manner (Kugley et al., 2016).

3.1.2.1 Literature search

Following Foo et al. (2021) and Higgins et al. (2022), for each systematic literature review we undertook the steps shown in Figure 4. An in-depth description of each step can be found in section 1 of Annex 1 (the following is an abridged version of this).

1. Defining and identifying the research questions and keywords: Overall, research questions had the following structure: ‘Would ‘conservation action/s’ benefit species?’ In most cases, each systematic review centred on a single research question, but, in some instances, several research questions were addressed within a single review. Keywords varied depending on the scope of the research questions but mostly encompassed synonyms of the conservation action, the focal species, and other key words.

2. Conducting the literature search:

a. Search engines: We conducted the literature search using two complementary search engines, Web of Science (WoS) and Google Scholar (GS).

b. Search strings: We used a selection of the previously identified keywords to produce appropriate search strings. Information on the time and date that search strings were used, as well as the number of records retained for screening can be found in Annex 2.

3. Screening and selection of relevant literature: References obtained from WoS and GS were uploaded and screened in CADIMA (Kohl et al., 2018). After eliminating duplicated records, we screened all references following a pre-defined set of inclusion criteria (Table A 2). References were screened at a title and abstract level, and those that advanced to the next stage were then screened at a full-text level. Inclusion criteria depended on the scope of the topic, but overall, we were looking for studies that tested, either indirectly or directly, the effect of the conservation action on seabirds. Information on the complete reference list obtained during the literature searches, and the level at which each reference was eliminated can be found in Annex 3.

4. Data extraction: In a Microsoft Office Excel spreadsheet, we recorded details of all references screened at a full-text level, including initials of reviewer, study title, authors, publication year, document type, whether we had access to the full text, and a summary of the study, including reasons for exclusion (this information can be found in Annex 4).

5. For those references that were deemed relevant for the final assessment, we also recorded information on the study species, duration and location of study, note on which seabird demographic parameter was tested, and relevant information regarding the methods, results, and conclusions of the study. Additional information specific to the actions was also extracted if deemed relevant. All this information can be found in Annex 4.

6. Additional literature: Making use of expert judgment and subject knowledge of the authors, we identified highly relevant key studies. All references within these studies were scanned to identify any additional relevant references. All newly identified references were screened at full-text level, repeating steps 4 and 5. While this step may decrease the reproducibility of the systematic reviews, we deemed it important to include these references to ensure that project included most of the key literature on the subject.

Steps 3–5 were performed by one of the authors (either CTH or TE). The inclusion/exclusion criteria, however, were agreed on prior to these steps. Early in this process, we met regularly to refine the inclusion and exclusion criteria, standardise data extraction methods, and to jointly screen several examples.

3.1.2.2 Analysis of evidence (scoring)

There are two components to scoring each conservation action in terms of ecological feasibility: a measure of the overall efficacy of the action, and a measure of our confidence in the estimation of that efficacy. Here we define efficacy as a combination of the likelihood that a compensatory measure is beneficial, and the size of effect found across studies. To do this it is necessary to first evaluate the individual studies across a range of scoring factors and then to combine these scores across studies. As some factors apply at a study level (e.g. sample size) and others are a property of the combined studies (e.g. concordance), some factors are calculated by taking means or medians, while others are evaluated by comparing across the studies. The scoring approach used here was developed for this study, however it considered best practice guidance on analysing evidence in systematic reviews (Higgins et al., 2022) and the approach used for a similar study of compensatory measures (McGregor et al., 2022) as well as similar work on understanding threats and pressures to seabird populations (Rogerson et al., 2021). The approach taken broadly follows that developed and advocated by Sutherland et al. (2021) for the objective evaluation of conservation actions.

3.1.2.2.1 Weighting of papers by relevancy

For the evaluation, we aimed to account for the relevance of each included study when forming an overall conclusion. This involved assigning scores to each study, which were than used to determine their relative importance to produce a combined score (i.e. a weighted mean). Each paper was scored against six factors (Table 7) measuring aspects of study relevancy; each factor was scored on a qualitative scale of 1–5. The overall score for a study was then the sum of these six factors (range of possible scores being 6–30). A study’s weight was calculated by dividing the study’s total score by the sum of the total score for all studies included for that conservation action.

The six scoring factors were chosen to cover key aspects of evaluating each piece of evidence:

• ‘Type of reference’ was scored highest for studies that were peer-reviewed and that included new data (i.e. primary literature); lowest for evidence lacking any clear peer-review and where evidence was preliminary (typically studies presented as talks/posters at conferences); and intermediate for secondary literature that synthesised primary evidence such as paper reviews, commissioned reports, book chapters, and master’s theses. Note that while master’s theses usually include new data, they represent a lower evidence standard than a PhD thesis or peer-reviewed research article.
• ‘Direct/Indirect’ was scored as true (direct) or false (indirect) depending on whether the evidence directly tested a conservation action. An example of direct testing would be a Before-After-Controlled-Impact (BACI) approach, were the conservation action was implemented and its effect on seabirds was measured at the site. An indirect measurement would be a study which modelled and predicted the effect of a conservation action theoretically, without practically testing it.
• ‘Species’ scored highest where at least one of the focal seabird species were included then progressively lower for higher taxonomic levels.
• ‘Similar geographic location’ scored highest when the study was conducted in the focal region (i.e. Scotland) and then progressively lower for those more distant studies where the local ecological context is most likely to differ.
• ‘Publication date’. Studies conducted in recent years were given the highest weight to account for the ecological context likely to change over-time but also that more recent studies will typically have access to more advanced research methods.
• ‘Sample size’. This factor is multi-factorial incorporating several aspects of sample size, including the number of individuals/nests, number of sites, and number of study years. Guideline thresholds are given (Table 7) for threshold sample sizes, but the weighting of different components of sample size will vary depending on the type of study.

Table 7. Relevancy scoring components scored for each study. The scoring scale is qualitative. All factors are scored on the same scale (1–5) but some factors exclude some levels (indicated with a dash ‘—').

This table can be viewed in Excel format.

3.1.2.2.2 Overall ecological efficacy

As introduced above, ecological efficacy is a combination of the likelihood that a conservation action is beneficial to a seabird population, and the size of effect found across studies. Each study was scored against two factors:

1. Statistical inference: A qualitative scale from 1 (low) to 5 (high), considering the statistical confidence of a real effect, this considered the study sample sizes, any stated statistical confidence (inter alia any p-values or confidence intervals), and that there was adequate consideration of potential confounding variables or alternative explanations. For reviews this was scored as 1 for non-systematic reviews, 3 for systematic reviews, and 5 for meta-analysis studies.

2. Degree of effect: A qualitative scale from 0 (no effect) to 5 (a high beneficial effect). This is a relative scale with a high effect considered to be a large effect on a demographic rate (e.g. >10% change in annual mortality); a weak effect being where a small effect is found (e.g. <1% change in annual mortality). Note that, theoretically, the degree of effect could also have negative values, which would indicate a detrimental effect. However, given that we were looking into conservation actions that are expected to have some degree of benefit, we did not expect to find any negative values and, therefore, were not considered within our scale.

The ecological efficacy for each study was the product of statistical inference and degree of effect which could take any value from 0 to 25 (Table 8). Higher scores reflect studies that found a strong beneficial effect of the conservation action on seabirds with high statistical inference while lower scores reflect studies that did not find a significant effect of the conservation action, or those with low statistical inference. The overall ecological efficacy for an action was calculated as a weighted mean of these scores using the relevancy weighting.

Table 8. Scoring matrix for ecological efficacy combining statistical inference and degree of effect. Shading (low - grey, high - purple) indicate the overall level of ecological efficacy. In both cases, 0 = low values and 5 = high values.

This table can be viewed in Excel format.

3.1.2.2.3 Confidence

Confidence is an overall measure of the level of certainty we have that the assessed estimation of ecological efficacy is an accurate reflection of the true ecological efficacy of a conservation action. Confidence is not dependent on the level of ecological efficacy, i.e. we could find a strong effect but have low confidence that this is true, or we could find a low or no effect but have high confidence in this.

Table 9. Overall confidence for a conservation action was calculated as the sum of three factors, each scored on a qualitative scale of 1–5.

This table can be viewed in Excel format.

We calculated three components of confidence (Table 9):

1. Median of relevance score (see 3.1.2.2.1) as an index (i.e. range 0–1) across studies.

2. Independence and quantity of evidence – a qualitative measure of the amount of evidence available incorporating both the number of studies but also how independent those studies are (e.g. independence will be higher where there are more diverse authors/study systems/sites).

3. Concordance – Calculated by taking the standard deviation of the degree of effect measure (see 3.1.2.2.2); highest when a mix of positive and negative effects were found, indicating a low concordance across studies; lowest when all studies showed similar results/degrees of effect.

The overall confidence score was the sum of the three confidence components, leading to a range of possible values from 3–15.

3.1.2.2.4 Worked example

We show a simplified worked example with four theoretical studies (A–D).

First, all studies are scored against the six factors shown on Table 7 to determine their degree of relevancy (Table 10). Here, paper A represents a highly relevant study, while paper D represents the least relevant study, with papers B and C intermediate. Were all four papers weighted equally, then each would have a weight of ¼ (0.25), but as can be seen in Table 10, paper A has a weighting over four times greater than that of paper D.

Table 10. Worked example showing how the study weighting was calculated using the relevancy factors shown in Table 7. Weight = Sum of each study / total sum (70).

This table can be viewed in Excel format.

After scoring each study by their relevance, we proceed to score their statistical inference and degree of effect (Table 11). In this example, paper A had an excellent study design with a large sample size, over a long period, using data from several study systems, and provided relevant statistical analyses, good interpretation of the results, and accounted for confounding variables. Its results also found that the conservation action had a highly significant beneficial impact on the species and, therefore obtained the highest possible score of 25. On the contrary, paper D had a poor study design, with small sample sizes and absence of statistics, and did not find that the conservation action had an effect on seabirds, scoring a 1, the lowest possible value. Papers B and C had the same intermediate score, but for different reasons. On the one hand, paper B had a low statistical inference, with small samples sizes and poor statistics but its results indicated that the conservation action had a strong effect on seabirds. Paper C, on the other hand, had a good quality study design with appropriate statistics but its results indicated a weak effect on seabirds.

Once the ecological efficacy is obtained for each study (statistical inference * degree of effect), we proceed to obtain each studies’ final score (weight * ecological efficacy). Paper A, having both the highest ecological efficacy and relevancy scores, obtained a 10.75. Paper D, the least relevant and with the lowest ecological efficacy obtained 0.09. Papers B and C had intermediate scores. We then proceeded to sum all four scores to obtain an overall ecological efficacy for the conservation action, obtaining 14.8 (Table 11). In an ideal scenario, where all studies exhibit a perfect ecological efficacy and the highest degree of relevancy, we would expect an overall final score of 25 (i.e. the maximum possible score). Such a score would indicate that the conservation action has a highly significant effect on seabirds. In this theoretical example, the score of 14.8 indicates that the evidence suggests a medium effect of the conservation action on seabirds, though leaning towards a medium-high effect (Figure 5).

Table 11. Worked example showing ecological efficacy (statistical inference x degree of effect) and final scores (ecological efficacy x weight). Ecological scores could have a potential score between 0 and 25. The weight values used to calculate the final score (weight * ecological efficacy) come from Table 10.

This table can be viewed in Excel format.

Finally, we estimate a score to assess the level of certainty and quality of the overall evidence, using the factors stated on Table 9. Possible values range between 3 and 15. In this example, the median score of the studies’ relevancy score (using scores re-scaled to 0 – 1) was 0.46 and, following Table 9, scores a 3. Regarding independence and quantity, we will assume that the four theoretical studies shared common authors that gathered data from a limited number of study sites, scoring a 1. The standard deviation (SD) of the degree of effect across the four studies was 1.6, resulting in a cumulative score of 2. When considering the sum of these three scores, which totals 6, we conclude that the evidence indicates a low level of confidence on the evidence (Table 12).

Table 12. Assessment of the confidence in the evidence for the theoretical example. Refer to Table 9.

This table can be viewed in Excel format.

Therefore, the conclusion in this worked example is that, with a low degree of confidence (6/15), the presumed conservation action has a medium effect (14.8/25) on seabirds. Applying a consistent scoring system to each conservation action allows for a more objective and quantitative way of comparing across actions. This facilitates a better understanding of which action is more likely to provide stronger significant benefits for seabirds, while also considering the confidence level in the evidence that supported that conclusion.

3.1.3 Scoping and targeted reviews – general methods

For the proposed compensatory measures (Table 4) that could not readily be allocated to conservation actions, we undertook scoping reviews (see Ecological feasibility (WP1)). For longline bycatch mitigation, however, where recent comprehensive work exists, including a recent detailed review commissioned by the Scottish Government (Kingston et al., 2023), it did not make sense to duplicate efforts by undertaking a full systematic review. Instead, we summarised the findings of these studies through a targeted review.

The purpose of scoping reviews is to scope out the potential conservation actions that correspond to each of the proposed compensatory measures and, where possible, identify which of these have most potential, so warrant further consideration. For measures assessed through the scoping reviews, we did not proceed with the practical feasibility assessments. To do so comprehensively would have required to first select the most promising conservation actions identified following the review, and then to conduct systematic reviews to assess their ecological feasibility, which was not feasible within the project’s timeline. However, for bycatch mitigation a practical feasibility review was undertaken as significant evidence already exists indicating its ecological feasibility.

For both the scoping reviews and the targeted review we provide a high-level state-of-the-art review that considers key relevant recent papers and reports. Papers and reports are identified by a combination of existing knowledge, recommended reports (e.g. from the steering group), and targeted literature searches.

There is a dedicated section for each review, then at the end of this section a combined table summarising details of identified conservation actions. Each section starts with the focal species identified (see Table 4 and Table 6), followed by a general description of the key types of actions falling under each topic, identification of key sources, and discussion on potential conservation actions, followed by a brief conclusion.

3.2 Practical feasibility (WP2)

WP2 follows a similar structure to WP1, comprising of six chapters, each relating to a selected conservation action (except for the prey fisheries related actions that are grouped into a common chapter). Given the focus of this component we could not follow the systematic approach used in WP1, instead we used a combination of expert opinion, targeted literature, and web searches. References largely consisted of peer-reviewed scientific papers, commissioned reports, and government websites.

Each chapter includes the following sections, with additional sections where relevant for a conservation action:

• A summary table: Each chapter starts with a high-level summary table highlighting the most important and relevant information concerning the practical feasibility of the conservation action and includes the following:
  • A description of the conservation action.
  • A summary of key uncertainties on the effectiveness and practicalities.
  • An overview of the potential ecological benefits, focusing on species-specific population benefits, both for relevant focal species and other seabird species, occasionally extending to wider conservation benefits.
  • A summary of the spatial scale and the degree of population benefit that may result from the action. Spatial scale will vary from local (involving benefits at a colony level) to regional (involving several colonies within a geographical region), national (pertaining to a Scottish level), and wider (involving impacts on colonies outside of Scotland). The degree of the benefit is evaluated on a qualitative scale ranging from: none (indicating no expected benefit); low (representing negligible or small population-level response that would not be detectable though monitoring); medium (indicating measurable population-level responses, although this may require detailed monitoring to distinguish from environmental variations); and high (reflecting measurable population-level responses detectable through standard monitoring techniques, typically observable by ten years post-implementation).
  • The sequence of steps required to implement an action (i.e. a timeline).
  • Monitoring summary.
  • General key considerations, which outlines key issues that would need to be considered if seeking to progress an action as a compensatory measure.
  • Recommendation, providing a summary conclusion for each conservation action.

The degree of detail will depend on the available evidence and knowledge, with some sections excluded or additional sections included when relevant. Some chapters include multiple conservation actions, in these cases columns are used to summarise information specific to each conservation action.

• Background: This section includes a high-level overview of the key points obtained in WP1 and highlights the importance of the action for seabirds.
• Steps for implementation: This section outlines the steps required for the practical implementation of a conservation action, covering the information needed to decide whether to proceed with the action up until post-implementation considerations. This information will be obtained from previous studies, especially those conducted within Scotland and on focal species, as well as by consulting experts and relevant government guidance.
• Ecological effects of implementing action: For each focal species, we provide an overview of the anticipated ecological effects that will be expected from the implementation of the action. In certain cases, species will be grouped together (e.g. ground-nesting colonial species). Additionally, we highlight potential benefits to non-focal seabird species and other species that may benefit, as well as any significant broader ecological effects. The time lag for population-level responses is also discussed. This draws on key references identified within WP1 and additional relevant sources.
• Research and monitoring recommendations: This section offers recommendations for monitoring the effectiveness of the implementation, mostly identifying data to collect and associated analysis. Information was mainly gathered from analogous studies but also through discussions with individuals experienced in implementing similar actions.
• Key considerations, potential barriers, and potential solutions: This section provides an overview of key considerations for implementing an action. Potential barriers to implementing are identified and potential solutions are considered. The core issues considered encompass policy, legal, financial, ecological, non-financial resources, and practical and logistical aspects. We also consider other action-specific issues, such as cultural, health and safety, compliance, and additionality aspects. We gathered information from relevant studies and consulted with RSPB staff involved in practical conservation, policy, and casework (e.g. site managers, scientists, policy officers, engagement officers) to sense check the issues included and identified any additional barriers overlooked.
• Species-specific aspects of implementation: When relevant, we identified seabird species-specific aspects to consider during action implementation.
• Overall conclusion: This section includes an overall conclusion, overall uncertainty in delivery as compensation, and recommendations, including recommendations for further work.