Scottish Marine and Freshwater Science Volume 5 Number 13: Population consequences of displacement from proposed offshore wind energy developments for seabirds breeding at Scottish SPAs

Appendix E. Quantification of uncertainty within exploratory runs

E.1. Quantifying uncertainty

The model was run five separate times for each species-by-scenario-by- SPA combinations, with each of the runs being generated using a common set of parameters. The variation between these runs tells us somewhat about the stochastic noise that is likely to be associated with the output from any single run. Note, however, that five runs are insufficient to provide a reliable estimate for the magnitude of the uncertainty that is associated within this noise.

When looking at chick survival, we summarise uncertainty by looking at the standard deviation in an overall summary (the proportion of chicks that survive for the duration of the breeding season) between the five model runs. Under an assumption of normality we also calculate crude estimates for the probabilities associated with exceeding particular thresholds. The key limitation of this approach is the number of runs: the very small number of runs (five) means that the mean and standard deviation of the wind farm effect may not be estimated reliably, and forces us to make a (potentially incorrect) assumption of normality in order to be able to estimate threshold probabilities.

When looking at adult survival, we focus on two sources of uncertainty: stochastic variation between model runs, and uncertainty associated with the magnitude of the published mass-survival relationship. Each run of the foraging model provides adult mass values for every individual within the simulated population at the end of the breeding season. These values can then be converted into adult survival values using the adult-survival relationships given in Section 2.4. The methodology within that section assumed that we used a value of b that was exactly equal to the value given in Oro et al. (2002) or Erikstad et al. (2009), but we actually go further - we account for the uncertainty in b by simulating 1000 values from a Normal distribution with a mean equal to the estimate of b given in the literature and a standard deviation equal to 0.359 times the mean. The ratio of the standard deviation to the mean (0.359) is based on the value obtained by Erikstad et al. (2009); this value is also applied to kittiwakes because the corresponding value within the Oro et al. (2002) paper (0.027) appeared to suggest an unrealistically low level of uncertainty. Each of these 1000 values is used to estimate an overall survival rate. For each species-by-scenario-by- SPA combination we therefore have 5000 simulated values for the overall adult survival rate (1000 for each of the 5 simulation runs). We assume that these values can be used to represent the uncertainty within the adult survival rate. We do not assume that these are normally distributed, but instead calculate probabilities and intervals directly from the simulated values ( e.g. by estimating the probability of the impact exceeding 4% to be the proportion of simulated values for which the impact exceeds 4%). Note that the values of b are paired: the same value is used to calculate the uncertainty in the baseline and in the runs that include wind farms. This pairing reduces the uncertainty associated with the impact of the wind farm.

E.2. Presenting uncertainty

Assume that we are interested in a specific question: for example, what is the impact of all four wind farms upon guillemot adult survival in the Forth Islands SPA under a 'moderate' prey scenario, a 1km buffer, and 100-100% levels for displacement and barrier effects?

Our exploratory results provided the 'best estimate' for the magnitude of this impact (this is, in technical terms, the mean), but also provided information on the uncertainty associated with this. The raw results of uncertainty tell us, in effect, the probability that the actual impact would be greater than every possible threshold - the probability that the impact will be more than 0%, more than 0.1%,… etc. We summarise these raw results in two distinct, but closely related, ways:

1) we calculate the probabilities associated the impact exceeding a small number of fixed thresholds: 0%, 1%, 2%, 3%, 4%, 5%, 7.5% and 10%;

2) we calculate intervals that will contain the 'true' impact with a particular probability: e.g. the 50% interval (there is a 25% probability that the true impact will be lower than the bottom end of this interval and a 25% probability that the true impact will be higher than the upper end), the 33% interval, and the 95% interval.

Note that the two forms of summary come from the same underlying information, so they are, by definition, consistent with each other - they simply focus on summarizing the same information in two rather different ways.

In terms of the terminology for hypothesis testing, the impact of the wind farm would be classified as significant if the 95% interval contains only non-zero values and as non-significant if the 95% interval contains zero. Hypothesis testing is not necessarily a particularly useful concept in the context of decision making, however, and in this context the use of significance as a threshold for action would correspond to a highly anti-precautionary approach: it would imply that a negative impact should be considered to be problematic only if could be identified with virtual certainty.

The values that we produce may be related to the terminology produced by the IPCC (2010) working group. In particular, the probabilities of exceeding particular thresholds may be converted into textual descriptions using the following table (taken from Table 1 of the IPCC report):

"Very likely" (probability of greater than 90%)

"Likely" (probability of 67-90%)

"As likely as not" (probability of between 33% and 67%)

"Unlikely" (probability of between 10% and 33%)

"Very unlikely" (probability of less than 10%).

The original table also includes categories for 'exceptionally unlikely' and 'virtually certain', but we have not included these because we do not feel that our uncertainty assessment is sufficiently precise to be able to meaningfully assign very small probabilities to events.

E.3. Sources of uncertainty

It is important to understand that there are some sources of uncertainty that we explicitly quantified in the exploratory analyzes, and that these are the sources that are summarized using probabilities; however, there are also sources of uncertainty that we cannot explicitly quantify. For the exploratory analyzes we explicitly quantified two sources of uncertainty:

1) natural variations in behaviour between populations of birds; and

2) uncertainty about the magnitude of the relationship between mass and adult survival.

Our assessments of uncertainty - and the probabilities that they produce - include both of these sources. There are other sources of potential uncertainty that we are unable to quantify, because we either do not have any information on them or else do not have enough information to be able to meaningfully quantify uncertainty. These include the location of bird foraging areas, the extent to which adult birds prioritise their own survival over chick survival, and the magnitude of the reduction in chick mass that would lead to death. Finally, there are sources of uncertainty which we did not seek to quantify. These mainly include sources of uncertainty that are explicitly included with the scenarios that we consider - e.g. the size and locations of wind farms, the link between prey and seabird distribution, baseline survival rates, total amounts of prey, and the rates at which displacement and barrier effects occur. These sources are dealt with in a qualitative way, through the comparison of different scenarios. Another source of uncertainty that we do not wish to consider is variability between individual birds within a population: we are interested in the effects on the overall population, and inter-individual variability is averaged out in the process of estimating this.

E.4. Reducing uncertainty

The uncertainty associated with the strength of the mass-survival relationship reflects the current state of scientific knowledge in this area. This uncertainty could only be reduced if a new, relevant, study were to be published on this topic using a larger sample size.

The uncertainty regarding natural variation between populations is rather different. Ideally, each of our simulation runs from the foraging model would have contained a number of birds equal to the size of the actual population for the SPAs being considered. If that have been the case then our assessments of uncertainty would have related directly to the uncertainty associated with the impact of wind farms on the entire population of birds within the SPAs at risk - this uncertainty could not then have been reduced any further, since it would reflect genuine variation between populations of birds.

In reality, our exploratory models runs were based on a relatively small proportion of the overall population (between 1% and 5% of the population, depending on species). This was unavoidable, due to the time constraints of the project and the computationally intensive nature of the foraging model, and ensured that all of the relevant scenarios could be run through the model within a reasonable timeframe. It does mean, however, that these exploratory runs will tend to have substantially over-estimated uncertainty. We considered two options for further modelling:

a) including a larger proportion of the population in each simulation run; or

b) running more simulation runs.

The latter approach would improve the quality (accuracy) of our assessment of uncertainty, but would not actually systematically reduce uncertainty. The former approach would systematically reduce uncertainty - as the number of birds per run increased, the uncertainty would reduce. Naïve calculations suggest that variability ( e.g. standard deviations) would be reduced by between 75% and 90% for chick survival if we were to run the simulations using the entire population rather than the subsamples that are currently used. The gain in precision would be less for adult survival, because in this case a proportion of the uncertainty represents the uncertainty associated with the mass-survival relationship (which would not be reduced by including more birds in each simulation run). These arguments led us to base the final set of model runs on a much larger proportion of the population than that use in the exploratory runs.