Flood protection schemes - assessment of economic, environmental and social impacts: guidance

8. COMPARE: Choosing the option

Introduction

8.1 A practical way of ensuring best value for money, which may be applied on a project-by-project basis, is to maximise the cost-benefit ratio of those schemes while seeking to achieve an appropriate standard of protection for the type of land in question.

8.2 However, it is important that all impacts (both positive and negative) of an option are taken into account during decision-making. Use of the decision-making process requires these impacts to be considered to help ensure that the best overall solution is identified. It is therefore necessary to weigh up those impacts that have not been valued in monetary terms and consider whether they are of sufficient magnitude, significance and/or duration to change the preferred option from that which would be chosen based on the economic criteria alone. This is a decision that has to be made by the project team, based on the evidence collected and recorded in during the appraisal. All choices and decisions will need to be justified. Where the impacts are considered significant, but have not been monetised, the potential to use tools such as ecosystem services or scoring and weighting to monetise the impacts should be identified.

Standards

8.3 The Scottish Government does not specify design standards for flood protection schemes, although some guidance is given to local authorities on the consideration of design standards in "Delivering Sustainable Flood Risk Management (reference 1) ( see also paragraph 4.10).

The decision process

8.4 The decision for funding flood protection schemes lies with the Local Authorities' Accountable Officers. In making decisions about which flood protection scheme(s) to pursue, there are several criteria which should be borne in mind:

• The extent to which the objectives have been (or could be) achieved;
• Whole life costs of the options (taking account of contributions by subtracting funds from other sources);
• Qualitative and quantitative damages, damages avoided and benefits of each option;
• Average cost-benefit ratio (benefits divided by costs, where the costs include contributions);
• Incremental cost-benefit ratio (the difference between the benefits provided by two options divided by the difference in costs, where the costs include contributions);
• Net present value (benefits minus the costs, where the costs take account of contributions);
• Technical issues;
• Implications of sensitivity analysis.

8.5 It should be emphasised that the decision-making process is not about the manipulation of figures. Rather, what is required is a reasoned interpretation of the information produced from the appraisal.

Sensitivity analysis and robustness testing

8.6 Within economic appraisal, the purpose of sensitivity analysis and robustness testing is to determine whether, within the reasonable bounds of confidence, and for the various assumptions made:

• The scheme is economically worthwhile;
• The economic return is likely to be achieved;
• The option choice is robust.

8.7 It is therefore important to determine those factors that would impact on any investment decision. If, for example, the cost-benefit ratio is highest for an option where there is significant uncertainty, it may be better to pursue an alternative with a lower, but more certain outcome. Chapter 6 incorporates a framework for evaluating such risks.

8.8 Having determined the most important factors, assessments of uncertainty should be made on the basis of experience and judgment. As a general guide, a range of possibilities should be considered for items such as:

• Costs (whole-life capital, maintenance and management) based on the key costs elements and sensitivity to changes in costs of key materials or resources;
• Timing to first failure of deteriorating defences (projects are often sensitive to the timing of impacts, particularly write-off of properties or onset of environmental damages);
• Threshold of flooding (many schemes will be sensitive to assumptions about the level, and hence frequency, at which flood damage commences);
• Calculation of extremes and their probabilities;
• Changes to major beneficiaries (for example, a major business in the benefit area could cease trading or relocate, or where damages to a commercial property account for more than 20% (say) of the benefits, assess how the damages would change if the commercial property changed use such that the damages would be reduced);
• Regeneration potential and development planning.

8.9 Even where it is not possible to quantify the uncertainty associated with each variable, it should be possible to identify a reasonable range of possibilities. All major risks should be considered both singly and in combination.

8.10 The effect of uncertainty may be easier to explore where the benefits (and damages) are valued in money terms, since the impact of changes in assumptions between options can be calculated. Qualitative and quantitative descriptions of impacts should help identify which of the non-monetised impacts have the greatest influence on the choice of option. It is not always necessary to quantify the uncertainties, but it is useful to try and quantify the impacts on the benefits of the options and the average cost-benefit ratio wherever possible. This is particularly important where the scale of the uncertainty is much larger than tangible differences in value between the alternative options. Where it is not possible to quantify the uncertainty associated with each variable, it should be possible to assess the relative scales of the uncertainties compared with the other options. All major risks should be considered both singly and in combination.

8.11 It is important to focus on differences between options, as this will help identify which factors are influencing the choice of one option over another. This information can then be used to help choose between options and to justify the choice of preferred option.

8.12 For major projects it is particularly important to identify switching points where a change in the assumptions would alter the option choice. Informed judgments can then be made of the relative likelihood of the different outcomes, to determine the best option.

Post project evaluation

8.13 Post project evaluations demonstrate whether past investment has been worthwhile and has achieved its objectives. The exercise will be considerably eased if the fully documented original appraisal is readily available to evaluators. For example, it will be possible to compare actual performance against the sensitivity analysis in the cost-benefit appraisal. This will enable a check on the areas of greatest uncertainty, to see whether variations in practice are within predicted limits

8.14 In undertaking cost-benefit analysis as part of post project evaluation of flood protection schemes it can be difficult to identify realised benefits, which are based on the avoidance of losses which would have occurred had the scheme not been implemented. Hence, it is difficult to determine whether the actual benefits are equal to those predicted. It will usually, therefore, be necessary to judge success on the accuracy of related predictions such as the costs of construction and maintenance, rates of environmental enhancement or measures of residual damage.