Flood protection appraisals: guidance for SEPA and responsible authorities

5. Describe and value: appraising the options

5.1. Components for the appraisal

5.1.1. The options should go through a robust and transparent appraisal of costs and benefits, and positive and adverse impacts. The outputs should inform decision-making and provide evidence of a sound decision-making process.

5.1.2. The following components should be assessed:

• Estimates of flood risk management benefits (flood damages avoided) ( Section 6, 7, 8 and 9)
• Wider positive and adverse impacts ( Sections 7 and 8)
• Adaptability to climate change and other drivers of future flood risk ( Section 9.8)
• Whole life costs ( Section 10)
• Uncertainty in costs and benefits ( Sections 10.8, 11.6 and 11.7)

5.2. Appraisal techniques

5.2.1. The flood risk impacts and wider impacts should be described, and where appropriate/possible, assessed in quantitative or monetary terms. There are advantages to estimating the impacts in monetary terms: these impacts can be readily compared with whole life costs to estimate the likely return on investment by calculating net present values ( NPVs) and benefit-cost ratios ( BCRs) (Box 5.1).

5.2.2. Some impacts, however, can be difficult to value in monetary terms and/or may require disproportionate effort (Section 5.5). It is therefore crucial that significant impacts that are not valued in monetary terms are always described, quantified (if possible) and brought into the appraisal through appraisal summary tables (Section 5.6). Understanding these impacts is critical to selecting sustainable options and they should not be omitted because they are difficult to value.

5.2.3. There are a range of techniques that can be used to appraise and help select the preferred option. The technique chosen will depend on the complexity of the problem being addressed, the detail and confidence in the assessment of flood risk, and the availability of data and methods for valuing different impacts.

5.2.4. Where the obligations only involve requirements that must be met, the technique of cost-effectiveness analysis would usually be appropriate to identify the most effective and sustainable method of meeting these requirements (see HM Treasury Green Book (2011) for a worked example). However, any wider benefits associated with such projects should be explored to see whether there is a case for doing more than the minimum requirement. There are likely to be only limited cases when cost-effectiveness analysis is appropriate to use in flood risk management options appraisal: therefore no further guidance is given here.

5.2.5. Appraisal techniques such as benefit-cost analysis ( BCA) and multi-criteria analysis ( MCA) provide metrics for comparing options. The appraisal should stimulate the selection of the most sustainable solution by analysing the consequences of all options. It may also help to identify ineffective or uneconomic options at an early stage.

5.2.6. There will be cases where it is not practical or possible to assign monetary values to all significant impacts for a benefit-cost analysis. In such cases, multi-criteria approaches (including scoring and weighting) can be used to complement or as an alternative to benefit-cost analysis.

5.2.7. If using benefit-cost analysis and multi-criteria analysis together in appraisal, it is important to ensure that they are robustly and consistently applied in order to: avoid double counting, make appropriate and consistent use of discounting; and ensure a common baseline.

Box 5.1: Techniques for comparing options
Net present value Net present value is the difference between the present value of a stream of costs and a stream of benefits. It is used to identify the net benefits of an option to demonstrate whether the economic benefits outweigh the costs and the magnitude of these benefits.
Benefit-cost ratio Benefit-cost ratio (benefits divided by costs, where the costs include contributions) should be used to examine the relative return of an option for every pound spent. A benefit-cost analysis presumes that all components (costs, benefits, adverse impacts) of value are taken into account. Options are only economically worthwhile if the benefits exceed the costs (the ratio of benefits to costs is greater than one). Furthermore, if all significant impacts are satisfactorily expressed in monetary terms, the option with the highest benefit-cost ratio will usually be the most appropriate choice. Appraisal summary tables (Section 5.6) should still be used in such cases to add to the transparency of the decision-making process (for example, to illustrate which impacts have been taken into account and how they have been described and valued in the benefit-cost analysis). Appraisers should always give due consideration to any significant non-monetised impacts as these are important for identifying sustainable solutions. Options should not be rejected solely on the basis of economic criteria alone. For example, an option with a low benefit-cost ratio may be worth taking forward if it delivers strong non-monetary benefits.
Incremental benefit-cost ratio Incremental benefit-cost ratio (the difference between the benefits provided by two options divided by the difference in costs) can be used to identify the additional benefits delivered by one option compared to another. It enables an understanding the opportunity cost - that is, whether there is an extra pound of benefit for each additional pound of cost. For example, it can help examine whether the increased investment required to achieve a greater standard of protection is actually the most beneficial choice. There may be a justifiable case for selecting an option which would provide a higher standard of protection than that offered by the option with the highest benefit-cost ratio, providing that the overall benefit-cost ratio represents good value for money when compared with other options.
Multi-criteria analysis Multi-criteria analysis ( MCA) can be used to combine a mix of both monetary and non-monetary benefits. The extent to which each option meets the identified criteria is measured, and explicit weights are given to each of the criteria to reflect their relative importance. Using this technique, options can be ranked and a preferred option identified. Guidance on multi-criteria analysis forms part of HM Treasury Green Book supplementary guidance (Department for Communities and Local Government 2009).
Scoring and weighting Scoring and weighting (a type of multi-criteria analysis) can be used to estimate the monetary values of impacts where other approaches are not available or are not appropriate for the impacts predicted under the options being appraised. To use scoring and weighting, the impacts from at least one category (preferably several) need to have been quantified and valued in money terms. A scoring and weighting methodology is published by the Environment Agency (2010a).

5.3. Principles for valuing costs and benefits

5.3.1. Options appraisal should use economic values rather than financial values. Economic values take the view of the nation as a whole (one person's loss can be another person's gain) and exclude taxes on goods and services such as VAT. They are therefore are different to financial values, which take the view of the individual and include VAT as well as other taxes on goods and services. In economic valuation, goods that are damaged or lost by households and businesses are valued at their remaining value (typically at half-life) and not the replacement cost.

5.3.2. Principles of valuing costs and benefits are described in Appendix 1. Key points are:

• Inflation should be ignored in undertaking the analysis;
• Real prices (i.e. 'today's' prices) should be used for all streams of benefits and costs. (Values with different base dates will require adjusting);
• Any 'negative costs' should be regarded as benefits;
• Any 'negative benefits' should be regarded as costs;
• 'Transfer payments' should be excluded from the benefit-cost analysis;
• The discount rates recommended by the HM Treasury Green Book (2011) should be used for all streams of benefits and costs.

5.4. Setting boundaries for the appraisal

5.4.1. Appraisers need to determine the spatial extent of the appraisal and the appraisal period. These boundaries should be set taking into account the consequences of the options. It may not be possible to assess all such consequences and a reasoned decision must therefore be made as to how far the process should be pursued.

5.4.2. A conceptual model of the key flooding processes can help understand the interaction of different actions across catchments or coastlines and, in turn, determine the spatial extent of the appraisal. See SEPA (2015a) modelling guidance for further information on conceptual models and spatial extents.

5.4.3. Options may have an impact beyond the immediate area and, in addition, the factors outside the immediate area can influence the effectiveness of an option. For example, urbanisation of the upstream catchment could increase flood risk downstream. The assumptions made about external conditions should be realistic, not simply convenient. Any significant impacts beyond the project boundary should be included or the project boundary adjusted to incorporate these impacts.

5.4.4. The costs and benefits of all options should be evaluated over the same period, based on the life (with maintenance) of the longest-lived option under consideration. Any costs for maintenance, repair or replacement of shorter-lived components (required for the integrity of the wider solution until the end of its lifespan) should be factored in.

5.4.5. The presumption is that for most appraisals, a 100-year timeframe will be appropriate to enable comparison of options. Using a shorter timeframe (such as 50 years) would mean that costs or benefits that occur after 50 years are not taken into account and this could affect which option is identified as preferred. For example, short time horizons can bias the appraisal against options that cost a lot now but which are less expensive to maintain, or which provide significant benefits and/or may be more sustainable over a longer timeframe. Longer timeframes also better allow for environmental or adaptation benefits to be included in appraisals.

5.5. A proportionate approach

5.5.1. The level of detail in appraisal should be proportional to the stage of appraisal (be it strategic, feasibility or design) and to the level of detail needed to differentiate between options. For flood risk management options involving a small amount of expenditure, a detailed benefit-cost analysis may not be justified.

5.5.2. At the early stages of appraisal, readily available data are usually sufficient. As the appraisal proceeds, data are usually refined to become more specific and accurate as needed to support decision-making. It is important that the effort applied at each step is proportionate to the time and resources available and that additional data are only collected where required.

5.5.3. Less detail is needed where the choices among options are clear, whereas more detail is likely to be required where there are complex issues and where differentiation is more difficult. A simplified analysis, where there are valid grounds for such an approach, should still be rigorous.

5.5.4. Appraisers must therefore decide on the appropriate level of detail and the streams of benefits and costs to be included. One of the skills needed for a good options appraisal is deciding when enough information has been collected to make a robust and defensible decision. This is usually where collecting more information will not make a significant difference to the decision. It is essential to demonstrate this clearly and openly to those that may be affected by the decision.

5.5.5. Data collection and modelling can be one of the most expensive parts of a benefit-cost analysis. Appraisals should seek to build on existing modelling work, for example, SEPA's strategic flood hazard and risk modelling, Scottish Water's assessment of flood risk from sewerage systems (carried out under Section 16 of the Flood Risk Management (Scotland) Act 2009), and modelling and surveys held by local authorities. Where existing modelling work (or surveys) are to be used for a new appraisal, appraisers must ensure that: (i) the scale and purpose of the modelling is appropriate for the new appraisal; and (ii) changes in the catchment and available data would not affect the results of the appraisal. Any new modelling work should be carried out to a suitable level of accuracy for the type of appraisal. See SEPA (2015a) modelling guidance for further guidance on the use of existing models and development of new models.

5.5.6. For feasibility and design stages, the appraisal should be sufficiently extensive to show with reasonable confidence whether it is worth adopting any of the 'do something' options. Ideally, the first streams of benefits and costs to be included should be those contributing the greatest proportion of the total. While it is not always possible to determine in advance which will be the largest, there are two areas which should generally be considered first; these are:

• Benefits and costs that have the highest probability of occurrence;
• Benefits and costs that accrue earliest in the timeframe of the appraisal.

5.5.7. The cost of carrying out the appraisal will vary depending on the types of impacts that are appraised and the precision required. For recreational benefits, it may cost as much to appraise these for a small scheme (or area) as a large one. The assessment cost also depends on the extent to which available data sources can be used, such as depth-damage data for residential (and some non-residential) properties ( Section 6.2). Furthermore, if the flood mechanisms and topography are complex, appraisal costs will increase.

5.5.8. Further guidance is provided in Box 5.2.

Box 5.2: How to determine a proportionate approach: additional considerations
How uncertain are estimates of flood hazard and flood risk? Flood models use simplifications and assumptions to represent complex natural systems and this leads to inherent uncertainty, which must be acknowledged when making decisions based on model results. SEPA (2015a) provides guidance on where uncertainty may arise in flood modelling and how it may be managed through the modelling process so that it can inform appropriate decisions. Understanding the uncertainty in flood modelling is crucial to ensuring a proportionate and robust appraisal. There is little advantage in spending time looking at flood damages in detail if the flood models that have predicted those impacts are coarse. Furthermore, the data used to estimate damages should be appropriate given the type and tolerance of the flood model. For example, depth-dependent property damage data ( Section 6.2) are not appropriate to use if flood depth estimates are highly uncertain. (Weighted damages, which do not require depth information, are more appropriate to use in these situations. See the MCM for more detail.) There will also be uncertainties in the receptor data used to estimate flood damages. The MCM contains some health warning for particular data and advises where sensitivity analysis ( Section 11.7) might be particularly important.
Are the impacts significant? If the economic impact on a receptor is likely to be around 10% or more of property damages (which can be estimated reasonably quickly), then it is worth spending time to assess the impact in more detail. It is also often worth considering non-residential property damages in detail as they can account for a large proportion of damages, even if they make up a relatively small proportion of affected properties. This is because standard depth and damage information for non-residential properties can often significantly over or under estimate damages, due to the highly individualised nature of these types of properties and their exposure to flooding.
Do the impacts differ across the options being appraised? Where impacts are very similar across all or most options, there is unlikely to be much value in considering these impacts in a lot of detail. Instead, the effort should focus on assessing those impacts that vary across options to enable decision-makers to differentiate between options.
How much time is required to describe, quantify or value the impacts? It can take a considerable amount of time to estimate the monetary damages of some impacts. Where those impacts are relatively small, it is essential to consider how significant the impacts are likely to be before starting to collect data or estimate damages. Those impacts that may require disproportionate effort to monetise should be described, and quantified if appropriate. If the choice of preferred option turns out to be reliant on differences between options in any one category, it may be possible to monetise these impacts later in the appraisal process to bring them into the benefit-cost analysis. The most likely significant impacts are usually the drivers for action (often damages to residential and non-residential properties or damages to critical infrastructure) and these should be identified at the outset. If a robust benefit-cost ratio is achieved on the basis of protection to properties alone, in many cases there may be little advantage to trying to estimate the monetary benefits to transport or of indirect impacts, given their complexity. Instead, these impacts should just be described.
Are suitable approaches available to value the impacts? Some significant impacts, particularly related to social and environmental criteria, may be difficult to value in monetary terms. Where valuing these impacts would incur disproportionate time or costs, or where techniques are not readily available, these impacts must at the least be described, and if appropriate, quantified. These impacts must still be taken into account during decision-making.

5.6. Appraisal summary tables

5.6.1. Appraisal summary tables should be used as a framework for systematically describing, valuing and, where possible, monetising the positive and negative impacts of options. The tables should provide a comprehensive assessment of the impacts of all options. They should also make transparent which impacts have been valued in monetary terms (and how these monetary values have been developed) and which have not. Assumptions should be clearly stated and any uncertainties associated with the description, quantification and valuation of impacts should also be recorded. For an example, see templates developed by the Environment Agency (2010b).