Social security agency: central functions location analysis

Annex C – Multi-Criteria Analysis Guidance

Different methods can be employed to compare options to aid decision-making and provide an audit trail and transparency about decisions at each stage of the process. The most commonly used ones (as set out in HM Treasury Green Book guidance) are Cost-Benefit Analysis ( CBA), Cost Effectiveness Analysis ( CEA) and Multi-Criteria Analysis ( MCA). MCA refers to a set of techniques for comparing options where assigning monetary values to costs and benefits is impractical. Figure C1 shows where MCAs fit in a standard appraisal process and more detail is provided in Green Book guidance. ^[13]

Figure C1 - Standard option appraisal process with Multi-Criteria Analysis as a decision-making tool

A key feature of a MCA is a performance matrix which plots the performance of options against set criteria. A basic matrix may only contain qualitative information whilst more sophisticated analysis can convert these into numerical values.

Setting the criteria in a Multi-Criteria Analysis process – key steps

Establishing the decision context - It is common practice to identify high-level objectives for which there will usually be sub-objectives.

Identifying options – to keep the list to a manageable number, shortlisting may be done prior to formal MCA being undertaken.

Identifying criteria and sub-criteria - Criteria can be grouped into sets that relate to separate components of the overall objective. Criteria need to possess certain qualities, e.g. be operational (it should be possible for performance against them to be differentiated between options) and complete (capture all or most aspects of the objectives).

Assessment of options against criteria in a Multi-Criteria Analysis process

MCA analyses fall into two broad categories - those without scoring and those with scoring.

MCA without scores is appropriate when information does not translate easily into scores or if scores take away the detail and make the analysis less rather than more robust.

MCA using scores can be done using a variety of techniques. Direct scoring involves simply assigning values to criteria based on evidence or expert advice. Value functions define the relationship between the actual values of a criterion ( e.g. jobs, distance) and a scale that allows a score to be attached ( e.g. 0 to 100). Pairwise comparisons involve comparing two options at a time against each criterion.

Ranking options in a Multi-Criteria Analysis process with scores

To aid decision making, rankings can be derived based on aggregate scores and individual criteria. If information is available on the relative importance or preference between different criteria, specific weighting can be assigned to each criterion which can influence the ranking. Information on weights may be implicit in the objectives. The weights should reflect the range of difference between the options and the importance of the difference.

Because of uncertainty of judgement around weighting, sensitivity analysis is often performed which involves shifting weights to examine any changes in the ranking of options.