Appraisal and evaluation
Scope
1. This section provides guidance on appraisal and evaluation techniques and procedures. The guidance is relevant to all organisations to which the Scottish Public Finance Manual (SPFM) is directly applicable, including bodies sponsored by the Scottish Government (SG).
Key points
2. Appraisal and evaluation are essential parts of good financial management. The general principles should apply to any proposal - whether project, programme or policy related - with implications for expenditure / use of resources.
3. The effort that should go into appraisals and evaluations and the detail to be considered is a matter of judgement.
4. Appraisal within the SG involves the preparation of Pre-Expenditure Assessments (PEAs). PEAs must be undertaken for any proposal with significant resource implications.
Background
5. Appraisal and evaluation are essential parts of good financial management. The general principles should apply to any proposal - whether project, programme or policy related - with implications for expenditure / use of resources. The effort that should go into them and the detail to be considered however is a matter of judgement. For example, proposals involving modest expenditure / use of resources may merit less detailed appraisal and evaluation. Good appraisal entails being clear about objectives, thinking about alternative ways of meeting them, estimating and presenting the costs and benefits of each potentially worthwhile option, and taking full account of risks - see Risk Management. Good evaluation after the event entails many of the same processes, together with a desire and willingness to look for better ways of doing things. A checklist of the main points that should be covered in appraisal and evaluation is provided in Annex 1.
6. The Green Book, Appraisal and Evaluation in Central Government published by HM Treasury provides more detailed guidance on the principles, techniques and procedures applicable in relation to appraisal and evaluation. The guidance in the Green Book has been adopted by the SG and applies to all organisations to which the SPFM is directly applicable. Specialist advisers (eg the SG's Analytical Services Divisions) should be consulted on the application of relevant guidance and the preparation of appraisals.
7. The guidance in this section forms part of the Client Guide to Construction Projects published by the SG. The guide provides a policy and procedural framework for construction works projects and also applies to all organisations to which the SPFM is directly applicable.
Appraisal
8. Appraisal is normally the starting point for any proposal - but it is not a standard procedure. It is an ordered, but flexible, general approach to the analysis of proposals with implications for expenditure / use of resources. It should include not only economic analysis, but also other important information such as financing and/or human rights implications, arrangements for project management and plans for subsequent monitoring and evaluation. The principle of proportionality should be applied, so the scale of the appraisal will vary depending on the scale of the proposed project and its complexity, but an appraisal should normally include the following steps:
- define the objectives;
- consider a range of options;
- identify, quantify and value the costs, benefits, risks and uncertainties associated with each option, including considerations of public private partnerships and the scope for shared services arrangements with other public bodies, optimism bias and distributional implications;
- analyse the information;
- decide what evaluation should be performed at a later stage; and
present the results.
More details on each step are provided in the Annex 1.
9. The analysis will not always point to a clear-cut recommendation. There may be risks and uncertainties attached to costs, benefits or both. There may be significant elements that cannot be easily quantified in monetary terms. For each option, the impact of all relevant factors and related risks and uncertainties should be set out systematically and an assessment made of where the balance of advantage lies. The Green Book gives more detailed guidance and points to other sources which can help, for example, to deal with risk and uncertainty, and with costs and benefits not easily valued, such as environmental effects.
Pre-expenditure assessments
10. Appraisal within the SG involves the preparation of Pre-Expenditure Assessments (PEAs). PEAs must be undertaken for any proposal with significant resource implications. Guidance in relation to PEAs is at Annex 2. The process involves assessing:
- the aims and objectives of the proposal;
- the options for addressing these objectives;
- the evidence base on the likely economic, social, and environmental impacts and value for money of the proposal, including cost-benefit analyses where appropriate;
- the financial and management arrangements for the proposal, including an assessment of the key risks to successful delivery;
- the plans for monitoring and evaluation.
11. Other organisations subject to the requirements of the SPFM may wish to adopt similar procedures. Further details on the requirements for PEAs are available from the SG's Analytical Services Divisions.
Evaluation
12. Evaluation examines the outturn of a project, programme or policy against its objectives. It adds value by providing lessons from experience to help future management or development of a specific project, programme or policy. Evaluation should be planned from the outset of the project, and should normally include the following steps:
- establish exactly what is to be evaluated and how past outturns can be measured;
- choose alternative states of the world and/or alternative management decisions as counterfactuals;
- compare the actual outturn with the target outturn, and with the effects of the chosen alternative states of the world and/or management decisions;
- draw up the results and recommendations; and
- disseminate and use the results and recommendations.
More details on each step are provided in the Annex 1.
Page updated: July 2021