Digital appraisal manual for Scotland: guidance

Introduction and key concepts

Digital is an emerging area of increasing importance, with significant investment likely to take place in the immediate and more distant future in Scotland. It is therefore essential we have a dedicated, clear framework to assess value for money of investments in digital in a robust, evidence-based and objective-led way, that can be consistently applied across business cases.

Given this, OCEA has developed the following Digital appraisal manual for Scotland (DAMS). The objective of the DAMS is to provide policy makers and stakeholders working on Scottish-based digital projects access to the latest information and best practice guidance that they will need when developing digital schemes, strategies and business cases. Appraisal and evaluation provides objective analysis to support decision making for the delivery of policy objectives and should be an integral part of all proposed interventions.

Key concepts

• appraisal and evaluation are often used interchangeably but mean different things in this context. Appraisal is the ‘ex-ante’ process of deciding how well a strategy or project will perform and checking that value for money will be achieved. Evaluation is the specific application of appraisal to the ‘ex-post’ assessment of completed projects – i.e. retrospectively considering the performance of a strategy or project
• ultimately, the aim of any business case is to assess the extent to which an intervention meets pre-set objectives, and delivers value for money. With any public investment, we must be able to clearly assess and demonstrate value for money. It should be emphasised that value is not always solely or primarily demonstrated or evidenced by monetised benefits – ‘softer’ benefits can be at the core of a business case. This appraisal guidance will allow us to better and more consistently assess and demonstrate value for money, in a way that links outcomes to strategies (including both the Digital Strategy for Scotland and overarching purpose and strategy)
• strategic business cases (DAMS reports) created using this guidance will allow the rationale behind potential digital interventions to be presented in an evidence led manner. Decision makers can use this to make more informed choices to meet the digital needs of Scotland
• the principle of being objective-led, rather than solution-led allows the appraisal of proposed options for intervention against objectives, DAMS criteria and established policy directives
• the appraisal and evaluation process should be applied proportionately but comprehensively. The scale and scope of the appraisal should be consistent with the proposed budget, resource and potential impacts of the intervention
• the appraisal process is a collaborative effort and should draw on stakeholders as well as specialists such as analysts, finance business partners and commercial experts
• the appraisal of a proposed digital intervention should be considered and divided across five dimensions: strategic, cocio-economic, financial, commercial and management
• the evidence base for impacts of digital projects (everything from economic impacts of improved connectivity, positive impacts upon health and wellbeing, reducing the regional digital divide, to environmental impacts and impacts on communities) is underdeveloped in many cases – the best way to develop an evidence base specific to Scotland is to implement rigorous monitoring and evaluation of interventions in order to fully understand impacts
• consistent and transparent appraisal and evaluation allows for a more defensible and coherent position regarding expenditure. It can also identify lessons to be learnt to improve the design and delivery of future digital interventions in Scotland

The Digital appraisal manual for Scotland (DAMS) will generally follow the high level policy development and review process known as the ROAMEF (rationale, objectives, appraisal, monitoring, evaluation and feedback) cycle as shown in Figure one. The pre-appraisal preparation involves developing a rationale and setting out objectives to support the appraisal of the proposal. Post-appraisal is monitoring and evaluation of the intervention and, lastly, feedback. All of these stages feed into the development of the business case.

The ROAMEF policy development cycle

• Rationale
• Objectives
• Appraisal
• Monitoring
• Evaluation
• Feedback

This cycle is continuous – it is essential that the lessons learned from utilising this policy development cycle feedback and inform future policy development.

This appraisal guidance will help produce a DAMS report that sets out the strategic business case for any potential digital investments. A business case provides the documentation with the reasoning and justification of a scheme. The DAMS report will include the rationale behind a potential intervention presented in a clear, evidence-led, manner. Such presentation provides the information required by a decision maker to make informed choices.

Best practice:

• start early – not as an afterthought once intervention under way
• appraisal should be proportionate to scale of intervention and expected impact
• plan and pre-budget for post-appraisal monitoring, evaluation and feedback
• reference data appropriately (source of evidence, explanation of robustness of evidence, relevance of evidence to the context in which it is being used)
• refer to lessons learned from previous experience and evaluation evidence
• refer to appraisal and DAMS report throughout intervention when in progress