External Review of ASH Scotland 2014

7 Performance Management Framework

Performance management within ASH Scotland

7.1 ASH Scotland has a comprehensive and detailed performance management and reporting framework which is reported on at each quarterly Board meeting. However, there is scope to clarify and improve the thinking underpinning the framework which was originally implemented in 2010, and sets out a series of five strategic outcomes, each with a set of up to 11 high level outcomes. The strategic outcomes reports provide information on ASH Scotland's contribution, the indicators used, and progress reports for each high level indicator. These outcomes are clearly relevant to ASH Scotland's strategic aims; however, the current detailed reporting framework on activity does not allow an assessment of overall performance.

7.2 The report would be more useful for the Board if it can indicate whether activities are on track, or where there is variation, and set out the reasons why. Setting targets, and using a simple system to provide a visual overview such as a traffic light (red, amber, green assessments of performance) system may help ASH Scotland assess how well it is progressing towards outcomes, using the outputs or milestones as proxy indicators. It would also be beneficial to discuss with the Board members what they require from performance reporting.

7.3 Performance management frameworks can be strengthened by the clarity of their underlying structure and its relationship to organisational strategy. ASH Scotland developed logic models for its work in 2011 with support from Evaluation Support Scotland. Logic models can provide a powerful tool for clarifying thinking about contextual drivers, inputs, outputs, and medium and longer term outcomes. Their value often lies in helping identify the links between investment (inputs) in activities (outputs) and their likely impact (outcomes). With the advent of a new strategic plan, there is an opportunity to review the existing models to test how planned activities will contribute to delivering outcomes.^[10]

7.4 The ASH Scotland leadership is already planning to increase the focus on outcomes in the 2015-18 strategy. There is scope for an approach with fewer outcomes to clarify and strengthen the organisation's strategic direction, and inform the development of a more concise, outcome focused performance reporting framework. This will enable ASH Scotland better to assess the impact of its own work and to provide better accountability to the Scottish Government, other funders and wider stakeholders.

Reporting to the Scottish Government

7.5 In addition to its own internal performance reporting, ASH Scotland also reports on its core grant funded and project activities to the Scottish Government on a six monthly basis. The core grant reports use a format which reports on 22 key objectives required by the Scottish Government. Each objective has a number of associated actions, so that this report is also very detailed. The report uses outputs as measures of achievement, and sets clear timescales for activities, and 'outcomes' are set for each objective. These outcomes are different to those used by ASH Scotland in its own performance management system, and some may be better described as 'outputs'. As used, the reports to the Scottish Government provide a wealth of information on the many activities achieved by ASH Scotland. The use of outputs rather than outcomes, however, limits their strategic value.

7.6 Reports to the Scottish Government on the specific projects (e.g. on inequalities) report on activities in relation to output measures. Each task is related to the Scottish Government Tobacco Strategy theme to which it contributes, for example cessation, protection, and inequalities. These reports appear more similar to quasi contractual specifications for time limited work, and give appropriate levels of detail for this purpose. This format, however, does not enable the Scottish Government easily to assess ASH Scotland's contribution to delivery of the national Tobacco Control Strategy.

7.7 ASH Scotland are concerned to provide the Scottish Government with the requisite information in regular reporting but a lack of clarity over what is needed has challenged them. During the course of this review, the Scottish Government indicated in discussion a preference in the future for more outcomes-based grant conditions than in the past and, therefore, also a preference for reporting which focuses on showing how activities have contributed to those outcomes as well as providing appropriate information to ensure full accountability. The current situation, as ASH Scotland develops a new three-year strategy, offers an opportunity to reconsider reporting formats and arrangements and for the Scottish Government to define more clearly what it requires in terms of reporting for the remainder of the grant period. A joint review by the Scottish Government and ASH Scotland would help identify how current formats could be improved to achieve better focused reporting which would aid future monitoring of how the organisation delivers value for money.