External Review of Alcohol Focus Scotland

7 Quality assurance

7.1 This review identified a current commitment within AFS to continuous improvement. The appointment of the new Chief Executive has led to positive developments in strategic planning, finance, HR and internal / external monitoring and evaluation.

Vision and leadership

7.2 AFS has a positive history of piloting and evaluating new projects and initiatives, and of the findings informing future development and wider roll-out of these initiatives. In addition, there is evidence that training is routinely evaluated, although the evaluation of training has been largely at a basic level (participant reaction and self-reported change in participant knowledge). The exception to this general rule was in relation to the follow-up evaluation of Servewise training, which attempted to gather information about behaviour change in license-holders.

7.3 While the general impression is that AFS has always recognised the importance of evaluation, it is only more recently that there has been a concerted effort to develop a more comprehensive, rather than piece-meal, monitoring and evaluation framework.

7.4 AFS now has logic models in place, and has identified expected outcomes for all of its existing programmes of work. It is expected that when funding becomes available to being work on the area of Protecting Young people from Alcohol Marketing, that a logic model will be developed in this area as well.

7.5 At the time of this review, there was no monitoring and evaluation framework in place, but as mentioned previously, one was being developed in collaboration with Evaluation Support Scotland, and was expected to be available in March 2012. It was reported that staff at all levels of the organisation had been involved in the development of this framework, and would in the future be involved in the collection of monitoring data once the appropriate indicators had been agreed.

Effective partnerships

7.6 There was some evidence of the use of benchmarking within AFS. For example, it was reported that staff salaries had been benchmarked against those of similar organisations. In addition, statements recorded in the minutes of meetings of the Executive Committee, senior managers and other staff indicated that benchmarking had been carried out (or requested) in relation to performance management procedures, ratios of non-admin to admin staff, and financial matters.

7.7 Benchmarking and learning from peers is a tried and tested means of supporting self-assessment, potentially forging mutually beneficial alliances, and quickening improvement activity. It is suggested that AFS should consider the development of a more strategic approach along these lines.

Governance and accountability

7.8 There is evidence from interviews and from the minutes of meetings of the Executive Committee, senior management and staff, that AFS is transparent in its activities and decisions. Executive Committee members interviewed for this review expressed satisfaction with reports received from the organisation and noted that these had improved considerably since the new Chief Executive came into post.

7.9 It was reported that there had been concerns voiced among some in the Executive Committee about AFS's previous links to the alcohol industry. The decision to withdraw from an endowment funded by the industry, and to not accept funding from the industry in the future was seen to be the right decision for the organisation.

Performance management

7.10 AFS has clearly taken steps to develop more of an outcomes focus across all areas of its work. The completion of a monitoring and evaluation framework (mentioned above) will need to be linked to ongoing strategic planning and workforce planning.