The Crerar Review: the report of the independent review of regulation, audit, inspection and complaints handling of public services in Scotland

08 WHAT NOW FOR EXTERNAL SCRUTINY?

8.1 Until now external scrutiny has not been approached in a strategic or systematic way. This in part reflects the range of external scrutiny functions and governance arrangements that have developed over time. Given a new core purpose to provide independent assurance within a wider performance management and reporting framework, the various functions of external scrutiny and the organisations responsible for carrying them out should now operate as a system and the different elements need to operate effectively together. We believe this will require a much greater degree of leadership and directed co-ordination than at present, and a much greater focus on performance management and associated self-assessment by service providers.

8.2 The main thrust of our conclusions is that, if the workload created by scrutiny is to be reduced, Ministers and the Parliament will need to 'let go'. This will require a new approach to the identification of and response to risk. Our findings point to a need for greater co-operation between the Parliament and Ministers and between both of these bodies and service providers. To devise a framework for the future, there needs to be a more flexible approach to dealing with concerns about public service delivery, including establishing a clear role for external scrutiny.

8.3 The public services delivery environment is evolving and will continue to change. Government and the public will need independent assurance about different priorities and different elements of service provision at different points in time. The scrutiny system needs to become flexible enough to respond to such changes, without requiring new organisations or regimes each time. The key elements of a revised system will include:

• Clear strategic priorities for external scrutiny;
• A central role for self-assessment by providers as the main source of information about compliance and performance;
• A clear system for governing the application of external scrutiny; and
• Clear accountability and governance arrangements.

8.4 Throughout this chapter we refer to changes as Short-term - 12 months; Medium-term - 2 years; Long-term - 4 years. The timescales are challenging and meeting them will depend on the commitment of the key stakeholders but we believe that work on all of the recommendations should begin immediately.

Strategic priorities

8.5 The external scrutiny system needs to have clear priorities enabling strategic deployment of scrutiny resources. The priorities should reflect where there is most need for assurance about public services in the public interest. Focusing external scrutiny outputs in a way that is more meaningful to the public should be an overriding principle of any change and will assist both in targeting external scrutiny and in informing stakeholders about performance. In the longer term the more that service users are empowered, the less external scrutiny will be needed as a safeguard.

8.6 Ministers and the Parliament should agree the priorities for external scrutiny. The recommendations in this chapter provide the basis for an assessment of such external scrutiny priorities. While it will be for Ministers and the Parliament to make decisions about these priorities, we recommend that financial audit should be one of them.

Self-assessment

8.7 External scrutiny does not exist in isolation. It is part of a wider framework of performance management and reporting. A central argument in this report is that responsibility for performance (good or bad) lies primarily with the service provider and that external scrutiny should complement performance management arrangements. The external scrutiny system proposed is predicated on robust performance management and self-assessment. The self-assessment should be firmly outcome-focused and include an assessment of all necessary compliance and performance information, and will include information which is currently checked via external inspection, audit and regulatory monitoring processes.

8.8 We recognise that the development of coherent performance management frameworks with common reporting criteria, which enable comparisons between services, is a major task which is still in development. Also, in some areas the capacity for providers to conduct rigorous self-assessment remains under-developed and more work will be required before full reliance can be placed upon self-assessment.

8.9 In the short-term, this would mean a continued role for scrutiny in assessing performance, but with an emphasis on working with providers to help them deliver effective and robust outcome-focused self-assessment systems.

8.10 In the medium term, the application of self-assessment will reduce the volume and burden of external scrutiny. As this reduces, there will be opportunities to rationalise the application of scrutiny and this will have significant implications for the number of scrutiny organisations in the long term.

Future use of scrutiny

8.11 Financial audit should continue to be an essential element in the scrutiny system. However, we do not believe that the full range of other cyclical external assessments of compliance with regulations or performance against targets, presently undertaken by inspection, audit and regulation organisations, will continue to be necessary. We propose that such cyclical assessment should happen only where all other options have been considered and ruled out. We propose, instead, that scrutiny be used in a much more focused and proportionate way and applied only where it is an appropriate response to any given issue. The nature and intensity of any external scrutiny response should be proportionate to the benefits it is expected to deliver. The circumstances where external scrutiny could be considered might include:

(a) Where there is the need for periodic independent assurance about whether services are safe, meeting regulatory requirements, and/or delivering value for money;

(b) Where service provider self-assessments are unsatisfactory;

(c) Where there is the need to assess the impact of a national policy priority;

(d) Where a serious service failure arises in one area and assurance is required about wider;implications.

8.12 We propose that there should be two core stages in considering the application of external scrutiny. First, there must be an assessment of risk. Second there must be an assessment of whether external scrutiny is the right tool to address the perceived risk and the nature of the external scrutiny required.

Risk

8.13 Any issue where external scrutiny is being considered should firstly be assessed against agreed risk criteria. We propose that core risk criteria be agreed by Ministers and considered by the Parliament, and informed by work scrutiny organisations have developed on risk assessment. ¹⁰ The criteria could include:

(a) Availability of evidence to support the perception of risk i.e. the extent to which the risk can be measured, including whether the issue identified is likely to be indicative of a wider problem;
(b) Societal concerns, including the number of members of the public that may be affected and the extent of any threat to the safety of individual service users and the public more widely;
(c) The current level of public expenditure on the area or service under consideration - it might be argued that the more public funds are expended on a service or policy, the greater the need to address any risk.

Nature of external scrutiny

8.14 Any scrutiny response should be proportionate to the risk. A;proportionate approach must also recognise differences in the size and complexity of service provider organisations, as well as the capacity of organisations to respond. The following questions should be considered:

(a) Why is external scrutiny required? Why is the service provider not able to provide the required information or assurance?;

(b) What will the external scrutiny achieve? There must be clarity of purpose and of intended impact, including the benefit to the public.

(c) Have the costs and benefits been clearly articulated? Are the benefits expected to outweigh the costs? Factors to be considered here are:

(d) For what time period is external scrutiny required? Is there a case for ongoing, cyclical scrutiny, if so when should this cease?

8.15 The nature, scope, intensity and duration of the external scrutiny activity would then need to be decided. We propose that, where an external scrutiny need is identified, and there is more than one existing external scrutiny organisation with a direct interest and relevant expertise, only one should be asked to do the work and to be responsible and fully accountable. The organisation should be appropriately resourced, including resources to bring in relevant skills and experience. This resource would be drawn from existing organisations, in either the scrutiny or service provider sectors. Creating a new external scrutiny organisation to attend to the issue should not be an option.

8.16 Each external scrutiny initiative or programme should have a timeframe, recognising that issues are resolved, services change and needs alter. We therefore propose that there be a 'sunset' clause within any decision to initiate external scrutiny.

8.17 A "one in one out" approach has been adopted for business regulation in order to control the overall regulatory volume and we believe something similar should be applied to external scrutiny in the public sector immediately. This approach was also suggested for the public sector by the Parliament's Finance Committee Inquiry into Governance and Accountability. The aim would be to remove or scale back existing scrutiny whenever any new scrutiny is introduced. Mechanisms should be put in place now to ensure that this is managed in a strategic way.

8.18 The nature and focus of the external scrutiny should be appropriate to the issue and designed to deliver the outcome required. Options might include:

(a) National scrutiny - such as reviews of the effectiveness of whole policies or delivery programmes, or the consideration of the effectiveness of policy-to-delivery chains;

(b) Community scrutiny - such as reviewing local area or community level impact of service delivery, such as is delivered through partnership working;

(c) Corporate scrutiny - looking at how the totality of a single organisation works across a range of service provision (an example would be Best Value audit);

(d) Population group scrutiny - single service or single-population reviews/inspections such as an individual school inspection.

8.19 In each case, a variety of methods could be applied:

(a) Testing of the performance management systems of the service, by focusing on a high risk issue or scenario relevant to the organisation e.g. management of MRSA in a hospital, management of "looked after" children on the point of leaving full time residential care;

(b) Unannounced inspections of a random sample of service delivery outlets;

(c) Lay inspections of a random sample of service delivery outlets.

8.20 Diagram 6 illustrates where scrutiny would sit within a new performance management and reporting framework.

Diagram 6: -The place of external scrutiny in a future performance management framework

Note: -where a service is provided jointly ( i.e. by more than one service provider), there is likely still to be a requirement for some form of self-assessment. In such circumstances it will be for the providers to agree who should take lead responsibility for its completion.

Leadership and control

8.21 Developing an efficient and co-ordinated scrutiny system will require firm leadership and control. Decisions about how external scrutiny should be applied need to be taken strategically. We believe that responsibility for such decisions should rest with Ministers, the Parliament or the Auditor General. It will be critical that such decisions are taken corporately by Ministers, and we believe that there will be a need for a Cabinet-level committee or group to take leadership on this matter and to maintain the momentum for change. The Parliament would be expected to play a more proactive role in endorsing and challenging proposals and in seeking assurance through external scrutiny, and it should identify a suitable committee or group to take responsibility within the Parliamentary system. The Audit Committee already routinely considers external scrutiny reports and it could perform a lead role. We are not proposing any change to the statutory status of the Auditor General, but we suggest that the Auditor General could make use of the advice that we propose below when considering audit activity.

Supporting decision-making

8.22 We anticipate that Ministers, Parliament and the Auditor General will need to draw on advice in reaching decisions about the application of scrutiny. This already happens, but the process should be more transparent and consistent, and should consider the views of a wider range of stakeholders. Such advice should be provided through independent expert panels with expertise drawn from the external scrutiny sector, service users and service provider representatives. We are not suggesting that there should be a permanent established group. However we do think panels would need leadership and co-ordination and some consistent understanding of their duties. This;could be provided by one agent, for example the Auditor General. The panels would consider requests only where they were referred by either Ministers collectively or the Parliament.

8.23 Individual panels would be selected according to the issue under consideration and would offer advice on whether external scrutiny was appropriate. If they felt that it was, they would also offer advice on the nature, scope and duration of the activity. They could be used to consider decisions about new external scrutiny and on rationalisation of existing external scrutiny activities. The advice of the group would be placed in the public domain. This mechanism would contribute to consistent decision making and would help to make decisions more transparent. However, final decisions will be for Ministers, or the Parliament, or the Auditor General.

Co-ordination of external scrutiny

8.24 Making changes happen will require co-ordination and strategic leadership of the external scrutiny sector. External scrutiny organisations will be important players both in developing the longer-term changes and in making them work. The single status for scrutiny bodies suggested below could lead to more flexible use of staff across external scrutiny organisations. All other external scrutiny organisations should be required to collaborate with the lead organisation and, therefore, there should be a duty of collaboration placed on all external scrutiny organisations.

Clear governance for scrutiny bodies

8.25 The report has discussed the proliferation of governance arrangements and raised the issue that this contributes to confused lines of accountability and constrains effective joint working. There is a need to develop suitable governance arrangements that take better account of the role of Parliament and overcome constraints on joint working. We suggest that one way to address this matter would be for all external scrutiny organisations to have one "status", with a clearly defined line of accountability to the Parliament and to Ministers. This will have implications for all existing regimes and there may be organisations or office bearers for which the same single status may not be appropriate. If so, such bodies should still comply with our principles of independence and accountability. Overall we believe that a robust assessment of the status of existing external scrutiny organisations should lead to greater clarity and simplicity of governance. The introduction of a single status should also set the scene for future rationalisation of the current landscape.

8.26 External scrutiny organisations should account for their performance and use of public money in common with all other public organisations. There is a need to develop better measures of the impact of external scrutiny on public sector performance and to track the real cost of external scrutiny (including compliance costs). The Scottish Government and the Parliament should work with scrutiny organisations and service providers to develop impact measures. Audit Scotland has already contributed useful thinking to the Review on improving cost measurement and this should be built upon. External scrutiny organisations should report against these measures to Ministers and the Parliament and reports should be made public. This would help ensure there is sufficient independence for external scrutiny organisations.

Accountability to elected representatives

8.27 We propose that the Parliament should be involved in all decisions about strategy for application of external scrutiny and should be notified of any referrals made to an advisory panel. Equally, where the Parliament is considering external scrutiny, Ministers should be notified.

8.28 The outputs of external scrutiny should be reported simultaneously to all stakeholders including Parliament, local government elected representatives, the Scottish Government, service delivery bodies and the public. This would increase the transparency of the overall system and would help to ensure that the findings of scrutiny are accessible. Enhanced reporting coupled with increased capacity to influence scrutiny would provide Parliament with an alternative means of bringing about change and could reduce the desire or necessity to effect change through legislation.

8.29 We noted in chapter 4 that there is an increasing interest in developing the scrutiny role of local elected members. Enhanced reporting to this group could help to build potential for local elected representatives to make more use of scrutiny reports, as a client of scrutiny. Local members could use scrutiny in this way to hold other local providers to account in their role of representing the public interest in a local or sectoral context.

Chapter summary and recommendations

Features of a scrutiny system

External scrutiny should operate as a coherent system. The features of this should include:

• Strategic priorities agreed by the Scottish Government and Parliament and focusing on areas where assurance about public services is most important. We recommend that financial audit should be one of the priorities.
• Self-assessment by providers as the main source of information about performance. Performance management frameworks are still being developed, so reliance on self-assessment is a medium/longer term goal.
• There should be two stages in considering the application of scrutiny. First, there must be an assessment of risk. Second there must be an assessment of the appropriate external scrutiny required.
• Where scrutiny is needed, if there is more than one existing organisation, only one should be asked to do the work and to be responsible and fully accountable. Creating a;new external scrutiny organisation to attend to the issue should not be an option.
• Each external scrutiny initiative or programme should have a timeframe with a pre-set 'sunset' clause.
• Cyclical inspection, audit and regulatory programmes should happen only where all other options have been considered and ruled out.
• Existing scrutiny should be removed or scaled back when new scrutiny is introduced.

Leadership

• Responsibility for decisions to create scrutiny should rest with Ministers, the Parliament or the Auditor General. At Ministerial level a Cabinet-level committee or group should take leadership on this. The Parliament should also identify an appropriate corporate mechanism.
• Independent expert panels, drawn from the scrutiny sector, representatives of service users and providers, should provide advice on the application of scrutiny. Coordination of the panels could be provided by the Auditor General.

Governance

• All external scrutiny organisations should have one "status", with a clearly defined line of accountability to the Parliament and to Ministers. An assessment of the status of existing external scrutiny organisations should be carried out.
• The Scottish Government and the Parliament should work with scrutiny organisations and service providers to develop impact measures for scrutiny, against which scrutiny organisations should report to Ministers and the Parliament.

Accountability

• The Parliament should be involved in all decisions about strategy for application of external scrutiny.
• The outputs of external scrutiny should be reported simultaneously to all stakeholders.
• Enhanced reporting coupled with increased capacity to influence scrutiny would provide Parliament with an alternative means of bringing about change and could reduce the desire or necessity to effect change through legislation.