The Crerar Review: the report of the independent review of regulation, audit, inspection and complaints handling of public services in Scotland

04 PUBLIC FOCUS

4.1 External scrutiny and complaints handling hold service providers to account on behalf of the public and public focus is one of the five key principles we are recommending. This chapter considers how public focus is currently used in external scrutiny and how it could be strengthened in the future.

4.2 There are different public stakeholders. Service users (and their carers or guardians) have an immediate stake in the delivery and external scrutiny of services that they use. The public more widely may have an interest in the effectiveness of services and will also have an interest in the way that public money is being used and accounted for.

4.3 These stakeholders may have conflicting interests. For example, service users are likely to support highest quality services supported by robust external scrutiny, whilst the general public may wish to balance these against cost to the public purse and value for money. Different external scrutiny mechanisms have traditionally reflected these interests, with audit more focused on public interests in propriety in the use of public money, whilst regulation, service inspection and complaints handling are more focused on the interests of individual service users.

4.4 Whatever the level of interest, the user-voice can be an important source of information for those undertaking an independent assessment of how well public services are delivering, especially where market models and consumer choice are not well developed or appropriate.

4.5 We commissioned two pieces of work specifically to examine issues relating to external scrutiny and the public. The "Scrutiny and the Public" study took evidence directly from members of the public and a literature review on the "Consumer Approach to Scrutiny" pulled together a critique of issues raised by research. ²

Public awareness

4.6 Although the public have limited awareness of scrutiny organisations and processes, there is a clear public expectation that there is and should be external scrutiny of services provided to them, or on their behalf. The "Scrutiny and the Public" study found that participants were more familiar with inspection and complaints handling than with regulation and audit. Respondents were aware of some external scrutiny organisations, in particular the Care Commission, HM Inspectorate of Education ( HMIE), the Health and Safety Executive and the Scottish Public Services Ombudsman ( SPSO). All respondents assumed services would be checked, even when they did not know how or by whom.

Public involvement in external scrutiny

4.7 Service providers have primary responsibility to account to the public for their performance. Service providers also have primary responsibility for engaging users and the public. As Byatt and Lyons ³ put it:

"It should not be the role of external scrutiny to act as an intermediary between service users and service providers or to form an independent view of user priorities. It is the responsibility of providers to seek and respond to the views of users. External reviewers can and should, however, review how effectively service providers fulfil this role. This does not preclude external review including an element of gauging directly the user experience of services."

4.8 There is agreement that the public have a legitimate stake in external scrutiny and that there should be greater public involvement in external scrutiny processes. Stakeholders also accept that current arrangements for involving the public are not fully developed. The work we have drawn on indicates that:

• External scrutiny organisations do not always place their work within the context of a consumer approach to public services;
• Service users are not always referred to in the organisational aims of external scrutiny organisations; nor are they routinely involved in the governance of external scrutiny organisations.

4.9 External scrutiny organisations attempt to capture the service user perspective and to involve the public in a variety of ways:

• Most external scrutiny organisations assess levels of satisfaction with providers and link;this to evidence obtained from other sources - e.g.HMIE, Care Commission, Communities Scotland;
• Some scrutiny organisations will make use of provider data on complaints - we consider complaints in more detail in chapter 11 of this report;
• Some attempt to gauge how well service providers involve their clients in decisions about the service to judge whether it is meeting its clients' needs; and
• Most now involve service users in their processes, for example in setting the standards, or as part of the inspection team, or in their own governance arrangements. But some questions remain about how user evidence should be weighted with or against that of professionals.

4.10 Examples of how and why external scrutiny organisations involve users and the public are summarised in table 1.

Table 1: Public involvement in external scrutiny activity

Type of involvement	Aim	Methods
Governance of scrutiny bodies	To ensure openness and transparency in the governance of scrutiny bodies and balance the provider interest with the public interest.	The recruitment of board members who are not professionals in, or employed by, the service under;scrutiny. For example the Care Commission must have two non-executive Board members who are users of care services or carers to service users by statute.
Inspection and review activity	To ensure openness and transparency in scrutiny activity and balance the provider interest with the public or consumer interest.	Use of "lay people" in inspections: in some cases, lay people are also consumers though this is not always the case. For example, Communities Scotland use tenant assessors, NHSQIS* use lay assessors.
Consultation with, or direct involvement of consumers	To ensure that the consumer voice is heard in the process of scrutiny.	Involvement of consumers and/or their representatives in the process of setting standards e.g. Care Commission- National Care Standards. Active engagement with consumers in reviews/inspections (most scrutiny bodies do this to varying degrees of success). Consultation with consumers about their experience of the service under scrutiny (most scrutiny bodies do this to varying degrees of success).

Source: "Scrutiny and the Consumer", Scottish Consumer Council
* NHS Quality Improvement Scotland

4.11 But despite these aims and methodologies, public involvement is not fully developed.

4.12 The "Consumer Approach to Scrutiny" literature review cites studies that show that the UK population does not place a high level of trust in government information. Therefore, if external scrutiny is to be credible, the public needs to be convinced that it is independent of government and that the process of external scrutiny is robust and reliable. The "Scrutiny and the Public" respondents generally accepted that external scrutiny organisations were independent, but they saw scope for "regulatory capture" where regulators get too close to service providers to give an unbiased view. One important way to overcome this is through public involvement in the external scrutiny process. Respondents also thought that the nature of the external scrutiny process was important and placed particular stress on unannounced inspection as the only way to obtain a true picture of service delivery as experienced by the public.

Consumer focused measurement of public services

4.13 The extent to which user-focused external scrutiny can be fully developed partly rests on the extent to which wider policy intentions and performance management frameworks are expressed in terms of outcomes for users. For example, we agree with comments made in the Auditor General's submission on the difficulty of assessing Scottish Government policies. We agree that the intended benefits to the public have to be better articulated. The Auditor General suggests there is an opportunity for external scrutiny organisations to work in conjunction with providers, the Scottish Government and consumer representatives to strengthen the user voice at all levels of public reporting.

4.14 The sources reviewed in the "Consumer Approach to Scrutiny" literature review are critical of traditional input and process measures for public services. Traditionally, external scrutiny organisations focus on the internal mechanisms of the delivery body, at an individual service or an organisational level. External scrutiny regimes assess leadership, management systems and internal processes, service outputs and performance management systems using a mix of performance and quality indicators, standards and statutory regulations.

4.15 We support the development of more outcome focused measurements of public services overall, including those used by external scrutiny. Outcome-focused scrutiny is important, and will become even more so if the Scottish Government moves to an outcome-agreement approach with public service delivery bodies. External scrutiny will need to adapt to that change as and when it happens. However, outcome measurement is still being developed for public services in Scotland and there are inherent difficulties in defining what outcomes should be and how to measure them. Outcome measures can include:

• Tangible outcomes, such as exam results for school leavers;
• Equity outcomes, such as the differences in attainment between different ethnic groups; and
• Consumer satisfaction, such as numbers of parents expressing satisfaction with school exam results.

4.16 External scrutiny organisations are already addressing measurement issues and there are several examples to draw on, for example - "Nine principles for user focus, equality, diversity and human rights for inspection" by the Audit Commission and Local Services Inspection Forum in England and "Principles for service user and carer participation" developed by the Joint Participation Steering Group in England. ⁴

4.17 While outcome measurement is being developed, we cannot conclude that the system is yet ready to shift fundamentally to focus on outcomes, but should do so once the Parliament is satisfied that outcome measurement tools are in place, sector by sector.

Accessibility of reports

4.18 Studies in Scotland and the USA suggest that where external scrutiny information is available, the public do not seek it out unless reports are meaningful to them. This involves being written in language that people can relate to, containing information that is relevant to what people want to know and being trusted. The "Scrutiny and the Public" study found that external scrutiny reports are not always accessible to the public and could be improved by:

• Making reports very short or including a summary report;
• Writing reports in plain English and avoiding abbreviations/jargon with which most service users/members of the public will not be familiar;
• Being very clear about how any summary measures or quantitative measures are calculated and what they mean;
• Containing both narrative descriptions and summary/quantitative measures. The latter were useful for an "at a glance" assessment, but some element of description is also needed to provide context, a feel for the overall judgement of a service and an assessment of less tangible or measurable aspects of service delivery.

4.19 These conclusions point to a need for a collective effort by external scrutiny organisations to make information more accessible, develop common performance reporting language and explore ways of explaining the complexity of what they have found as straightforwardly as possible.

Concerns about greater public involvement

4.20 Despite the general consensus in favour of public involvement, some concerns were expressed about the potential costs of supporting more extensive public involvement; and the dangers of imposing a burden on the public by asking for feedback too frequently. There were also concerns that service user expectations could potentially act as a restraint on improvement, in circumstances where consumers are satisfied with poorly performing services and are resistant to change in service delivery. However we believe that this danger should be offset by other challenges to improve service performance and the development of robust performance standards. More consistent public involvement and a more informed public could also drive up expectations and standards.

Chapter summary and recommendations

• Service providers have primary responsibility for engaging with users and the public.
• The public have a legitimate stake in external scrutiny and that there should be greater public involvement in external scrutiny processes. External scrutiny bodies currently involve the public, but arrangements are not fully developed across all services.
• The public believe that to be credible, scrutiny should be independent of government, and whilst they have limited awareness of scrutiny activity, they expect it to be done on their behalf.
• We support the development of more outcome focused measurements of public services overall, including those used by external scrutiny. External scrutiny organisations should work in conjunction with providers, the Scottish Government and consumer representatives to develop outcome measures and to strengthen the user voice at all levels of public reporting.
• Scrutiny reports should be written in language that people can relate to, containing information that is relevant to what they want to know and being trusted. Scrutiny bodies must make information more accessible, develop common performance reporting language and explore ways of explaining the complexity of what they have found as straightforwardly as possible.