Mental Health Scrutiny and Assurance Evidence Review

Literature Review

This section offers a review of the evidence base. It also describes the roles of each scrutiny body in relation to their mental health activity and identifies strengths and gaps in scrutiny activity.

Scrutiny Arrangements in Scotland

The Public Services Reform (Scotland) Act 2010 placed a duty on scrutiny bodies to cooperate and coordinate their activities with each other. A networked model was adopted in Scotland to improve the efficiency, effectiveness and economic performance of scrutiny bodies such as HIS, MWC and CI. Whilst the MWC’s remit is concerned fully in regard to people with mental health issues and the services they use, HIS and CI acknowledge this aspect forms a smaller part of their remits. The volume of scrutiny inspections carried out by each of the organisation on specific areas of mental health is captured in Table 2.

HIS is a national improvement organisation and one of its key roles is independent assurance of quality of care. This sits alongside remits for improvement and redesign support for health and care services, independent assessment of evidence to underpin high quality care, and support to ensure citizens and communities are at the heart of change. Its evidence, improvement and community engagement functions all sit as equal partners alongside assurance under its overall quality management framework. Its priorities are driven by a range of stakeholder needs including system and policy priorities for evidence and improvement alongside key themes identified through assurance and scrutiny activities. HIS has the power to exercise the functions of Scottish Ministers to support, ensure, and monitor the quality of healthcare provided or secured by the health service and the discharge of the duty on NHS boards to encourage public involvement. HIS also evaluate and provide advice to the health service on the clinical and cost effectiveness of new and existing health technologies including drugs.

The MWC holds a duty to undertake enquiries and investigations under the Mental Health (Care and Treatment) (Scotland) Act 2003 (2003 Act). The MWC is narrowly defined within the legislation as their duty is specific to people detained under the Act, and the Adults with Incapacity Act. People receiving the support of mental health services in hospital or using community settings are more likely not to be subject to the Act compared to the numbers of people who are. The MWC visits people to check that they are receiving appropriate care, where there are indications that appropriate care and treatment is not being received investigations are completed (MWC for Scotland, 2022a). The MWC aims to visit each hospital once a year and The State Hospital twice per year. The MWC undertake themed visits and produce closure reports to follow up on recommendations and to check progress, which are now published for transparency. Alongside the investigative duty, the MWC also has a number of other duties to discharge such as visiting individuals and services, providing information and advice, and influencing and challenging. The MWC is specific to mental health in their scope and remit, scrutinising services who deliver care to people with mental health conditions (Mental Welfare Commission, 2022b).

The CI do not have a locus to inspect the majority of the services detailed in the table. CI therefore have a limited role in inspecting secondary mental health services, which are the predominant focus of this review. The Public Services Reform (Scotland) Act 2010 gives the CI powers to inspect any social care support service or combination of social services, or the organisation or coordination of any social services, relating to the whole or any part of Scotland. As such, from the list of services in the table only those that are community based might be part of the Care Inspectorate’s remit. This would only occur as part of a joint inspection with Healthcare Improvement Scotland, to reflect the integrated nature of such services. There have been no specific inspections of Community Mental Health Teams, so no categories within Table 2 are relevant for the CI. However, from March 2022, the CI regulated 52 care homes for individuals experiencing mental ill health. A sizeable proportion of the 1,499 support services and the 1,054 housing support services will also provide care and support for this client group.

Jointly with HIS, the CI Adult Strategic Team conduct programmes of themed inspections, scrutinising how the integration of community-based services positively support people and carers’ experiences and outcomes. These themed inspections are not focused on the quality of specialist care for each care group but are a means of identifying groups of people with similar or shared experiences through which to understand how health and social care integration arrangements are resulting in good outcomes. Physical disability and complex needs were the themes of the first phase of these inspections from 2022 and from mid-2023 the CI will focus on mental health. The CI stratification of case records for adult service inspections includes mental health. As such, their inspections consider, and report on, the impact of mental health provision on service user groups.

Effective Communication and Information Sharing

Information sharing protocols and memorandums of understanding are in place between the MWC and the Health and Safety Executive (HSE), SPSO, CI, HIS and Scottish Government on areas of mutual interest to promote effective working relationships with other organisations (Mental Welfare Commission, 2022). In order to provide scrutiny, organisations such as HIS, MWC and CI require a platform to share, discuss and act upon emergent themes and any issues identified from their scrutiny activities. The Sharing Intelligence for Health and Care Group (SIHCG) provides this platform. SIHCG provides a mechanism that enables seven national organisations to share, consider, and respond to intelligence about health and social care systems across Scotland. These organisations consist of Audit Scotland, CI, HIS, MWC, NES, Public Health Scotland, and SPSO (Healthcare Improvement Scotland, 2022). The SIHCG aims to support improvement in the quality of care provided in Scotland by making good use of existing data and intelligence ensuring that if any agency on the group has a potentially serious concern, then this is shared and acted upon appropriately. The group aims to share concerns at the right time to ensure that emerging problems can be identified and addressed. The organisations also inform each other about aspects of health and care systems that work well and sharing intelligence helps the different organisations within SIHCG carry out their work in an informed manner (Health Improvement Scotland, 2022). Their remit is generic, cuts across all health-related activity, meaning the scope for focus on mental health is limited, and as such, reduces its ability to contribute to mental health scrutiny. SIHCG is currently under review to move away from an NHS Board focus.

The Mental Health and Learning Disability National Scrutiny and Assurance Coordination Group (NSACG) was established in November 2022, with a rolling Chair. Unlike the SIHCG, the NSACG has a sole focus on mental health and learning disability, its aim is to provide opportunity for early, and where appropriate, coordinated intervention and support. It aims to clarify roles and support accountability among members in addressing issues for effective delivery of mental health and learning disability services, while recognising and respecting each organisations’ independence current legislative arrangements. It also seeks to join up planned programmes of scrutiny, as appropriate, to ensure a cohesive approach across the organisations and reduce the burden on mental health and learning disability services. Currently it is an operational group, and still in its early stages, which means that its success is currently unclear.

National and International Good Practice

The rapid review aimed to source areas of good practice from other countries with a view to influencing change in Scotland. Information was synthesised from England, Australia, New Zealand, Ireland, Japan, Ireland, Germany, Wales, Italy and North America. Key issues such as challenges with measurement and approaches to scrutiny are presented, with examples of single regulatory bodies in England and New Zealand and framework models in Ireland and Australia.

Single body and framework

The English approach utilises a single body - the Care Quality Commission (CQC) -to regulate all health and adult social care services. Unlike the networked model used in Scotland, the CQC combines the work of HIS, MWC and CI. However, the Scottish and English models are two fundamentally different systems and not like for like. England has a single body for assurance and scrutiny but it also has a range of different organisations which cover the work which HIS do in one body (CQC, NICE, various national quality improvement organisations, and no national equivalent to HIS Community Engagement Directorate). In addition, the CI has a much wider role than CQC covering, for example, some of what is overseen by Ofsted in England.

The CQC’s role is to ensure that health and social care services provide people with safe, effective, compassionate, high-quality care. Health and adult social care providers in England are required to be registered with the CQC; facilitating improvement through monitoring and inspecting of services to see whether they are safe, effective, caring, responsive and well led. Similar to the CI, the CQC publish findings including quality ratings and have the legal powers to take action where poor care is identified. The CQC also communicates independently, publishing regional and national views of the major quality issues in health and social care, and encouraging improvement by highlighting good practice (Care Quality Commission, 2021).

The approach to external scrutiny in New Zealand is similar to that in England, in that it has a single regulator, but the regulatory role is different. New Zealand’s services are monitored by the Health and Disability Commissioner (HDC), which acts as an independent watchdog, providing people using health and disability services with a voice, supports the resolution of complaints, and holds providers to account to improve practices at an individual and system-wide level. The HDC is independent of consumers, of providers, and of government policy, facilitating them to be an effective watchdog for the promotion and protection of patient and consumers’ rights. A single regulatory system has its benefits, such as reducing the potential confusion of not knowing what the multiple scrutiny bodies are responsible for, what each one of them does, and where to look for guidance on matters. This approach could be considered an option for Scotland, however, a well-functioning multi-body system would likely mitigate the major overhaul needed in creating a single body. The Independent Review of Inspection, Scrutiny and Regulation (IRISR) across social care support services (Scottish Government, 2022) has indicated that major structural reform, as part of the National Care Service is unlikely.

Ireland have adopted a framework approach and established a clinical governance framework for quality in healthcare in 2011, with an aim to clearly communicate the fundamentals of clinical governance. The initiative took an approach that made clear within the clinical governance descriptor the actions staff would take in its implementation such as, specifying clinical standards and making measurements open to the public and transparent. This approach aimed to minimise some of the distrust between the patient and health professionals and create confidence that decisions were made on clinical quality statements (Flynn, Burgess and Crowley, 2015). The conclusion of the national initiative drew a variety of insights that could be applied to Scotland. The communication and sharing of information using an active listening approach with patients and staff was key. A trauma informed practice approach not only minimised distrust but also led to an increased understanding of what mattered to them, such as their experiences of care and what motivated staff.

Australia adopted a national strategic approach to mental health reform, through the creation of a mental health information strategy, due to a limited connection and exchange of data sets between their scrutiny bodies. Their National Mental Health Performance Framework was established in 2005 (updated in 2020). Their use of performance indicators and quality benchmarking across states and territories created a national approach to outcome measurement in mental health services and was considered a world leader in this regard at the time (Brown and Pirkis, 2009). It is important to note the difference between a performance framework and a scrutiny framework. However, there may be opportunities for Scotland depending on the metrics being developed from on-going standards work alongside existing measures from a variety of data sets to inform the development of a similar performance framework for mental health in Scotland.

Benchmarking and use of information

Herbstman and Pincus (2009) cite inadequate adoption of quality improvement practices and the complicating factors of having numerous quality/performance measurement initiatives and indicators across government bodies, health services, professional and non-governmental organisations as hampering the collection of good quality data. Whilst this raises an important point, underperformance is perhaps more complicated than suggested above. Herbstman and Pincus (2009) also argue that the failure to establish a single body to provide strategic leadership and oversight has led to a lack of coordination and the disjointed development of scrutiny services in some countries. Attempts have been made to address issues such as these through identifying a number of differing approaches to improving practice. The ability to make comparison between services nationally and internationally through benchmarking is an extremely useful exercise. A number of countries have reported on their individual systems and processes.

The need for good quality data led the Republic of Ireland’s healthcare system to use real time clinical measurement. This practice supported effective decision-making, thereby improving transparency and public/professional trust in the governance of quality and safety (Flynn, Burgess and Crowley, 2015). The importance of clinical and personal outcome measures alongside care experience data is clear to scrutiny bodies and access to this information to inform assurance activities would be of value. An example of data being used effectively in Scotland is the Scottish Patient Safety Programme Mental Health. This programme has been used to drive improvement to ensure that everyone in adult mental health inpatient wards experience high quality, safe and person centred care every time. Benchmarking and the sharing of information, therefore, are important factors to be considered within the Scottish context. Public Health Scotland produces benchmarking data across Scotland, including Mental Health Quality Indicators, which are published but not reported equally or fully across NHS Boards. Previous work led by Scottish Government in partnership with NHS Scotland Information Services Division (incorporated into Public Health Scotland) developed mental health benchmarking reports. There is utility in comparing services nationally and internationally with one another and for improving trust in scrutiny process and related activities.

England and Wales adopted a model using a single regulatory body, the Care Quality Commission (CQC), which provides regulation and scrutiny of services. The former Mental Health Act Commission and its functions were subsumed into this structure and many functions that are Rights Based continue within the CQC. However, this approach may pose a risk of systems based processes with associated metrics setting or appearing to set the scrutiny framework. The CQC continues to carry out visits similar to the way the MWC does as well as Inspections in the CQC model. The CQC monitors, inspects and regulates NHS trusts using CQC Insight. This system provides inspectors with data, a ratings overview, performance monitoring indicators, intelligence overview and featured data sources. CQC Insight analyses information from a range of sources and uses common indicators to monitor performance across all types of NHS trusts. This is also tailored to each sector or type of service. CQC Insight for providers of specialist mental health services includes analysis of the findings from visits to people detained under the Mental Health Act 1983 and relevant notifications under the Act (Care Quality Commission, 2022). This appears to be a very thorough approach and it would be interesting to explore if the data have been used to influence changes in practice across health and social care settings. It would also be useful to know what systems are used to share this intelligence given the sensitivity of the content. The provision of a Scottish specific measurement and benchmarking system could see similar positive effects within Scotland.

The UK has an NHS benchmarking network that undertakes some mental health benchmarking but is not Scotland specific. Scotland has very limited routine information about the mental health system on workforce, finance and activity, and outcomes. This has made policy and planning for mental health improvement very difficult, as little quantitative evidence around the provision of mental health services exists across Scotland and the means of identifying variability in provision between areas. This led to a commitment by the Scottish Government to support Health Boards and HSCPs to improve the data available and the capacity to compare across areas. The UK NHS Benchmarking Network (NHSBN) is acknowledged as already having a substantial evidence base around Mental Health service provision, workforce, finance, quality standards, and ongoing improvement initiatives. It has made sense therefore, for the Scottish Government to facilitate Boards to participate in this existing network, which would bring benefit as well as reducing the data burden involved in any future data collections. The Scottish Government has now successfully negotiated a contract with the NHSBN that includes membership for all Health Boards that allows access to NHSBN data (including Scottish Boards’ data) for the purpose of performance and improvement activities. Membership to the Network provides NHS Boards with a number of member benefits without having to dedicate additional resource. This includes submission of local data and participation in the three annual UK wide events in Mental Health (Adult and Older People’s Mental Health; Learning Disabilities and Autism; and Children and Young People’s Mental Health), as well as providing networking opportunities for lead clinicians and service managers across Scotland and beyond to share their experiences and good practice. It is worth noting, however, that data gathered and analysed through the Network is not publicly available.

Lived experience

Lived experience in mental health refers to people with personal experience of mental illness and recovery. It can also refer to family members and/or carers who have experience in caring for people living with mental illness and recovery. people with lived experience are essentially ‘experts by experience’ due to their first-hand experiences of mental illness, their use of mental health services, and their recovery. Therefore, they can provide a unique and expert perspective when working alongside scrutiny bodies.

The IRISR strongly recommended that the scrutiny system should be inclusive of the voice of lived and living experience, while Strang (2020) highlighted the necessity for people with lived experience to be a part of all elements of scrutiny of a service: the design, clarity of governance and leadership, engaging with people and communication, including children and young people. Lundqvist, Lars-Olov, Schroder (2015) identified that people with lived experience and staff do not always have the same perspective of good quality care, thus the inclusion of all voices is necessary for providing information that will improve practice. An example of this is when adverse incident reviews cause dissatisfaction because the facts do not correlate with service users own understanding.

People with lived experience and their perspective should be involved in all aspects of quality assurance projects from initial vision to analysis and the presentation of results, thereby enhancing the evolution of quality assurance within organisations (Weinstein, 2006). Unfortunately, it appears that lived experience involvement has been seen as a token gesture at times (Moore, 2008; Ryan et al., 2019; Weinstein, 2006). It is more common for people with lived experience views to be sought during initial consultation, but rarely beyond this, with their perspectives lost to the planning and delivery stages of scrutiny. In North America for instance, people with lived experience involvement in planning and evaluation of quality assurance was viewed as problematic due to a lack of role clarity, discomfort from professionals and power dynamics (Weinstein, 2006). Contrastingly, in Japan, former people with lived experience publicly speak at conferences and government panels about the quality of care in mental health services (Ito, 2009) leading to an increase in shared decision-making about services.

The Scottish approach has seen scrutiny bodies increasingly use people with lived experience in their preparation for inspections, as part of their inspections, and their involvement in the interpretation of findings and this is seen as good practice. The Crerar review in Scotland noted most scrutiny bodies now include the voice of lived experience in some capacity.

People with lived experience can and should be supported to be meaningfully involved in scrutiny and assurance practices. To facilitate and encourage this, having continuity in the care professionals working with individuals is necessary, to avoid a breakdown in trust and ensure people with lived experience feel listened to. Having this voice included, ensures organisations are able to learn from experience with a non-blame culture embedded, and gather a more meaningful response.

Summary

In Scotland, scrutiny bodies have their own particular remits, and as discussed, some more than others focus on mental health as part of their scrutiny portfolios. There are positives associated with this approach, for example, the MWC role is solely around the scrutiny of services delivering care to people with mental health conditions. HIS provides an evidence-base, delivers improvement and implementation support, and scrutiny practices. The CI regulates social care and social work services, undertakes strategic scrutiny, often in partnership with other scrutiny bodies, supports quality improvement, and has a duty to further improvement. However, there appears to be negatives associated with this approach too. The recently published Independent Review of Inspection, Scrutiny and Regulation (IRISR) by Dame Sue Bruce found that systems of inspection, scrutiny and regulation can be duplicative, which poses challenges for both service providers and those in receipt of social care support. The IRISR recommended that roles and responsibilities between organisations are clarified, and that inspection activity should be streamlined. Similarly, the current mental health scrutiny landscape and different remits of scrutiny bodies creates a potential dilution of awareness in health and social care services, third sector and the wider population over the purpose of the individual scrutiny bodies. In addition, mental health inspection and regulation practices lack a coordinated approach to inspections across the bodies.

This rapid review has identified a number of key learning opportunities in the literature regarding good practice at local, national and international levels. Adopting a national approach and an overarching framework has been seen in Australia and Ireland to support the consistency of scrutiny practices. The standardisation of quality indicators and measures used to monitor and evaluate patient outcomes in order to improve the quality of care is considered necessary. A systematised and managed approach by government to the measurement of data, its collection, and reporting is important so ensure different audiences such as patients, scrutiny bodies, policy makers and healthcare providers have timely access to information in an understandable format. Finally, the increased recognition of the importance people with lived experience involvement can offer in all aspects of the scrutiny process and the assurance of mental health services.

Survey Results

There were three questions that overlapped on the two questionnaires (Part A and Part B) issued to senior personnel in Health Boards/IJBs/Special Health Boards. Findings from the 23 respondents are outlined, with quotes identified as either Health Board (HB) or Scrutiny body (SB) for ease of interpretation.

There was broad agreement from 17 (74%) respondents that pertinent issues were identified through local governance arrangements, 3 (13%) were neutral and 3 (13%) disagreed. The same proportion (17, 74%) believed that findings are presented accurately; 5 (22%) took a neutral stance on this issue and 1 (4%) disagreed.

Only 8 (35%) agreed that external scrutiny bodies worked well with service providers, following up and providing support to implement recommendations made; 10 (44%) took a neutral stance and 5 (21%) disagreed. There was a suggestion that:

‘More work could be done once strategic scrutiny has been completed to support improvement and signpost good practice’. HB

Findings from the remaining questions from eleven respondents to Part A are outlined below.

Governance and scrutiny

When reflecting on the frequency of inspections only 4 (36%) agreed this was sufficient, 4 (36%) disagreed and the remaining 3 (27%) were neutral. The Health Boards commented that scrutiny should be ‘based on risk and intelligence’ as well as being a cyclical approach.

Only 3 (27%) considered the co-ordination of external bodies to be adequate, 4 (36%) were neutral and 4 (36%) disagreed. It was thought the strategic scrutiny group did not always put in place effective co-ordination of scrutiny:

‘Having come from a highly regulated environment (England) where everyone knows the standards expected of them, coming to Scotland was a shock. There are many many groups/people/advisors/regulators involved and it makes the landscape cluttered’. HB

There was however broad agreement (7, 64%) from Health Board respondents that the local governance arrangements raise standards of healthcare, 4 (36%) remained neutral. In relation to external scrutiny arrangements 5 (45%) agreed they raised standards, 4 (36%) were neutral and 2 (18%) disagreed. This does perhaps indicate that the respondents have more faith in local governance than in external governance.

Gaps in scrutiny activity

There was a suggestion that there are gaps in external scrutiny activity, 5 (45%) agreed with this and 6 (54%) remained neutral.

Six (54%) respondents agreed that governance procedures were equitable across the lifespan of patient care, 2 (18%) were neutral, 2 (18%) disagreed and there was 1 (9%) non-response. Fewer agreed (4, 36%) external scrutiny was consistent across all Health Boards, 3 (27%) were neutral, 3 (27%) disagreed and 1 (9%) non-response:

‘It is a patchwork of regulators/standards and interested parties; I would say this increases the risk to patients’. HB

Leadership

A few issues emerged in relation to leadership, firstly the manner in which responses were reported. Only 3 (27%) respondents considered leadership of external scrutiny bodies adequate, 5 (45%) remained neutral and a further 3 (27%) disagreed. The free text comments associated with this section of the questionnaire implied a level of negativity towards existing leadership arrangements across scrutiny bodies collectively, rather than individual leadership, and a considerable proportion of neutral responses were recorded.

Involvement of People with Lived Experience

A leadership issue also emerged in relation to the presence/absence of people with lived experience at senior levels of the scrutiny workforce; the prominence of the people with lived experience voice and their ability to challenge management:

‘There needs to be greater leadership within external scrutiny bodies to ensure lived experience is not seen as a data gathering exercise, but is instead about lived experience leadership within scrutiny bodies (who are able to challenge management/clinicians)’. HB

The impression from Health Boards was that People with Lived Experience (people with lived experience) are engaged (7, 64%) and their voices are heard through local governance arrangements, 2 (18 %) were neutral and 2 (18%) disagreed. Six (54%) disagreed that people with lived experience involvement extended across all stages of the process, 2 (18%) were neutral and 3 (27%) agreed. A representative suggested that:

‘The problem lies in the idea that people are just involved, they should be leading and driving rather than being involved. There is a need to ensure scrutiny and assurance is not "done to" and instead "done with". This needs to be reflected in the workforce of scrutiny bodies to ensure openness and transparency’. HB

The overall impression was that more could and should be done to seek lived experience viewpoints. Respondents also reported that their Community Partnerships and NHS have worked hard to secure patient and family involvement throughout service provision, especially through the COVID pandemic. They acknowledged that the involvement of people with lived experience and their carers is important and can always be improved. Examples of local projects which were working well usually involved people with lived experience at the centre of the work and it was thought this might be due to local governance arrangements that facilitated involvement i.e. co-production with people with lived experience.

It was noticeable that there was an unusually high proportion of neutral responses in this section. One possible explanation for the reluctance to submit a negative response is because it is an assessment of colleagues and a neutral score is less contentious. The alternative explanation is that respondents do not have anything positive to say.

Part B: Twelve responses were received from Health Boards/IJBs. On more than one occasion, a collective view was gathered from a wider multi-professional group, and then submitted via either the Medical or Nursing Directors.

Response to inspections

Nine (75%) participants believed that scrutiny bodies were thorough in their approach and that findings were presented accurately, 3 (25%) disagreed. Nine (75%) respondents also largely believed their own organisations were duty bound to act on the recommendations made, 3 (25%) remained neutral.

One of the more contentious issues outlined by the group - where there was greater variation - was the communication between organisations (HBs and SBs) on the implementation of recommendations; 5 (41%) believed it to be adequate, 4 (33%) remained neutral and 3 (25%) considered it inadequate. This perhaps indicates that some Health professionals were keen for more communication with scrutiny bodies following inspections and wanted visits to aid with implementing recommendations.

Eight (66%) agreed that follow up from scrutiny bodies is adequate, 2 (17%) were neutral and 2 (17%) disagreed. Nine (75%) agreed that reporting mechanisms for feedback on recommendations is clearly articulated by scrutiny bodies, 3 (25%) disagreed. This suggests that Health/Social Care professionals are broadly happy with feedback, yet there is always scope for improvement in this area.

Eight (66%) respondents also agreed that scrutiny bodies supported the inclusion and participation of people with lived experience more readily in the follow up process, 3 (25%) were neutral and 1 (8%) disagreed. Eight (66%) also believed there was a reasonable timeframe for recommended changes to be introduced, 2 (17%) were neutral and 2 (17%) disagreed with this statement.

Overall, a more positive response was received from respondents of Part B than Part A. One possible explanation for this is that the Chief Executives who responded to Part A had overall responsibility for their services and essentially need to be more critical/analytical in their approach.