Mental Health Scrutiny and Assurance Evidence Review

Interview Results

Findings are presented from the twenty interviews conducted with professional stakeholders from Health Boards/IJBs, CI, HIS, MWC, NES, SPSO, The State Hospital (TSH), and VOX. Participants were willing to engage in discussion and shared both positive and negative experiences openly. With the exception of one Health Board, one council, and one member of a scrutiny body, all invitations to interview were accepted. It was heartening to see the commitment to progress the Scottish agenda in a positive manner, despite some differences in experience and opinion.

Five clear themes emerged from interviews: ‘Scrutiny Activity and Leadership’, ‘Assurance Approach’, ‘Lived Experience’, ‘Measuring and Reporting’ and ‘Learning and Development’, see summary in Table 2.

*people with lived experience represented by an organisation (VOX) are captured in HB responses.

The first two themes ‘Scrutiny Activity and Leadership’ and ‘Assurance Approach’ highlight the tension in a number of key areas and consider the disconnect that is evident across Scottish scrutiny bodies; there is evidence of good working practice in individual organisations, but an overall lack of cohesion. The third theme ‘Lived experience’ relates to their meaningful involvement. The fourth theme ‘Measuring and Recording’ identifies areas for development and the final theme ‘Learning and Development’ outlines opportunities for improvement.

Each theme will be reviewed in turn. Quotes are again identified as either Health Board (HB) or Scrutiny body (SB) for ease of interpretation.

Scrutiny Activity and Leadership

Cluttered and Changing Landscape

Several respondents commented on the number of different organisations that contribute to the external scrutiny of services and the different focus of each. Their understanding was that most elements were covered technically (by scrutiny bodies), but:

‘It is unclear who is doing what, and things might get missed amidst the plethora of information’. HB

Opinion was divided as to whether the differences were welcomed or a source of frustration, consequently creating confusion and lack of clarity. One respondent commented on the fact that Scotland does not have a single regulator and it is perhaps worth considering this and exploring how it performs by comparison to other national and international areas. Despite the absence of a single regulator, it was noted there are other ways in which scrutiny bodies, regulators and assurance mechanisms – like the Commission – come together. The example outlined was the Sharing Intelligence Health Care Group (SIHCG), which meets monthly, but has a wide remit that extends well beyond mental health and learning disability. Organisations also have a monthly huddle to discuss key quality improvement work priorities.

There was agreement from the majority of respondents that the creation of one large complex organisation was not necessarily seen to be the best or most effective solution, given the vast scale of health services that Boards provide. However, work is required to provide the necessary clarity about roles of the different organisations and how they fit together.

Functions

It became apparent that individually each scrutiny body felt able to identify issues through their different functions, whether it was through oversight, regulation or scrutiny. Yet there was:

‘No collective oversight’. HB

The Commission (MWC) described a variety of its functions, which allow it to have a good sense of what is happening in each area. Through various channels (described earlier), the Commission believes it is able to get a sense of what is happening both at an individual level and structurally.

HIS and CI acknowledged that the work they engage in with people who have mental illness or intellectual disability only contributes to a small proportion of their wider role. HIS contribute heavily to quality and improvement of services, rather than the focus on individuals. Respondents reported the substantial shift in the quality assurance system that has been created and the quality framework, highlighting this as an area of good practice. The Scottish Patient Safety Programme HIS lead on was viewed as a particularly sound initiative and a good example of improvement work.

The CI inspect standards of care across a wide range of organisations including, for example, child and adolescent, adult care services as well as care homes. Their body of work extends across the lifespan, but they do not scrutinise NHS delivered mental health services. There is an obvious crossover of activity that could be streamlined, whilst still maintaining unique purpose and function of each organisation. The existing scrutiny services complement one another and if they worked in tandem this would add rigour to the overall process.

Resources versus Demand

Some consider the output from each of the scrutiny bodies quite exceptional and several senior clinicians highly praised the organisations, sharing the view scrutiny bodies could do even better with more resources. Respondents from scrutiny bodies did acknowledge that:

‘There’s a limited amount we can do with the resource we have’. SB

There was a concern that they could not cover everything they wanted with existing resources and feared this might lead to something being missed. One respondent noted that Scottish Government were good at:

‘Coming up with some wonderful policies and having fantastic intentions ….but they have no resources’. SB

There was a suggestion from HBs that if the SBs all worked smarter and in a more cohesive fashion there would be less pressure on Health Boards/IJBs. This could result in fewer visits/inspections and a better outcome, in that, a more comprehensive overview could be achieved with less duplication of effort; work smarter rather than harder. Despite these good intentions resourcing might still curtail activity.

Collegiate Approach

Taking a collaborative and partnership based approach was seen as an important step, to help move away from silo-working both within (SBs) and between Health Boards/IJBs and SBs. The suggestion was that Health Boards/IJBs could:

‘Actually work together with the assurance bodies, to make it more of a process. We’re striving for improvement and quality and understand why this is important, but also think about it in that very collegiate, collaborative way.’ HB

This facilitative approach between Health Boards/IJBs and SBs could support staff to gain experience in working alongside one another in preparation for future scrutiny events:

‘One of the bodies (SB) did come to us to test a tool to see if it was fit for purpose and we were able to support them to see if it did what it aimed to do, which was good. As part of that process, our staff were getting a taste of what it was like to go through that process so I think it’s important to continue that partnership working.’ HB

An initiative by CI to employ ‘lay inspectors’ who have the opportunity to shadow and assist with inspections, in order to gain experience of the scrutiny process, is seen as an example of good practice.

Assurance Approach

Rights or Process Based

Differences in opinion emerged in relation to the focus of scrutiny activity, whether it ought to be rights based or process based. For example, should we scrutinise the way in which a person’s human rights are satisfied through receipt of health care, or assess the merits of the organisational processes that are followed in order to improve an individual’s mental health. A number of participants touched on this live issue, commenting on the tension between the two. The wider context is important because influences such as the Human Rights Bill (currently under review) may well influence and dictate that scrutiny will require being rights based. The impact on practice could be considerable. Whilst both were felt to be of importance, there seemed to be a conflict with it being one or the other approach or whether these could be complimentary to one another.

Context

A plea was made from a number of participants for those undertaking the scrutinising role to give greater consideration to the wider context within which any review/inspection is taking place and reflect this more strongly in reports. In situations where there has, for example, been a serious incident immediately before an inspection, then focus on the serious incident will take precedence and preparations for inspection will be hampered. The perceived inadequate consideration given to context was a major cause of angst for Health Boards, and high on the list for recommended changes:

‘If one thing comes out of this research, please make it acknowledgement of the context’. HB

From the perspective of people with lived experience, the current method of scrutinising services was perceived as failing to capture the context and impact on the person and their family in its entirety, focussing simply on an assessment at a particular time point (i.e. during in-patient stay). Being part of the mental health system can have an enduring effect on people and their families, sometimes extending from months to years. Respondents felt passionately about the fact that only a small part of any service is scrutinised if you set standards and measure the quality and value of in-patient service alone. The early intervention and follow up support form a hugely important part of the experience for people with lived experience of mental health or learning disability services. Service scrutiny should therefore be just as rigorous at primary care level and in the community. This is an identified gap in the provision of scrutiny, albeit it is on the current and future agenda of the scrutiny bodies.

As services progress, staff - at all levels - need to understand why governance and assurance is such an important and integral part of their work. It is more than just inspection and ticking boxes. Services are constantly evolving and there is a desire to fill gaps that have become more obvious over the years. All services have to be considered. There has to be more focus on what scrutiny and assurance looks like and how that is delivered in a consistent and a proportionate way going forward. Examples of other services where gaps exist include people with a learning disability and children’s services at the point of transition into adult services.

Differences in Awareness and Focus

There was a suggestion from some Health Board representatives that some people performing reviews and undertaking scrutiny of services are less aware of what is happening ‘on the ground’ in clinical practice, especially if they have been detached from services for a lengthy period. Questions arose about how those who are long out of service are able to assess what is relevant now. A respondent from one of the scrutiny bodies noted that:

‘There's a real gap between what’s happening in the thinking of organisations such as mine and what is happening on the ground’. SB

There was a perceived need for personnel working within scrutiny bodies to find the means to maintain close links with Health Boards/IJBs in order to maintain their knowledge and understanding of current clinical practice/social care related issues. Opportunity for engagement in relevant clinical professional development is another route through which this need could be met. An example of good practice was offered from respondents working in social care settings, whereby strategic inspectors meet monthly with the Social Care Clinical Advisors. It is unclear if this facility is available to all across Scotland.

Communication

To enable closer working ties and relationships, communication was considered key, and a mechanism for addressing disconnect between stakeholders. What is communicated, at all stages, needs to be in a language all can understand. Therefore, everyone involved, including people with lived experience and the public, understand the context, their role in events and the wider picture for moving forward:

‘If they (SBs) can provide user friendly communication then it doesn’t matter whether you’re a student, nursing assistant or director of nursing, but you’re fully cited on what’s happening. I think that will make the picture clearer for everybody and they can then understand the part they play in it’. HB

Issues relating to communication between professionals, people with lived experience and lay people emerge regularly in reviews and the importance of effective - user friendly - communication cannot be underplayed. Use of overly complex/technical language can cause division, so the necessity of communicating in a manner that everyone can understand is critical to successful communication.

Power

On occasion, it may feel like the scrutiny bodies are in a position of power, but the Commission, for example, has been described as a ‘critical friend’. The process of scrutiny is predominantly viewed as valuable and essential, with leaders recognising a need to embrace the concept, but equally noting the responsibility on the scrutinisers to work well with the Health Boards/IJBs to make it the best experience for everybody involved.

One of the SBs felt they were currently able to exert influence largely through their ‘soft power’ and were concerned that if they gained the ‘harder’ power (through statutory legislation) there would be a risk of losing influence through the current approach. The challenge is getting the balance right.

‘When you make something regulatory as in a set of rules, it can have the unintended consequence of driving the behaviours and the processes to make sure you meet the rules which is not in the patient interest or their families’. SB

The issue around powers was raised in the Scottish Mental Health Law Review (SMHLR) and Scrutiny Body respondents commented they do want more, because in those exceptional circumstances where they feel they are being are ignored, they need to have the necessary capabilities to enforce their recommendations. The hope is that actually having the powers will be sufficient for them not to use them.

Improving public awareness and increasing visibility of the different organisations in order to dispel any myths in relation to existing statutory powers was also viewed as critical.

Collaboration

Scrutiny Body respondents accepted that they do not always have the capability to achieve what they aspire to within their own organisation, sometimes due to limited powers, and have to consider who is the best organisation to support them at different point points in time:

‘I think that is where we lean on each other to good effect’. SB

There has been considerable effort to bring together groups in order to share intelligence and knowledge at a strategic level. One example of this is the Sharing Intelligence for Health and Care Group (SIHCG). Members of the SIHCG use it to share information and intelligence and while they reported that they find it useful; they accepted that the function and performance of the group requires further consideration and improvement. Data protection, for example, is perhaps used at times as a barrier to sharing information both within and between scrutiny bodies. Overall, the members of SIHCG appreciated attempts made by Government to take this in hand.

Respondents from Health Boards/IJBs were not always aware of the SIHCG in practice, nor did they fully understand the remit. There was a suggestion that a better facility to share information between regulators would probably pick up emerging issues and prevent hazardous situations occurring, and they could see a benefit if the focus was on mental health, rather on the much wider agenda assumed by SIHCG.

Partly as a response to the perceived gap in mental health, the Mental Health and Learning Disability National Scrutiny and Assurance Coordination Group (NSACG) was recently formed. It is an operational group that aims to improve the sharing of mental health specific information and coordinate hospital based inspections between existing scrutiny bodies. If the role and remit of this group was strengthened it could serve a more crucial function in joining up the work of the scrutiny bodies.

Lived Experience

Inclusivity and Meaningful Involvement

Without exception, all participants were very positive towards the idea of inclusion and working with people with lived experience. Examples of good practice were highlighted, such as MWC and CI having people with lived experience on their Board and the wider role of these representatives. Other scrutiny bodies do have an element of people with lived experience involvement and employ people with lived experience to assist with reviews, but it appears to be on a smaller scale and less well established.

The possibility of a hypothetical new scrutiny body led by people with lived experience was explored; the suggestion was welcomed with open arms by some and met with a fair degree of hesitancy by others. The concerns about involvement at a senior level raised by some senior staff within Health Boards and in some scrutiny bodies included: this might lead to individuals presenting only one view; singular past experiences of people with lived experience may overly affect their judgement and influence their opinion; it may overly stress the individuals and be counterproductive for people with lived experience. A representative from HBs was quite clear that people with lived experience are ready to and genuinely want to be working at a higher level within these influential organisations, and that it was an interesting concept to have their own organisation. There was a suggested compromise, rather than another new scrutiny body, to work within and improve involvement of people with lived experience within existing systems. A ‘test of change’ and establishment of proper support systems was seen as a plausible option.

Revaluing Lived Experience/Prominence of the Lived Experience Voice

Interviews found that the voice of people with lived experience needs to feature more prominently. Part of this issue was the perceived need to retain the fidelity of people’s opinions from the initial interview through to final reporting. In addition, it was important to keep the people with lived experience voice pure and not lost or amalgamated with other viewpoints. There was legitimate concern that this granularity and purity of voice often became lost or rephrased in the process. The fear was that reports that were written did not reflect their feelings.

Another aspect of revaluing was the inclusivity of people with lived experience in the scrutiny process to help close gaps in understanding and make sense of how people find services:

‘It’s amazing how we don’t have a snapshot sense of how people find services. It’s all very very detailed and complicated and becomes almost meaningless’. HB

Including people with lived experience was helpful for remembering what was important in terms of value when deciding on the unit of measurement. The inspections are particularly important in relation to how they involve local communities and individuals but scrutiny bodies need to be mindful that people are individuals, they live with their families, have friendships networks and communities, they do not live within a service.

‘We wouldn’t have a fire extinguisher sitting in our bedroom in our own home. We wouldn’t talk about who our carers are, we would talk about our friends and our family. We don’t have a menu at home. We talk about scrutinising for outcomes yet we professionalise care services in a language, a narrative, a behaviour that’s actually not the way we live our lives’. SB

There is a real opportunity for scrutiny to address this while still making an important contribution to improving the lives of people in Scotland.

Measuring and Reporting

Standards of measurement

There are many different ways in which the value, worth, effectiveness and efficiency of a service can be measured. The topic of what and how we measure is one that caused considerable debate. Respondents spoke of a focus on hard/tangible evidence such as waiting times and soft evidence such as people’s experiences of using the various services. The implication was that scrutiny bodies tend to measure what is easy to measure and not what is right to measure. Historically measurement was based on standards:

‘The workplace is clean, the workplace is safe, where we are now entering into are areas that are tougher, we’re talking about leadership, we’re talking about sustainability, effectiveness of care and these are perhaps more difficult judgements to make’. SB

Some agreed that information is captured well, but what emerges must be weighted differently, in order that is it presented in a more balanced manner, for example, to include a greater balance between professional and people with lived experience perspective. It was suggested that trust and empathy are not emphasised enough in reports; these are things that are not easy to change. The view that scrutiny bodies take is that trust comes from how they responsibly, fairly and transparently report what they find and do that in a way that is publicly available to service users. They want to be able to:

‘Report what they see without fear nor favour’. SB

However, it has all got be done in a responsible way that reflects the operating context of the service at that point in time.

What to measure

Respondents want clarity from Scottish Government on what to measure, what standards they expect and to see those standards being meaningful and not just about joined up care. There needs to be a shared framework that both service providers and SBs are working to, for example the new mental health standards that were published in September 2023.

‘We have a quality assurance system that informs all aspects of our work and again the key aim of that is to make sure while we’re carrying out a wide range of assurance activities, some of which are quite specialist and narrowly focused and others much broader, we’re doing so in as consistent way as possible’ SB

Respondents also want consistency of reporting where what was said during visits is reflected accurately in the reports, with no surprises.

Finally, respondents want to capture how people with lived experience feel about their care and treatment, so that it is focused on their outcomes and not just on the things that are easy for scrutiny bodies to measure. As it stands this has not been addressed and should be considered a gap.

‘What we don’t do is provide scrutiny framework that gives enough evidence around people’s happiness, wellbeing’ HB

This less tangible type of measurement could be built into a revised or refreshed set of standards that are truly focused on some of the softer elements that are very subjective to an individual, but it is actually, what makes us individuals. This is possibly a new direction for scrutiny and assurance that requires careful consideration.

Learning and Development

Learning opportunities

It became apparent through discussion that there are distinct gaps in knowledge at different levels. Within Health Boards/IJBs, there is an acceptance that staff require awareness training in governance and assurance related issues. They can take their guide from various sources, for example:

‘Carers Support and Support in Mind Scotland have got lots of experience and actually have been really, really valuable in making us think differently about how we engage and how we take things forward’. HB

The focus on education and training was also raised through conversation with HIS who wanted to focus peoples’ attention on learning from one another’s good practice with a focus on improvement. One Health Board had taken steps to set up a ‘mini review’ in preparation for an inspection, in order to aid understanding of the process, however this good practice did not appear to be widespread.

Supporting change

There is a suggestion that senior officials need to think about their reaction to scrutiny bodies picking up failings and shortcomings; the belief is that they need support to deal with this. There is a sense of frustration from scrutiny bodies who make recommendations for change in their reports, but these have not been enacted during follow up visits. They would like assurance that the changes will be implemented but are often met with ‘we’ve got this in hand’. Services argue the timeframes, advice and follow up support are not always appropriate or realistic, so how can they successfully implement the desired change.

Knowledge Transfer

Sharing the findings of reviews between services, scrutiny bodies and government support the notion of collaboration to address concerns through targeted scrutiny:

‘I think key to it is how we continue to collaborate with partner organisations but also with service providers and with government around the findings from scrutiny and thinking about where serious concerns or emergent concerns are and how we continue to target scrutiny to best effect’. SB

Connecting similar issues, situations or events together and thematically reporting them across scrutiny bodies via sharing of intelligence was considered useful for identifying and sharing key issues. The notion was not necessarily to identify themes by service or local area, but to aggregate up and possibly generate themes around topics such as governance, resourcing, leadership, education or training.

Encouraging shared learning and the identification of risk avoidance in systems was an important aspect of proactive learning across the scrutiny bodies:

‘So things like submitting all adverse incident reviews into the monthly group helps identify and encourage shared learning and avoidance of risk in our systems’. SB

Reporting aspects of services that are not working to their full potential could be impactful in their sharing of information. The importance of highlighting where things are not working well was stressed by respondents and the need to report this.

Data sharing systems

Improving the means through which data is accessed and shared is invaluable, it can expedite processes and prevent duplication of work:

‘There is the ability for public organisations like NES to work with others on sharing data via cloud services such as azure and amazon web services. So there’s opportunity there for us to get much better at scrutiny information, reporting, intelligence, reporting for government, blending resources to outcomes, that it doesn’t need individual scrutiny bodies to invest heavily in these new systems’. HB

It is clear that work needs to be done to ensure data can be accessed and shared more readily, perhaps through existing or similar systems, as highlighted by NES. This could assist report writing following joint visits and thematic reviews. There is an ongoing commitment to digital transformation and proposed investment from Scottish Government which may serve to address this issue.