Adult secondary mental health services: consultation analysis

9 Implementation and measurement

9.1 Introduction

The final part of the consultation sought initial views on what services will need do to implement the standards and how the Scottish Government might measure the standards. While the government welcomed everyone’s views across the whole consultation, this part of the consultation was considered to be of most interest to people with experience of working in or running services.

9.2 Question 45

Almost three-quarters (72%) of all consultation respondents answered Question 45 which asked respondents what support they think services would need to implement the standards.

The main themes are presented below, and points raised are reflected in the selected organisation quote below:

“The key elements are going to be guidance to show how this will look in practice, time for staff to engage with and understand the standards, training for key staff including board members and the resources, including staff resources to ensuring implementation. It will be difficult to achieve the standards without investment in recruitment and retention in key areas. There may also need to be some investment in physical buildings and infrastructure to improve patient experience. We would also encourage awareness raising activities as early as possible to support ongoing understanding and identify any issues or concerns”.

Scottish Public Services Ombudsman

Theme 1: Workforce expansion

Some respondents (individuals and all organisation categories) believe that there is a need to invest in the expansion of the mental health workforce in order to support the effective implementation of the standards.

A point raised by these respondents is that the implementation of the standards could result in additional administrative burden for the workforce. Many respondents reported that the mental health workforce are currently overworked and that additional duties could have negative consequences, including on service delivery.

Some respondents suggested that the recruitment of additional administrative staff could help to mitigate this issue, whilst others suggested the recruitment of additional clinical staff to ease workloads and therefore free up time for additional administrative duties.

Theme 2: Workforce development

Some respondents (individuals and all organisation sub-groups) highlight a need for relevant training to be provided for the mental health workforce to ensure that any data required is collected in the right way, is of high quality, and that a consistent approach is undertaken across services.

Related points raised by a few respondents include that there may need to be improvements in current IT systems used to help facilitate data collection and reporting (for example, a centralised system to help ensure a consistent approach).

Theme 3: Standards need to be sufficiently clear

Some respondents, including Health Boards, HSCPs and Services, suggest that it would be important for the standards to be sufficiently clear and capable of measurement in order to inform reporting.

Further, a few of these respondents raise related points, including that:

• the standards as presented in the consultation document are considered vague – and could benefit from being more specific and measurable
• the production of various sets of standards for different aspects of mental health services has the potential to lead to confusion – and a single set of system-wide standards may be a better approach

9.3 Question 46

Table 9.1 provides the quantitative response to Question 46.

Views are mixed. Half of all consultation respondents who answered Question 46 either agreed or strongly agreed (50%) that some of the standards should be measured using a validated self-assessment tool. A relatively large proportion of respondents are unsure (that is neither agree nor disagree with the proposal).

Base = 80 (individuals = 44 and organisations = 36)

Percentages may not total 100% due to rounding

Around two-thirds (68%) of all consultation respondents provided further qualitative feedback in response to Question 46, and the main themes are outlined below.

Respondents who agree with the proposal

Theme 1: Workforce involvement in the design process

A few respondents (individuals, mental health organisations and Health Boards, HSCPs and services) emphasised the importance of involving the mental health workforce in the design and development of any self-assessment tool. These respondents feel that workforce input to the process would increase buy-in and ownership of the final output, as well as add value to the process through gaining valuable knowledge, experience and insight from frontline workers.

A few of these respondents note in their consultation response that they have experience of similar approaches used in other areas – and which are considered to work well in practice. It is felt that lessons learned could be shared as part of the process of developing a validated self-assessment tool for some of the standards.

Theme 2: The tool should not replace external regulation

A few respondents (individuals, mental health and other organisations) were positive about the role of validated self-assessment – but emphasised that this should not be at the expense (or be a replacement for) the independent and external regulation of services.

Theme 3: A standardised self-assessment tool

A few respondents (individuals and Health Boards, HSCPs and services) highlighted that any self-assessment tool should be standardised to enable consistent application across mental health services. Standardisation is also viewed as essential in order to support meaningful comparisons between areas and services.

Respondents who disagree with the proposal

Theme 1: Mental health services are under too much pressure

A few individuals believe that mental health services are under too much pressure, and that the use of a validated self-assessment tool could have a detrimental impact on the capacity of services.

Theme 2: Data issues

A few respondents (individuals, mental health organisations and an organisation who supports specific target groups) raise some data related issues and concerns with a validated self-assessment tool, namely that:

• a lack of compliance with the tool could result in poor quality information, data gaps, and less meaningful results
• some organisations may wish to portray themselves in the best possible light – the importance of continuous improvement is therefore emphasised as important

Respondents who neither agree nor disagree with the proposal

Theme 1: Reiteration of previous points

Respondents who neither agree nor disagree with the proposal raised similar points to those outlined above. This includes that:

• a validated self-assessment too could lead to positive bias in reporting
• mental health services are under too much pressure and are time-poor
• any validated self-assessment tool should not replace the need for external regulation

9.4 Question 47

Table 9.2 provides the quantitative response to Question 47. This shows that three-quarters of all consultation respondents who answered Question 47 either agreed or strongly agreed (75%) that some of the standards should be measured using a range of indicators.

Base = 77 (individuals = 44 and organisations = 33)

Percentages may not total 100% due to rounding

9.5 Question 48

Around three-fifths (61%) of all consultation respondents provided further qualitative feedback in response to Question 47.

Respondents who agree with the proposal

Theme 1: It could help promote consistent service provision

Some respondents (individuals, Health Boards, HSCPs and services and mental health organisations) support the use of a range of indicators to measure some of the standards as the provision of a national set of standards could help to ensure consistent provision of quality care and support across Scotland.

Some data issues were raised, including:

• the ability to gain information and feedback from service users would need to be considered fully in relation to the ability to gather such information (Angus Health and Social Care Partnership)
• challenges may arise in terms of the sources of information, who has ownership of this and would this be accessible for the purposes of providing a response – for example, “there are multiple sources of data which may be held locally or nationally and may be under the auspices of health, social work, social care, third sector organisations etc. Indicators may need to take account of these factors” (South Lanarkshire Health and Social Care Partnership)

Respondents who disagree with the proposal

There are very few responses from respondents who disagree with the proposal, and no themes emerged from the feedback.

Respondents who neither agree nor disagree with the proposal

Theme 1: Indicators should recognise regional differences and diverse nature of people with lived experience of accessing mental health services

A few respondents (all individuals) are concerned that a set of national standards may not fully recognise or take into account regional differences – for example, differences between urban and rural areas, and differences in levels of deprivation.

A few Health Boards, HSCPs and services also note that the use of indicators should reflect the diverse nature of the client group served.

“…the need to use a range of approaches to evaluate service provision or outcomes for patients, including direct feedback on experience, but also including the use of reliable and valid measures, developed for a specific purpose e.g. assessment of severity of PTSD symptoms; and the ability to tailor choice of outcome measure used to the needs of the client group and their presenting difficulties. It therefore follows that the use of indicators should also reflect the diverse nature of the client group served. The indicators used should reflect the needs of the population served, based on a thorough assessment of this”.

NHS Greater Glasgow and Clyde

Theme 2: Mental health services are under too much pressure

Similar to previous questions, a few individual respondents believe mental health services are under too much pressure to be able to deal with this additional requirement.

9.6 Question 49

The Scottish Government included a few examples of ways it could measure the standards. They were keen to gather views about these initial suggestions, as well as any other suggestions respondents may have of how to evidence the successful implementation of the standards and measure progress against them.

In 2023, the Scottish Government will undertake further targeted engagement on measurement with key stakeholders following this consultation. Partners will have further opportunities to feed into this process of adapting and refining the self-assessment tool and indicators before they are agreed.

Please give us your views on these possible questions to include in the self-assessment. Please provide any further suggestions for self-assessment questions you may have.

Just over two-fifths (42%) of all consultation respondents answered Question 49 which asked for views on these possible questions to include in the self-assessment. It also asked respondents to provide any further suggestions for self-assessment questions.

Theme 1: Suggestions for additional questions

Some respondents (individuals, Health Boards, HSCPs and services and mental health organisations) provided suggestions for additional questions which could be included in the self-assessment question set.

A few examples include:

• current gaps in workforce/unfilled posts
• are there any non-mental health services that you have difficulty accessing for your patients (that is radiology, rehabilitation, neurology services)
• a question about how services determine an inappropriate referral, how this is communicated and what happens when there is disagreement about the most appropriate service to meet an individual’s needs
• time taken/spent with people
• how services ensure people are supported to make informed decisions about their needs and transitions of care

Please refer to Appendix F for more examples.

Theme 2: Supportive comments on the possible questions

A few respondents (individuals, Health Boards, HSCPs and services and health improvement organisations) provided positive and supportive comments about the possible self-assessment questions. For example, this ranges from “a good starting point” to “these seem reasonable”.

Theme 3: The questions are too complex and difficult to measure

There are also a few negative comments from respondents (individuals, Health Boards, HSCPs and services and organisations who support specific target groups).

The main points raised relate to: the complexity of the proposed questions; language is not considered user-friendly and could be simplified in some way; some of the questions are felt to be too vague and could be improved to ensure they are clear and measurable.

9.7 Question 50

Just less than two-fifths (38%) of all consultation respondents answered Question 50 which sought views on these suggestions for possible indicators to include. It also asked for any further suggestions for indicators.

Theme 1: Specific comments on the possible indicators

The suggestions for possible indicators attracted feedback from some respondents which is summarised below.

Indicator B – “Proportion of people who agree with the statement: “I was asked about my needs and my personal circumstances and requirements were considered in planning my care and treatment”:

A few comments are made about indicator B – feedback is mixed, with the most common message that this indicator could be further clarified.

Indicator C – “Proportion of people treated in the community compared to inpatient settings”:

This indicator attracted a few negative comments from respondents. Points raised include that: it is not clear what this indicator is trying to measure; and what would be a good or acceptable result (for example, a high level of community care indicate good quality community care or a lack of capacity within in-patient care).

Indicator D – “How long people stay in inpatient settings”:

There are also a few comments regarding indicator D, again mostly negative. For example, it is suggested that the indicator relating to length of in-patient stay may not foster a positive attitude towards in-patient care.

Indicator H - “Mental Health workforce staffing levels and skill mix”:

The feedback suggests that respondents are not sufficiently clear on what indicator H is trying to measure.

Theme 2: Suggestions for additional indicators

Some respondents (individuals, Health Boards, HSCPs and services, mental health and other organisations) provided suggestions for additional indicators which could be considered by the Scottish Government. By far the most common suggestion is to include waiting times as an indicator. Please see Appendix F for other suggestions made by respondents.

Theme 3: The indicators are too generic

A few respondents (individuals and Health Boards, HSCPs and services organisations) feel that the suggestions for possible indicators are generic and could benefit from being more specific in nature.

9.8 Question 51

The Scottish Government recognise that currently not everyone has the same experiences or outcomes when they engage with mental health services. We want these standards to help make sure that services meet your needs whoever you are and whatever your background. A total of 41% of all consultation respondents answered Question 51 which sought suggestions for how the Scottish Government could support services to reduce inequalities in the outcomes and experiences of people who use services, including in the measurement of the standards.

Theme 1: Increase staffing levels

A few respondents (individuals and a mental health organisation) believe that the mental health workforce needs to be expanded in order to ensure delivery and achievement of these standards.

Additional points

Please refer to Appendix F for more detail.