Extended Distress Brief Intervention Programme: evaluation

4 Findings

4.1 DBI flow diagram

* 1685 referrals made but only 1570 DBI interactions were completed during the May to December 2020 time frame.

** Includes cases where DBI was stopped due to escalating level of risk.

4.2 DBI delivery by Level 1 (NHS24)

4.2.1 Total uptake of Level 1 DBI service

Between 1 May 2020 and 31 December 2020, NHS24 undertook a Level 1 intervention with 1685 individuals. Referrals to all Level 2 organisations steadily rose over this period. Between mid-May to early-June, NHS24 made an average of 4 referrals per week; between early-June to mid-July, NHS24 made an average of 15 referrals per week; and by mid-October, the average referrals made by NHS24 had risen to 61 referrals per week. This in part reflects the evolution of the NHS24 Level 1 service from mid-May to mid-July (See section 2.2). However, the change in demand for the service may also reflect the changing restrictions around COVID-19, changes in the availability of other services or support for people in distress, and consequent changes in people's mental wellbeing.

4.2.2 Perceptions of the training received

Overall, training was well received, and attendees appreciated the enthusiasm and knowledge of the trainers. It served as a good introduction to what DBI is and Level 1 staff's role in providing the service.

"I was really sold by the training, I thought it was just great, a great thing to be able to offer people on the phones and I still definitely agree with that." [1104203_L1]

For some of the more experienced practitioners in the NHS24 Mental Health Hub, the level of training was too basic. Some felt there should be separate training for those more experienced, whilst others felt that the course served as a useful refresher of their existing knowledge.

"Yeah it was sort of pitched a bit low if I'm honest. I felt that, yeah, it was sort of more about, there was a lot of content about managing the distress which I thought obviously we were already doing, so it would've been I think more about referral criteria and also the background to how it came about was all very interesting, but in terms of the managing distress and experiences of distress, I felt it was probably, went on that a wee bit too long, it's more about the workings of the service and, yeah, it would've been better." [2209202_L1]

From the interviews, it was evident that not everyone received the training prior to working as a Level 1 practitioner. However, in our sample of respondents, this was not perceived as detrimental to the service as many felt learning on the job, doing their own research on the topic and working through training scenario calls were the best way to learn how to deliver Level 1 DBI. All interviewed felt confident in this approach to learning, knowing they could call on senior staff if unsure of how to proceed with a call.

"I felt alright probably because I had looked more in-depth into what the service is and what it provided, so it probably gave me, you know, the knowledge I suppose when I was making a referral for anyone or discussing a referral with anyone, I was able to explain what the service was and how it would assist them, so I felt okay yeah." [1609202_L1]

In addition, the supportive nature of the NHS24 Mental Health Hub environment, where PWPs could call on mental health nurse practitioners for advice and to make the final decision regarding referral, resulted in PWPs feeling confident in their DBI role.

4.2.3 Implementation of Level 1 service by NHS24

In general, participants felt that DBI aligned well with the existing processes followed by PWP staff and did not create an additional amount of work.

"It links up with the risk assessment, like always assessing risk in this job and I think once we've done it, it adds maybe another two or three minutes onto a call which is nothing, you know, it's a drop of water in the ocean really and absolutely something that's worthwhile, as long as it's therapeutic and it's worthwhile and it's helping them." [2610203_L1]

All Level 1 practitioners interviewed felt that DBI was a positive addition to the existing NHS24 Mental Health Hub service. The overarching opinion was that it allowed practitioners to provide a follow-up to the support provided, that they knew would be in place in a timely manner and would be helpful in supporting an individual with their distress.

"… I think it's a really good option, as I said, especially for catching people so they don't fall through the net, it's easy for us to say to somebody to go speak to their doctor, whether they actually do this and within mental health, you know, addictions, things like that, the uptake for people to actually go and do these things isn't very good..." [1409201_L1]

"…I know COVID limits this because of the face-to-face contact, the danger and people shielding, but even the telephone support cause, I mean, I have to say that quite a lot of our calls are the lost and lonely, people that are lonely and isolated, and even those calls can be appreciated by them to have somebody at the end of the phone, a good ear or sympathetic ear to do that." [1509201_L1]

The 24-hour referral process emphasised how DBI provided a much-needed alternative to existing signposting conducted by the Mental Health Hub.

"… it's like that piece of the jigsaw that's missing, yeah I feel as though it's good because I honestly think that it's the stepping stone that the patients need, you know, they don't always really need to jump to a community psychiatric nurse or, you know, they're in distress and someone's going to help them." [2109202_L1]

"… you know instantly when they've got X, Y and Z due to COVID, instantly you think of DBI and you're like 'yes, thank god' you've got something that you can give to them so it's really good." [1106201_L1]

Many of the practitioners interviewed discussed NHS24's capacity to deal with Level 1 calls. The rapidly expanding service, together with the increase in NHS24 Mental Health Hub availability, ensured that although practitioners felt busy, they felt demand for the service has been met adequately. However, some felt a reduction in waiting times for calls to be answered would be an improvement.

"It's manageable yeah at this point in time, yeah absolutely I think it's manageable. Yeah at points it's felt that, d'you know, the queue to get into the service was enormous but that probably happened a lot less than we anticipated I think, so I think we've managed it reasonably well. But I think in terms of the patient journey I think it would be better if we could get more staff to answer the phone a wee bit quicker because obviously you don't want someone waiting online and distressed for very long at all." [2209202_L1]

Feedback from Level 2 providers regarding Level 1 referrals was extremely positive in terms of appropriateness of referrals and level of details provided.

"Yeah they've been very good and very appropriate, yeah, any ones we've had have been absolutely spot on, good information and always appropriate yeah they've been helpful." [2809202_L2aU]

"I have to say that NHS24 referrals, the quality of them are really good. I don't know why that is but they're definitely lengthy and detailed and really accurate." [2209201_L2bU]

Several Level 1 practitioner interviewees felt that provision of a more detailed checklist as to what is and is not appropriate to refer to Level 2 would help them improve service delivery. They also stated they would benefit from further details about the role of Level 2 in DBI provision and more in-depth information around the whole DBI user journey, rather than just a focus on Level 1. Level 1 practitioner interviewees also stated that increased sharing of anonymised case studies and more team discussion would have supported them to deliver DBI.

"I would like to have a checklist or something or a flow diagram or something telling me is this appropriate/is it not and stuff. And hearing more personal stories about when it's worked…" [1609201_L1]

"Maybe at debriefs we could be having more of a chat about our experiences over the past few days, even if it was one night a week, with just DBI referrals and, you know, 'who did you refer, what was the reasons why?' you know, and that would probably be more helpful as well." [1409202_L1]

The way in which NHS24 deals with calls means that each call that is received can only result in a single outcome (e.g. referral to a service, advice etc). A few interviewees mentioned this as a limitation and felt that the option to refer individuals to DBI in addition to another signposted service would be helpful.

"I know I've had a few that I've thought 'that would've been perfect for a DBI but I'm going to have to get him to contact his doctor tomorrow so I can't', whereas if they say to you 'I'm contacting my doctor' you're like 'oh I can do DBI for you'." [1104202_L1]

"There's been plenty times where I've thought DBI would be appropriate but obviously we only have that one outcome, and I've got to deal with that more immediate situation first and foremost, … there is definitely times where I think that it would be beneficial to be able to have two outcomes just so that somebody's getting all the support that they could potentially be needing." [2710203_L1]

4.2.4 Individuals' perceptions of compassionate care at DBI Level 1

Out of the 1085 Level 2 practitioners responses received to the routine evaluation question, 87% of individuals rated the NHS24 Level 1 service as 8/10 or above (very compassionate-completely compassionate) for compassionate response. Only 1% of respondents rated the service as being not very/not at all compassionate (See Appendix 1, A1). This finding was supported by the interviews with individuals using the service.

"…the chap on the phone was brilliant, you know, he was very conscientious and I felt listened to, I felt heard more than anything, that was… you know, and as stupid as I felt for having this issue at all never mind… you know, having to ask for help with it, but I didn't feel stupid is what I'm trying to say." [1112201_L2dU]

"I can't remember the woman's name, but she really had a lot of time for me. I know there are people who are physically ill with COVID symptoms, but they really did make time for someone with mental health as well. [120120_L2aU]

"the girl that I spoke to and I can't remember her name now, she actually really cared, she did really care and that spoke volumes to me because apart from the girl in the NHS nobody else really did." [1112203_L2cU]

4.2.5 Impact of DBI Level 1 on individuals' perceived ability to manage their distress?

There were 1076 responses to the routine evaluation question about individuals' perceived ability to manage their distress. Most individuals (80%) felt they were fairly–completely able (score of 6-10 on distress thermometer) to manage their current level of distress following their interaction with the NHS24 Level 1 practitioner. Some (6%) felt they struggled (score 1-3 on distress thermometer) to manage their current level of distress, while a lesser number (3%) felt they could not manage their distress at all (score of 0 on distress scale) (Appendix 1, A2).

This finding was supported by what the individuals who received DBI told us when we interviewed them.

"They were really, really helpful. They listened, they allowed me to speak, they really helped me calm down…I just remember feeling really supported and they really helped to calm me down, and then they offered a few different routes, one of them being the distress team." [1109202_L2cU]

4.2.6 Implementation contexts of the Level 1 DBI service

A Level 1 intervention was provided to 1688 people during the evaluation period. The numbers of interventions in the areas of largest urban density were broadly proportional according to their population. A quarter (25%) of these interventions were from the NHS Greater Glasgow and Clyde Health Board area (35 people per 100,000 population). A further 17% were living in NHS Lothian (36 people per 100,000 population) and 12% were in NHS Lanarkshire (36 people per 100,000 population)^[4]. A further 45% of calls were spread across the remaining 10 Health Board areas, with 1% of cases where the Health Board was not recorded (Appendix 1, A3).

Penumbra and SAMH received by far the largest proportion of referrals (41% and 41% respectively), reflecting the multiple Health Board areas that these two organisations provided DBI support for. (Appendix 1, A4 further details the Health Boards covered by each Level 2 provider. Appendix 1, A5 provides a breakdown of referrals each Level 2 provider received across each month of the evaluation period.)

Characteristics of individuals receiving Level 1

The age distribution of people who interacted with Level 1 is strongly skewed towards the younger age groups: 23% were aged between 16 and 24; 27% were aged between 25 and 34; 19% were aged between 35 and 44 and 16% were aged between 45 and 54. Overall, roughly equal proportions of males (49%) and females (51%) received a DBI Level 1. However, there was a slight variation between male and female for across the age ranges (16-24: 52% women; 25-34: 45% female, 35-44: 44% female; 45-54: 54% female; 55-64: 55% female; 65-75: 53% female). Data is further grouped by male/female (Appendix 1, A6). Most Level 1 contacts came from more socially deprived areas: 35% came from the most deprived quintile (SIMD 1), compared to 9% from the least deprived quintile (SIMD 5) (Appendix 1, A7).

Recording of alcohol or substance use between males and females was evenly split across all NHS Health Boards. NHS Borders, Grampian, Lothian, Orkney and Western Isles had less than 10% of individuals presenting with alcohol or substance use at Level 1. NHS Dumfries & Galloway recorded the highest incidence of individuals presenting with alcohol or substance use at Level 1 (29%). The remaining Health Boards recorded between 12% and 16% of individuals presenting with alcohol or substance use at Level 1. A breakdown of individual Health Board statistics is provided in Appendix 1, A8.

4.2.7 Presenting problems and contributory factors reported at Level 1

Individuals interacting with the Level 1 service presented with multiple problems. The main presenting problems were depression/low mood (68%), stress/anxiety (61%) and suicidal thoughts (48%). A breakdown of presenting problems experienced and the number of individuals presenting with each problem is provided in Appendix 1, A9. These three problems were consistently reported across all age ranges and with little variation between men and women or by area-based measure of deprivation (SIMD).

Individuals interacting with the Level 1 service were also asked to list contributory factors to their anxiety/distress. A breakdown of contributory factors is presented in Appendix 1, A10. Relationship issues (32%), life coping issues (28%), money worries (26%) and employment issues (25%) were the most mentioned contributory factors. Across the four main contributory factors, there was a slight variation in the proportion of contributory factors reported by gender. Men were slightly more likely than women to report money worries and employment issues (59% and 58% respectively), while relationships and life coping factors were reported fairly evenly by men and women (49% and 48% respectively).There was minimal variation (5% at most) in the presentation of contributory factors according to social deprivation (SIMD).

4.2.8 Distress level reported at NHS24 Level 1

The majority of individuals reported that their distress levels (as measured using the 10-point Distress Thermometer (1 = low, 10 = high) were moderate to high at Level 1. Where a distress rating score was recorded (n=1,213), 77% of respondents presented with a distress score of 6 or above (Appendix 1, A11). Referral to Level 2

4.2.9 Contact with 24 hours

Overall, 100% of people referred from Level 1 were attempted to be contacted within 24 hours, with 85% of people successfully contacted by a Level 2 organisation within this time period. This rose to 94% in the days that followed. While there was some variation in successful contacts by Level 2 provider, a minimum of 63% of all DBI cases were successfully contacted within the 24-hour time frame. Further details are provided in Appendix 1, A12 (grouped by Health Board) and A13 (grouped by Level 2 provider). A few individuals (6%) did not respond to contact (either phone call or letter) to engage in the Level 2 service. There appeared to be no common characteristics across these individuals. Both Level 2 practitioners and individuals who received DBI recognised the importance of quick contact.

"I think it's really important because I think that's you catching them in that point of distress and through my own experience I know what it's been like talking about mental health and once you get it out there's sometimes a part of you that doesn't really want to go over it again, so I can see there being quite an avoidance if we had to leave it any later to try and contact they people. So we are very strict about getting them within that 24 hours." [2409201_L2b]

"Well it made me realise that they hadn't forgotten and that they actually were doing what they said they would do. She spent a long time with me on the phone the first time she phoned, she listened to what I had to say and she was wonderful, she really was. She made me feel… she made me feel so good." [1112203_L2cU]

4.2.10 Uptake of referrals to Level 2

Uptake of the DBI service between May and December 2020 is shown in the DBI flowchart (Figure 1) with percentages of completed cases by Health Board area presented in Appendix 1, A14. Most individuals (94%) who were referred between May and December 2020 were contacted successfully.

Most individuals (79%) received between 1 and 7 (inclusively) Level 2 sessions (see Appendix 1, A15).

Of the 202 individuals who received one supportive phone call from a Level 2 provider but did not wish to further engage in a Level 2 intervention, 25% of individuals were recorded as feeling that DBI is not what they needed. Some individuals (11%) did not take up DBI support as they stated they were receiving support from elsewhere. Others (18%) accepted the first session but then did not attend further sessions (Appendix 1, A16).

4.3 Level 2 DBI delivery

4.3.1 Key variations in patterns of engagement at Level 2

In 10 out of 14 Health Boards, the average (mean) length of engagement for individuals receiving Level 2 support was 14 days. Overall, 80% of referrals to Level 2 involved contact of 14 days of less. Engagement, measured in the percentage of days individuals engaged with DBI varied by Heath Board area (Appendix 1, A17).

4.4 DBI Level 2 impact

This section presents findings on the impact of DBI on an individual's level of distress, what happens to people after they had taken part in DBI, including perceptions of a compassionate response, and the impact that Level 2 intervention had on individuals' self-management of distress.

4.4.1 The impact of DBI Level 2 intervention on individuals' levels of distress and wellbeing

Individual distress scores were measured at Level 1, start of Level 2 and end of Level 2. The average changes in distress score at each of these time points, grouped according to reason for stopping DBI are provided in Appendix 1, A18.

The largest reduction in distress (as measured by the distress thermometer) occurred in those who engaged in Level 2 support and had a planned discharge/closure. At Level 1, the mean score for this group was 8. By the end of their Level 2 interaction, there was a mean reduction of 4. Smaller reductions in distress ratings were found for the following groups: individuals who engaged in initial supportive phone call but declined further support (average reduction 2), and individuals who engaged with Level 2 support with unplanned discharge/closure (average reduction 2). Distress increased in individuals that were classified by the Level 2 practitioner as inappropriate referrals (average increase 2) or when DBI was stopped due to escalating risk (average increase 1).

Examining individuals' change in distress score from the start of Level 2 to end of Level 2 interaction, 76% of individuals' distress ratings decreased by at least 1 point; 8% of individuals' distress ratings increased by at least 1 point and 8% of individuals had no change in their distress rating (Appendix 1, A19).

People engaging in 13 or 14 DBI sessions (1% of individuals) recorded the highest average positive change in distress score (average improvement of 4 points) from start to end of Level 2. Receiving from 1 to 7 sessions (79% of participants) resulted at best in an average improvement of 3 points (when 7 sessions were received). There was a downwards trend in average improvement as the number of sessions received decreased (Appendix 1, A20).

The mean change in distress rating indicates that an improvement was seen for most people across all presenting problems. Except for a very low number of people who presented with specific problems (where at best, no change in distress rating scale was experienced), the mean improvement in distress score was at least 2 points. The wide variation in minimum and maximum distress score changes indicates that, for some, DBI had an extremely positive (improvement of up to 10 points) or very negative (got worse by up to 9 points) impact on their distress rating (Appendix 1, A21).

For people presenting with one of the 4 most common contributory factors, the mean improvement in distress score recorded from the start to end of Level 2 interaction was 2 points or more. However, for those with a contributory factor of homelessness, communication difficulties or anger problems their mean improvement in distress score was between 1 and 2 points. As with the presenting problem data, there is wide variation in minimum and maximum distress score changes relating to contributory factors: for some, DBI had extremely positive (up to 10 point change) or very negative (up to 9 point change) impact on their distress ratings (Appendix 1, A22).

We analysed the data to identify if there were any relationships between how individuals' distress ratings changed according to their gender, Health Board area, and SIMD quintile. There was no evidence to suggest distress rating changes were influenced by these factors.

4.4.2 What happens to people after they have taken part in DBI?

70% of people (1099/1570) were recorded as exiting DBI with a planned discharge. 13% of individuals (204/1570) engaged in one initial supportive phone call but then declined further support. Details of completion reasons for all participants are provided in Appendix 1, A23.

Individuals who received a planned DBI discharge were signposted to several non-statutory and statutory organisations. Level 2 practitioners recorded this information as free text which made it difficult to analyse. Specific key terms along with wildcard characters were used to gather the data on signposted services to develop high-level statistics on services referred to. The largest number of individuals, 25%, were signposted to their General Practitioner (statutory service), 20% were referred to self-help websites/apps (non-statutory), and 27% were referred to helplines (non-statutory). Practitioners also signposted people to a range of local non-statutory services that differed by geographical region.

4.4.3 Impact of DBI Level 2 intervention on individuals' self-management of distress

Statistics on a respondent's ability to manage immediate distress were analysed and grouped according to reason for terminating the DBI (Appendix 1, A24). Any engagement with the Level 2 service resulted in an average score of 6-7 (out of 10 points) or above for ability to manage immediate distress. Full engagement in Level 2, including a planned discharge, had the highest average score for ability to manage distress (a score of 8 out of 10).

Any engagement with the Level 2 service resulted in individuals having a considerably higher average ability (between 6 and 9 points) for managing future distress. Those individuals who had fully engaged in Level 2 support, including planned discharge, had the highest average score (8 out of 10 points). Statistics on ability to manage immediate distress, grouped by Level 2 completion reason are provided in Appendix 1, A25.

4.5 Impact of COVID-19 on DBI provision and service use

4.5.1 Mode of delivery, implementation challenges, and the impact of COVID-19 on practitioners and individuals presenting with distress

As a consequence of COVID-19 restrictions, Level 2 services were remotely delivered rather than providing face-to-face support as was the case in the pilot DBI areas prior to the pandemic. Appendix 1, A26 provides data on the mode of delivery used by each Level 2 provider. Ninety-six percent of all Level 2 sessions were delivered via the telephone. Video conferencing was not used for any sessions delivered by LAMH or TRFS, and the remaining organisations only used it for less than 1% of all sessions. Other modes of delivery (such as text messaging) were used in a maximum of 3% of sessions across the different Level 2 organisations.

There were mixed views on the preferred medium to deliver or receive the DBI support. Some individuals that used the Level 2 service indicated they would have liked using video conference, whereas others appreciated the anonymity and convenience a phone call provided.

"I think it would've been yeah, I think it would've been nice to actually see the person I was talking to, like I said it just sort of gives you that bit of a bond with them especially when you are sort of in that position and you are opening up, I think it helps if you're actually able to see the person." [1104201_ L2d]

"I found the telephone calls really, really helpful for that specific reason because I didn't feel judged, I didn't feel… I felt stupid saying the things that I was saying but I still felt like I could say them because I couldn't see the look on that person's face, you know." [1112201_L2dU]

"So, it's quite nice that they [individuals receiving Level 2] can actually sit in the comfort of their home and they don't need to get out of their jammies or do anything like that, they can keep their blinds shut and still just answer the phone, so I think there's been a wee bit more engagement there as well." [2209201_ L2b]

Some Level 2 practitioners felt that individuals who received DBI remotely engaged a lot more with the service than when support was provided by other modes of delivery.

"…the engagement rate of clients wanting to be supported and to engage with us has improved and gone up over lockdown, or over the COVID period, because if a client's anxious, just leaving their own house and coming into town for an appointment might be just that bit too much for them that day, you know, whereas sitting on their couch with their own cup of tea and familiar things around them speaking to somebody on the phone is not quite such a daunting task, and we've found that the engagement level is much higher I think over the last few months than it would've been in the office." [2610201_ L2e]

Many Level 2 providers described the challenge of supporting an individual over the telephone rather than face-to-face. Practitioners mentioned the increased importance of being able to pick up nuances in voices and interpreting silence when no visual or physical cues were present as with a face-to-face interaction.

"it's the same job that we're doing but just over the phone which is a shame because I think that one to one, and I think that that contact as well is quite important because you can't really judge people's reactions or you can't really judge what's going on just through talking to them over the phone, you know, I think there has to be some sort of human contact there." [2902202_ L2a]

"So initially when we moved to working from home and doing telephone support I really struggled with it, you know, I just missed that human contact and you're able to read that sort of body language and stuff like that, you know, but it took me a good three or four weeks to try and I guess find my way of doing it as well." [2209201_ L2b]

In addition, practitioners felt it was more difficult to judge the appropriateness of the situation for having a supportive conversation, where they had to rely on the individual to judge if the environment was 'safe' to engage in the DBI intervention.

"I think the first few times are the most nerve-wracking and also cause you don't necessarily know a lot about the person, you don't know if they're, I mean, you ask them 'is this okay, are you in a safe place, is this convenient, bla, bla, bla' but you can't tell if there's anybody else in the room or if there are children, I mean, I have started a conversation and then heard very young children in the background and gone 'look, I'll phone you back, when's a good time' because I knew we were going to go into a subject matter that would not be good for those children." [1102202_ L2e]

One aspect of virtual support that can be viewed as both an advantage and a disadvantage is the ability to support more people per day than with face-to-face services. Practitioners felt that although supporting more individuals was undoubtedly advantageous, they had to be careful to ensure they still had time for themselves for aspects such as admin, getting support after difficult conversations and ensuring boundaries between work and home life.

"If I was doing face-to-face appointments I'm in my car most of the day, the maximum I would see in a day is three people due to travel across [DBI geographical area], but I can talk to maybe seven or eight, maybe more clients a day sitting in the house, so I prefer that side of it, I'm getting to support a lot more people." [2409201_ L2b]

"But I have found it actually works well now, you know, because the amount of community visits and office visits that we would set up and then because they would DNA, they would not attend, but we would pretty much just get a big cream pie in the face, you know [laugh] all that time wasted driving all the way through [DBI geographical area] to meet in cafes and libraries and people wouldn't show, and we get it as well because of their anxiety and they may be just couldn't force themselves to go out." [2209201_ L2b]

"So, each morning we have a chat on Teams just checking with each other and [our manager] frees up time at the end of the day if we need to speak to each other and you just kinda have to get used to email and texting more or calling if you want to speak to someone. But I suppose it's not the same as if you're in the office and you've maybe had a difficult call, you can't just instantly offload to someone, so it's not the same in that sense." [2809201_L2d]

Overall, there was a feeling that both face-to-face and virtual sessions had different advantages/ disadvantages.

"I think it would be good to have a bit of a hybrid to be honest and for some people it seems to have worked, both clients and practitioners providing support over the phone, but for some, again both clients and practitioners, they're really dying to get back to face-to-face. Personally speaking I would like, as I said, a bit of a hybrid. I would like the flexibility to be able to support clients over the phone but it is nice to have that, you know, for clients that feel that they would benefit more from it, I think it would be nice to have that option of seeing them face-to-face, yeah." [2610201_ L2e]

4.5.2 Impact of COVID-19 on implementation of the expanded DBI programme.

A key component of DBI is the joint development and completion of a Distress Management Plan (D-MaP). It was unclear how completion of a D-Map would be affected by the intervention not being face-to-face. For those who engaged and had planned discharge/closure, D-MaP completion was almost 100%. Similarly, those who engaged, either in one supportive phone call or with an unplanned discharge/closure had D-MaP completion rates of 60% and 88% respectively (Appendix 1, A27).

Despite the challenge of not being able to complete the D-MaP in a collaborative, face-to-face environment, the general view of Level 2 practitioner interviewees was that they were able to adapt to the situation and still ensure the D-MaP was an integral part of the support process.

"…it's obviously better if it's a more kind of collaborative approach to doing a distress management plan but for some people it's hard over the phone, and some people aren't that really kind of aware about their mental health and the ins and outs of it and what's producing their problems for them either so that does make it a bit harder over the phone, yeah." [1410202_ L2d]

"I always say in the first call when I'm explaining it, it's completely up to them how they want to tackle it, so I always say 'if you would like me to start it and you to have a look at what I've inputted?' or 'if you want to crack on and go for it and put in your thoughts, feelings, emotions, coping mechanisms, if you want to handle that that's totally fine', or 'if you wanted to talk about it during our phone calls, kinda working through it that way' but I usually do try and send it to them throughout, even if I am doing it I want them to see it." [2810203_ L2d]

People who received a DBI were asked "Prior to presenting in distress this time was there ever a time when you presented in distress in the past to A&E, the police, the ambulance service or your GP?" If there had been a previous presentation of distress, then they were then asked by Level 2 providers "Thinking about your overall experience this time, how do you feel that the overall response this time compared to your previous experiences?" In total, 425 provided a comparable rating for the DBI service. For individuals that engaged in Level 2 with a planned discharge/closure, 92% (n=390) felt the service was an improvement on previous interactions with frontline services for distress. Within this category, half (n=196) gave a comparable response rating score of 10 out of 10, indicating their perception that the Level 2 provided a much better response than their previous experience(s) of presenting to a frontline service (Appendix 1, A28).

Of individuals that engaged in Level 2 that had an unplanned discharge/closure and reported a comparable rating (n=17), most felt the service was an improvement on previous interactions with frontline services for distress, with some of these individuals providing a comparable response rating score of 10. Individuals choosing to receive one supportive phone call and providing a comparable rating (n=4) rated the service the same as (n=1) or better than previous interactions with services for distress. Where the referral was classed as inappropriate and a response rating was provided (n=2), a comparable rating score was given of no change or slightly worse than previous interactions with frontline services.

4.5.3 Impact of COVID-19 on practitioners and individuals presenting with distress

Only 8% of individuals questioned about their distress at Level 1, referred to COVID-19 as a factor contributing to their distress. However, many Level 2 providers felt that while individuals did not necessarily state that COVID-19 was contributing to their distress, when explored further, the pandemic, associated restrictions and closure of other supportive organisations had an indirect impact on the distress they were experiencing.

"… seems there's more kind of generalised anxiety, a bit more kind of, you know, 'I've not felt like this before'. It's almost as though, if you imagine you're a person who just normally gets generalised anxiety where little things have a big impact, you know, I think there's just more of that, even general people who you work with, there's more just generalised anxiety that's around, yeah. So it's not been like, you know, loads of people have died and just more this like low hum which builds up over time." [1110202_ L1]

"…it's not necessarily the presenting factor or even a contributory factor that's noted in the database but I think just people's normal coping mechanisms they're being hindered by COVID. So the things that you could normally say 'oh I would normally go and meet my friend' or 'I would normally just go and hit the gym' or, you know, 'I would do whatever', people haven't been able to do that so some cases it's taken them a lot longer to get back on their feet or they're just kinda scrabbling about because they don't really know what supports to turn to." [2909201_ L2b]

Responses from individuals using the DBI service also reflected that much of their distress and COVID-19 related anxiety stemmed from them not being able to do what they usually do, or not having contact with people or services that they usually have access to.

"I was volunteering so that was good, that was like a social thing I was going that, yeah, obviously that had to stop." [1120201U_ L2dU]

"I now know what's wrong with me but I struggled cause of COVID to get to the right places to get the right information." [110420_ L2fU]

"I used to have a routine where I would see my oldest son at three at a contact centre, and then my other son that was through social work and they had their own building as well, so I kind of had a routine and that's all, like, stopped now cause there's COVID." [1112202_L2dU]

"I can't hack this, I've been told there's the possibility of a second lockdown … I don't think I'll survive a second one mentally because a lot of things have happened to me the last four years, … you know, I haven't had closure on any of that, you know, I've just had to, like, tough it out and get on with it. And then when COVID hit you know, you're like god it's never ending, you know, you start to question your faith, you think is this a punishment, you know, like the beauty of life." [110420_ L2fU]

DBI practitioners gave a wide range of responses when asked about whether COVID-19 had impacted on some groups more than others. Level 2 practitioners felt the pandemic had impacted on everyone in different ways and did not mention one group over others. Practitioner interviews uncovered a common perception that COVID-19 had increased levels of anxiety and distress, both for individuals with pre-existing problems and people who had not previously struggled with their mental health. As mentioned above, for individuals with existing mental health problems, their distress and anxiety were often exacerbated by not being able to access their usual support mechanisms and services.

"I think COVID perhaps has exacerbated it a bit certainly, if we think of patients would call with a diagnosis of personality disorder or anxiety, I think they have primarily been the focus of calls that are received by NHS24, and I think there is an increase in that and part of that has been due to, and they'll admit it themselves, they've got less input from their Community Mental Health Teams, they're not seeing their Community Psychiatric Nurse as often, maybe their clinic review with their consultant has been delayed or postponed for a period of time, and so they are struggling and certainly that patient group would be the patients that I speak to most and I would definitely say there's been an increase in them phoning in and perhaps the levels of anxiety and behaviours they're experiencing and displaying." [1102203_L1]

For individuals who reported not previously accessing services for distress or anxiety, the pandemic and associated restrictions evoked emotions that they had not experienced before.

"Certainly I've had a lot of people who, when we ask that question at the end, you know 'have you ever presented in distress before?' there's been a lot of no's, and they wouldn't ever have seen themselves as ever needing to be in this position; so I think COVID has surprised them as well and how they've reacted, and I have had one person say that to me, you know 'I'm usually a really happy easy going person, I never thought I'd be in this position'. Yeah, it's almost like something came left-field and hit them because of COVID." [1102202_ L2e]

"I'm stuck in my own world, COVID has meant I've not been able to see family members and meant I wasn't able to say goodbye to my dying grandad at the start of the year 'cause the borders, he lives in a different country and they had just shut the borders, …there was just a lot of events, like, family, like, big deal events that sadly I wasn't able to do and that really sucks." [1109202_ L2cU]

"Normally I would go tell my mother or something like that or my partner, my partner already knows but she's in the same boat as me because she can't do nothing about it either, COVID just wrecked everything, but you know, if there was something really troubling me sometimes I would go down and discuss it with my mum after discussing it with my partner." [110420_ L2fU]

Level 2 practitioners reported that their services were quick to adapt and provide virtually what would normally be face-to-face services.

"I think in time we have adapted and some of the services have adapted too and found ways to work on materials remotely. Look up maybe activities that they can do at home and thankfully there was a couple of creative organisations that still operated online and they would offer the groups on Zoom and they were sending materials to people so there was some opportunities there." [3009203_ L2c]

Level 2 practitioners recognised the impact that COVID-19 had on local group availability and the informal support and a sense of community these services provide to people in distress.

"I don't know that people have kinda specifically said to me they're missing their groups at the minute but I'm missing it because I'm missing, like, I would like to be able to say 'oh it would be really good for you just to nip round there once a week, why don't you nip down and a get a wee cup of tea, it's a lovely wee community café' so a lot of our tools are kinda cut off and you're just having to say 'yeah, what about going for a walk or what about speaking to so and so?' so your hands are a wee bit tied in terms of the breadth of support you can offer people, but I would say probably it's the natural supports people are missing and the change to their living situations that's impacting a wee bit more as well." [2902201_ L2b]

4.6 DBI and wider service provision

This section summarises practitioners' and individuals' views about the value of DBI and how it fits with wider mental health service provision.

4.6.1 DBI – fit for purpose during COVID-19 and beyond

There was widespread consensus among Level 1 and Level 2 practitioners about the value of the DBI. Practitioners highlighted aspects they felt were important, including the aim to contact individuals within 24-hours of being referred to Level 2, offering up to 14 days of support, the value of completing a D-MaP and signposting to other support organisations.

"There definitely is a gap for people that don't necessarily have the mental health diagnosis but are in distress and they need help and they need just encouragement and support to self-manage out of the situation. So, I think definitely it fits what its purpose, what it's meant to achieve." [2610201_ L2e]

Some Level 1 practitioner interviewees felt that some individuals were being referred to DBI because it meant they received support quicker, when referral to statutory agencies may have longer waiting times.

"… to be honest with you it's the expediency as well of the service. I mean, they'll call people within 24 hours which I think is very good because you know referrals to anything can take months. I mean, if you get referred to CAMHS [Child and Adolescent Mental Health Services] tomorrow you're talking about two months before you even now get your first appointment. I'm not saying that they can offer a service immediately right away but having that contact within the first 24 hours. What the patients that have had DBI are telling us is what they appreciate is that speed, people getting in contact with them right away." [1509201_L1]

Despite moving to a telehealth rather than face-to-face service, Level 1 and Level 2 practitioners felt that the service adapted well to the circumstances and continued to be a helpful means of supporting those with anxiety and distress.

4.6.2 DBI and wider mental health service provision

While the practitioners we interviewed overwhelmingly supported the DBI service, they did raise several issues regarding how the service operated within the wider sphere of mental health provision.

Firstly, they raised concerns about DBI being used inappropriately, where individuals with long-standing mental health issues or problems that are outwith the remit of DBI were being referred to the service and could lead to an increase in distress. Although some Level 2 practitioners were trained in mental health, some were not, and felt uncomfortable/unqualified to deal with some of the problems presented. Inappropriate referrals were also frustrating for the individual and devalued practitioner perceptions of the usefulness of DBI.

"We have some people that have some really pretty serious mental health issues and I would explain to them that we can't work with those issues because we're not medically trained but, you know, we can work with the distress round about it and we can try and teach you some healthy coping mechanisms and things like that." [2809202_ L2a]

"But that's not what the service is for, we're not there to fix Post Traumatic Stress Disorder and deal with personality disorders and we make that clear to the team members, that as long as we know what our service is and what we're able to do and able to provide then it shouldn't, you know, if somebody has… I'm just thinking of a client that's got kind of like nine personality disorders and has been through every service, that shouldn't make us feel that we're out of our depth and intimidated." [2610201_ L2e]

"And it's really kinda frustrating for us as well and puts us under a lot of pressure because we're not medically trained, so if we've got somebody on the phone threatening to kill themselves, phoning the community mental health team and they're like that 'they're not open to psychiatry so we can't do anything about it, send them to their GP or the hospital' and you're like that 'oh god come on, youse are the ones that's trained to deal with this, we're not." [2810201_ L2f]

In some cases, individuals were referred to DBI as an available source of support until services that were more appropriate became available.

"I did need to speak to my GP urgently but they weren't interested, so working with the mental health nurse at NHS24 said 'well we've got this new service, it's fandabbydozy [brilliant], we know it's not what you need at the moment but we're going to recommend you for it, put you through for it anyway because you need something, you're in distress, you need something' and that's how I came to be on the thing. So basically, by the time I was due to come off, is it two weeks the DBI is over? By the time I was due to come off that I was due to get my doctor's appointment just after that, so they delayed the last appointment till after I had actually spoke to the doctors to make sure they were actually doing something." [2909201_ L2bU]

The demand on wider service provision, including mental health services, impacted on the role of DBI. As highlighted above, due to the quick turnaround of a DBI referral, individuals were sometimes referred to DBI because referral to another, arguably more appropriate service, had a long waiting list. For individuals with anxiety or distress, having to wait a long time for support was perceived to have considerable consequences.

"… speaking to the right person at the right time, that's fundamental and I kept on getting told it's resource and fair enough, I understand that, yeah, I think if I was… I mean, nothing was suicidal or anything like that but I think if you were really at the limit, I think that would be a risk cause you are waiting three weeks' type thing." [281020_ L2bU]

"I made a referral to [support organisation's name] the other week there for someone who was very isolated and struggling with low mood and having suicidal thoughts and so on, but even with them there was a waiting list, so even though that's what they do, they signpost people to social activities and help them find groups, they're not that well-funded to keep up with the demand, you know." [1410202_ L2d]

"Personally speaking I think that that's investment that needs to take place in other services because I think managing distress in itself is quite a big thing and I think that adding to what we already do almost kind of doubts the management of the distress part of it. So I think if there are services, and there are, that support and manage the mental health side of things that that's where the investment needs to be for those particular services, and that DBI should continue providing distress management rather than kind of diluting and adding to it I think." [2610201_ L2e]

4.7 Comparisons with main DBI study

Concurrent with this evaluation, a larger and more comprehensive evaluation of the main DBI pilot programme was being carried out Distress Brief Intervention Pilot Programme evaluation: findings report - gov.scot (www.gov.scot). The main evaluation drew on a wider range of data sources, collected over a longer period, including outcome data that was independently gathered on the individuals receiving a Level 2 intervention. Consequently, the findings of the larger evaluation are more in-depth. Notwithstanding these differences, it is possible to broadly compare the high-level findings across these two evaluations.

As far as can be ascertained using different methods across each study, the changes in: Level 1 service provision – from Police Scotland, the Scottish Ambulance Service, NHS Accident and Emergency (A&E) departments and Primary Care to NHS24; the mode of service delivery at both Level 1 and Level 2 (from face-to-face to telephone); and the impact of the COVID-19 pandemic (all evaluated in this extended evaluation ) did not appear to contradict the key findings described in the main evaluation. For example, in both evaluations Level 1 practitioners and individual participants recognised the potential benefit of DBI in providing a practical and timely solution to many individuals in distress. As in the main DBI evaluation, some of the more experienced Level 1 practitioner interviewees perceived the level of DBI training to be too basic.

The main presenting problems in both evaluations were the same: depression/low mood, stress/anxiety, and suicidal thoughts. As with the findings of the main DBI pilot evaluation, Level 1 practitioners in this extended evaluation felt that DBI empowered them to offer a compassionate and constructive response. Level 2 contact was attempted in 100% of cases and successfully achieved within 24 hours for most individuals in both evaluations.

Though using different methods to measure service user outcomes, average distress scores for most people decreased between Level 1 and the end of Level 2 in both evaluations.

Both evaluations highlight that DBI appears less suited to some individuals presenting to DBI with long-term mental health issues and other complex needs. This finding emphasises the importance of refining the appropriateness of referrals and reviewing whether inappropriate referrals are highlighting further gaps in existing services in terms of meeting needs that DBI is not designed to meet.

Unlike in the main DBI pilot evaluation, where the guideline of not exceeding support lasting 14 days for Level 2 intervention was met in 44% of cases, in this evaluation this guideline was met in 80% of cases. It may be that the difference in usual mode of Level 2 delivery influenced the duration of Level 2 service provided. In a small number of Health Board areas, small numbers of people remained in DBI for a considerable length of time (37- 42 days), while in one other Health Board no individuals were seen over 14 days. The three areas with the longest period of support provided were associated with the same third sector service provider. This suggests that duration of intervention may not be associated solely with personalised intervention but may also be associated with organisational practice. Similar issues were noted in the main pilot evaluation study.

Neither evaluation found a relationship between changes in distress score and gender, Health Board area, or area deprivation (measured by SIMD quintile).