Distress Brief Intervention pilot programme: evaluation

5 Training

5.1 Level 1 training delivery, modes and content

The DBI Level 1 training course aimed to provide Level 1 practitioners with the skills and confidence required to make appropriate referrals and deliver the DBI programme as planned. With no existing training for people in distress available during the set-up of DBI, the University of Glasgow adopted an integrative, biopsychosocial approach to the intervention, drawing from best practice in other areas of mental health and suicide prevention (O'Connor & Kirtley, 2018, Kok et al., 2016). Up to the end of December 2020, 997 frontline staff had received either the DBI Level 1 e-learning module or trainer-facilitated classroom training across all four pilot sites (by March 2021 this had risen to n=1,816).

The way training was delivered varied across pilot sites and across different Level 1 services. Training was delivered both online and in the facilitated face-to-face sessions. On occasion, Level 2 DBI practitioners were brought in to support Level 1 training. Additional brief top-up training, described as 'buzz sessions', was developed and provided at each site by local lead agencies. These additional training sessions enabled practitioners to answer questions, share experiences on delivering DBI and reflect on issues such as the appropriateness of different types of referrals. There was no fixed structure or frequency of buzz sessions; they were developed and delivered according to local implementation needs. Individuals who had undertaken online training particularly valued the buzz sessions, as they enabled relationships to be developed between Level 1 and Level 2 agencies.

5.1.1 Online Training

Online training for Level 1 frontline practitioners was generally delivered through a learnPro module, an online learning platform commonly used by the NHS. The module lasted for around 1 hour and focussed on providing a compassionate response, the DBI definition of distress and the basic practicalities of making a referral. Online learning was initially the main delivery format in most NHS settings and met with a mixed response. Some practitioner interviewees considered it optimal, especially where there were time constraints, whereas others felt it was more of a 'tick-box exercise' and did little other than introduce basic processes. Practitioners in some Level 1 services, particularly Primary Care practices, saw the learnPro module as a disincentive to engage with the training.

"There'll be a number of people that if they were unable to access DBI without doing the learnPro there would probably a greater uptake in the actual accessing and use of it which sounds a bit odd given it wasn't too onerous, but it was still the mental step of having to log in to do, … in the general practice world people are under pressure and stressed and stuff like that means that they'll push things to 'I'll do that later." Level 1 practitioner, primary care

Over the course of this evaluation, the learnPro training was refined by adding additional top-up training. Glasgow University trained some practitioners to be trainers. There was an expectation that trainers should be familiar with and immersed in the DBI programme, which permitted Level 2 providers, where appropriate, to deliver training to Level 1 providers. This allowed face-to-face delivery of Level 1 training to more people and enabled the enthusiasm and experience of existing Level 2 practitioners to be shared with new trainees.

"The training package is one of the best training packages that I've been involved in, but it's the quality and delivery of that training as well. If you're just standing there with a PowerPoint letting somebody read that or handing them a workbook and just saying, read that, that's not going to be beneficial to that person. Every session that I do is an interactive session, we have dialogue, we have a discussion, every session runs over the time that I'm usually given, but do you know what, that's the sessions that make a difference, that's the sessions that somebody will remember as opposed to sitting reading the workbook." Level 1 service Lead

Time was not made available for ambulance practitioners to complete their online training during working hours therefore they were expected to complete it in their own time. Some practitioner interviewees felt this made completing training more challenging and acted as a disincentive to their participation, a finding also reflected in the relatively low total numbers of ambulance practitioners trained.

5.1.2 Face-to-face training

There was a consensus among Level 1 practitioner participants that discussions occurring during face-to-face training which drew on experiences from actual cases and hypothetical scenarios, made it the preferred delivery mode. Police Scotland opted for face-to-face training from the beginning, as this was their organisation's preferred mode for any training delivery. The training was co-delivered with Level 2 practitioners and uptake among Police Scotland was high. The police formed the highest proportion (49% - almost half) of Level 1 practitioners trained.

"We've seen with the police training being delivered face-to-face and in partnership with our Level 2 staff actually you get a lot of police referrals, that they have that relationship between the police and the Level 2 staff because they've met at training so they're confident and comfortable in putting forward a referral." Level 1, service lead

When Level 2 practitioners were brought in to support Level 1 training, it strengthened connections between Level 1 and 2 services and their practitioners and further encouraged Level 1 practitioners' engagement with the DBI programme. In some areas, Level 2 services directly approached local GP surgeries to ask if they wished to be involved as a Level 1 service. This was a highly effective recruitment strategy, but a resource-intensive approach.

DBI Level 1 training has impacted upon the wider service system by highlighting gaps in existing practices and acted as a catalyst to developing further training in managing distress and assessing the level of risk. This occurred within emergency NHS settings but could be extended to other services to address concerns about making judgements without the appropriate skills.

"And also when I did the face-to-face we also covered how are we going to properly triage someone to determine their level of mental health risk." Level 1 practitioner, A&E

Some practitioners, particularly if they were clinically qualified, felt that the Level 1 training sections on identifying distress were unnecessary; they felt their clinical training gave them specialist skills beyond those outlined in DBI Level 1 training.

5.2 Level 1 pre and post-training confidence ratings

University of Glasgow developed a brief evaluation of practitioners' confidence to deliver Level 1, to be completed immediately before and after training. NHS Health Scotland provided the evaluation team with confidence ratings for a total of 997 frontline practitioners. This included those working within the police, ambulance service, A&E, Primary Care, Social Work and community and crisis mental health teams who were trained between October 2017 and December 2020 (Table 5.1). Of these 997, all police (n=490) and 176 healthcare staff in Lanarkshire completed the training via facilitated face-to-face sessions; the remaining 331 completed their training via the learnPro online training course.

Overall, the training increased practitioners' confidence to understand distress, deliver a compassionate response, make referrals and understand what was involved in Level 2 support (Table 5.2).

5.3 Level 1 skills, competencies and confidence in delivering DBI

In March 2020 we invited DBI Level 1 and 2 staff by email to participate in an online training survey (issued by the DBI evaluation team) to examine the longer-term impact of the DBI training in terms of skills, competencies and usefulness of the training when delivering support as well as any additional training needs (see Appendix 4). Overall, the survey suggests that Level 1 training was generally well-received; among Level 1 staff responding to the survey (n=172), most found it relevant to their role (94%), engaging (81%) and enjoyable (73%).

The majority reported that it had provided them with the knowledge, skills and confidence to carry out a DBI Level 1 intervention (80%) and felt committed to delivering Level 1 as a result of the training (85%). A similar proportion (79%) reported that they had made use of what they learned in the Level 1 training when delivering a Level 1 intervention. A minority (9%) agreed with the statement "I did not learn anything new from the DBI Level 1 training".

Although confidence was generally high following training, some Level 1 practitioners who were interviewed reported that they doubted their ability to safely judge whether DBI alone was enough for an individual whose level of risk of self-harm was elevated.

Just over half (53%) of the Level 1 practitioner survey respondents agreed^[4] that their DBI training enabled them to more consistently provide a compassionate response to people in distress. More participants (72%), agreed that, as a result of the DBI Level 1 training, they were more able to provide a constructive response to people in distress. So, while the DBI training played an important role in helping practitioners to provide a more consistent compassionate response, participants perceived it was even more effective in equipping them to provide a more constructive response to meet the needs of those presenting in distress.

Correspondingly, Level 2 practitioner interviewees noted changes in frontline practitioners' approach to supporting individuals in distress following training.

"The police or the ambulance are not just turning up, dealing with the situation in hand and going away, they're showing empathy and compassion for the situation that the people have found themselves in and offering them some kind of solution." Level 2 practitioner

DBI training also changed perceptions about the nature of distress among Level 1 practitioners. There was evidence to suggest that their training led to a reduction in the stigma previously associated with distress-related behaviour where it was associated with mental health issues. To explore the impact of the DBI Level 1 training on potential direct stigma, respondents were asked the extent to which they agreed that they were more likely to treat someone fairly because they were seeking help for their distress. Just under half of the respondents (45%) agreed that as a result of the training, they were now more likely to treat someone fairly who was seeking help for their distress, indicating a possible reduction in stigmatising behaviour amongst this group. Thirty-four percent of respondents selected neither agree nor disagree with regards to this statement, and 22% disagreed.^[5] This suggests perhaps that these respondents felt that they already treated people in distress fairly and that the Level 1 training had little or no impact on this.

Similarly, in the interviews and focus groups, Level 1 practitioners often mentioned that the perceived emphasis on teaching compassion could be a barrier to engaging in training. A sub-set of Level 1 practitioners felt that some aspects of training (e.g. focusing on compassion in the Level 1 training) did not add to their current knowledge. They felt it was potentially patronising and made them feel less engaged with the DBI programme. GPs and mental health practitioners saw this as an existing component of their routine practice and its inclusion in the training could be perceived as suggesting that current practice lacked compassion.

"You think as a doctor that you give a compassionate response so why would you need to go and understand what it is to give a compassionate response because you would always do that anyway…" Level 1 practitioner, primary care

Level 1 training also drew attention to how distress might underpin other negative behaviours, which was particularly resonant with some respondents.

In the early implementation stages, there were concerns that the Level 1 training was not clear enough on the appropriate level of distress for a DBI referral, leading to individuals being referred unnecessarily or when DBI was insufficient to meet their needs. Level 2 practitioners fed back when there had been inappropriate referrals and delivered additional training updates (buzz sessions) to provide clarity on the appropriate distress level for DBI referrals. This appeared to help reduce the number of inappropriate referrals.

"When we first started, especially the police, I didn't think they quite understood, you know, we were getting people who were in a complete and utter crisis and sometimes it was an inappropriate referral, so what we done was we started attending the training just so that we were able to say it doesn't have to wait until someone's at the complete and utter crisis, it could be someone who's having a neighbour dispute or his ongoing thing, you know, it doesn't have to be that someone's being turned up to hospital as a suicide attempt, you know. So, I think once we were able to give them a bit of that reassurance of what a person in distress is, cause I think perhaps our understanding of distress and their understanding of distress was maybe a bit different at times. And then I feel like after we attended a few of the training we were getting more appropriate referrals and the referrals actually started streaming in." Level 2 practitioner

5.4 Level 1 unmet training needs

The Level 1 practitioner survey respondents were asked to provide details of any unmet training needs they felt they had, as well as any additional training they had undertaken which was helpful to them in delivering DBI. Around one in seven (15%) cited additional training needs which they felt they would benefit from, including:

• A refresher session on the original training.
• Training on the DBI referral systems, including learning about the support that other agencies can offer.
• Opportunity to shadow Level 2 practitioners.
• Training on mental health.
• Solihull training.^[6]

A similar proportion (15%) said that they had received relevant additional training including:

• Mental health training such as Mental Health First Aid, mental health awareness and self-harm awareness.
• Suicide prevention training such as ASIST.^[7]
• DBI Train the Trainer provided by the University of Glasgow.
• Training on gender-based violence.

5.5 Level 2 training delivery, mode and content

The two-day Level 2 DBI training programme was initially delivered by the University of Glasgow. From February 2018, the training was transferred to established local site managers and coordinators who received a facilitator training pack to deliver DBI Level 2 training. Most Level 2 practitioners highly valued the training and practitioner interviewees noted that it had increased understanding of compassion, distress and the evidence-base underpinning the DBI intervention. Overall, Level 2 training survey respondents agreed or strongly agreed that they had enjoyed the training (83%) and that it was engaging (87%) and relevant to their role (86%).

"The information that's there is really worthwhile and really balanced. I think that it's given me a lot of credibility to be able to kind of pull information from the slides and things like that that I wouldn't have had before, you know, about research and things like that. So it's great because it's all backed up and I can kind of give the credit to Glasgow University and say, look, I'm here delivering but this is where it's coming from and it's giving that kind of a bit of credibility." Level 2 practitioner

Following the introduction of a facilitator training pack for Level 2 training, new practitioners often had informal on-the-job training, delivered by Level 2 practitioner trainers followed by a shorter in-house formal training session. Some Level 2 practitioner interviewees commented that this approach to training gave them practical knowledge, which was useful, but the background information and more general elements should have been given at an earlier stage.

"We'd done a little mini training almost between me and the local coordinator when I first came in and I done a lot of shadowing, but because I think they wanted to do a bigger training session instead of just it being myself and my other colleague, then I felt like the training was something I was doing already, you know, I was doing this already but I guess it helped me know that I'm doing exactly what I'm meant to be doing in my job, but before that I spent a lot of time with the local coordinator, especially when it first started, going over distress management plans, going through all the toolkits, you know, shadowing, done a lot of shadowing, a lot of reading." Level 2 practitioner

Some practitioners said they would welcome a stronger emphasis in the training on the reality of implementing the DBI Level 2 intervention and self-care.

"The DBI training, in the first instance, could have included more of what was actually expected during a face-to-face DBI. It can be quite stressful to be a DBI practitioner so possibly more training on self-care and understanding of how to continually deal with other people's distress. Also training in the changing issues that people are dealing with, i.e. with the universal credit roll-out, this can change the issues that people are attending with so in the event of large scale changes like that training would be best put in place. (Felt that they weren't contributing to outcomes)." Level 2 practitioner

Almost all Level 2 practitioners had previous experience in providing support work with people in distress and/or with mental health issues. They perceived that the experience and knowledge they brought with them had been essential in their new DBI role. Many of these practitioners reported using techniques learnt in previous roles to support individuals referred to DBI. These DBI practitioners also reported advantages of being familiar with local services which enhanced their ability to effectively sign-post.

Some Level 2 practitioner interviewees noted that the Level 2 training enhanced their existing skills and changed their approach, especially around listening to individuals. However, some DBI Level 2 interviewees expressed reservations about the quality of delivery and lack of emphasis on practical elements during the training when it was delivered by in-house practitioners.

"My experience of the training and the original team's experience of the training were two totally different experiences. So, this is no slight on any of my colleagues or anything like that but mines was very much sitting in a room and listening to people reading off a script. I don't feel it was very practical the training, as in 'this is how you would fill out a D-MaP, this is what we're trying to get out of the D-MaP' that sort of thing." Level 2 practitioner

As DBI was rolled out, anonymised case studies of how individuals had been managed were shared at initial training, refresher training, buzz sessions, and DBI Gatherings. Some Level 2 practitioners suggested creating top-up online training which could feature some of the most relevant anonymised cases.

"We now have actual cases that we've worked on, so to use the experience from these and the knowledge from these as training schools. You know, like case studies, like to be able to see...because I think it would be really good to have like an online thing where practitioners could go in and practice different things, and that would be a good tool for this type of thing to see what services you could connect them into, what tools you could give them, that type of thing." Level 2 practitioner

5.6 Level 2 skills, competencies and confidence in delivering DBI

By March 2020, a total of 107 staff across the six DBI Level 2 provider organisations were trained in DBI Level 2. Overall, DBI Level 2 practitioners felt that the training had given them the knowledge, skills and confidence to deliver the DBI level 2 intervention. In the Level 2 practitioner survey, agreement was highest for the statements 'DBI Level 2 training gave me the knowledge I needed to deliver DBI Level 2 interventions' (89%) and 'I make use of the learning I gained at the DBI training when carrying out DBI Level 2 interventions' (89%).

It should be noted, however, that 16% disagreed that the DBI Level 2 training had given them the skills to deliver DBI Level 2 interventions, and 15% disagreed that the DBI Level 2 training had given them the confidence to deliver DBI Level 2 interventions. This suggests that a sizable minority of Level 2 practitioners felt that the Level 2 training had not adequately prepared them for the job.

The majority of Level 2 practitioner survey respondents agreed that they were more able than before to provide a more constructive (69%) and more compassionate (65%) response to people in distress as a result of the DBI training. To explore the impact of the DBI Level 2 training on potential direct stigma, respondents were asked the extent to which they agreed that they were more likely to treat someone fairly because they were seeking help for their distress. Agreement with this question was lower for Level 2 respondents (45%) than for Level 1 respondents (58%); however a similar proportion of Level 1 (22%) and Level 2 (19%) practitioners disagreed - again suggesting that these respondents felt that they already treated people in distress fairly.

At Level 2, some practitioners felt that their previous training in supporting people with suicidal thoughts or behaviour was highly important and some suggested it should be incorporated into DBI training. Equally, some felt that training in managing trauma-related distress would be beneficial, as this was often a contributing factor to individuals' distress.

5.7 Training costs

Initial training for Level 1 and Level 2 providers took place in 2016-2017. The University of Glasgow was awarded a contract, in the region of £225,000, to develop the training and deliver it in the four pilot areas. Approximately half of this money was spent on staff costs for developing and delivering the training to Level 1 and Level 2 practitioners and the rest was spent on travel, materials and organising the in-person group training sessions. The above total training cost was spent on Level 1 and Level 2 training in approximately 60%-40% division. Materials for in-person training included training packs with slides and other handouts.

The activities to set up and deliver the training involved: 1) Desk work: literature and evidence review, governance, obtaining approvals, administrative tasks; 2) Engagement and consultation with DBI pilot programme members, other stakeholders, local implementation partnerships; 3) Interviews and focus groups: frontline services staff; 3rd sector services staff; individuals with experience of distress, service use and help-seeking; 4) DBI Level 1 and Level 2 training programmes: content development, drafting and preparation; iterative programme review and revision; and 5) Organisation, travel and delivery of facilitated training sessions for DBI Level 1 frontline services practitioners and DBI Level 2 services. There was additional spending of £10,000 to build the learnPro training platform and make the platform accessible to Level 1 staff.

Initial training on DBI Level 1 for the Health Care Practitioners (Primary Care in hours, A&E, Scottish Ambulance Service) was estimated to take 537 person-hours. Police Scotland conducted separate training and data was not provided on the total number of hours. The initial training for the DBI Level 2 practitioners was estimated to take 616 person-hours. A detailed breakdown of the set-up and maintenance (ongoing) training costs along with the number of personnel trained and investment in personnel hours at Level 1 and 2 across the 4 pilot sites is presented in Appendix C.