Distress Brief Intervention pilot programme: evaluation

8 Key Findings

This section brings together the key findings from across the evaluation, relating to the overarching research questions and anticipated DBI outcomes. A high-level synthesis of these findings is in Appendix C which presents a synthesis of the results from all three elements: summarising the key findings, and considerations for delivery and resourcing decisions.

In this section where appropriate, summary findings known as Context Mechanism and Outcome (CMO) configurations are reported in a box towards the top of each section. These causal explanations describe what works in DBI, for who, why and in what circumstances, according to each section topic. Implications arising from the key findings are also presented.

8.1 Impact of DBI on individuals

Immediate referral and 24-hour contact (C) validated and reassured people that their needs were recognised (M), which decreased self-stigma and self-perceived levels of distress (O)

Increased perception of Level 2 practitioners' compassion and engagement (C) made individuals feel empowered (M), which led to individuals feeling less distressed at the end of the intervention (O).

Referrals of people with severe and enduring mental health or addiction needs (C) can create unrealistic expectations of what DBI can offer and lead to disappointment among some individuals (M) which may exacerbate distress (O).

The DBI Level 1 response has direct, immediate benefits for the individual. Most individuals thought that the Level 1 provider had helped them cope with their immediate distress. Those in less distress at the start of their Level 2 intervention were more likely to rate the Level 1 provider highly in terms of helping them to cope. This suggests that Level 1 provider intervention is important in helping individuals to cope with their immediate distress.

Level 1 worked less well for younger adults, those with higher levels of distress and those presenting to A&E. Although most individuals felt they were treated with a high level of compassion by Level 1 frontline practitioners, this varied - with younger people, those with higher levels of distress, and those presenting to A&E more likely to give a lower rating of compassion than others. Although most thought that the Level 1 provider had helped them cope with their immediate distress, this was not scored as highly as compassionate response. Younger people and those presenting to A&E reported a lower level of help from Level 1 practitioners to cope than others.

DBI Level 2 is working well for the majority of individuals. Nine out of ten (90%) of individuals showed a continued decrease in their distress over the period of the Level 2 intervention. However, for around one in ten individuals their distress level was higher at the end of the Level 2 intervention. Changes in individuals' level of distress following DBI Level 2 were not associated with age, gender, area deprivation, the main presenting problem, Level 1 referrer, Level 2 provider or length or intensity of the intervention. Level 2 may be working less well for women but better for younger adults in terms of their final level of distress at the end of Level 2.

Delivering compassionate care at Level 1 and Level 2 was central to helping individuals to understand their distress and reduce it. Individuals' perception of Level 2 practitioner compassion and care was positively associated with: greater decreases in distress and agreement that DBI had helped improve understanding of why they felt distressed and agreement that DBI had helped them to recognise when they start to become distressed. In turn, being helped to understand why they felt distressed was positively associated with an individual's decrease in distress. Practitioners and individuals felt that a combination of compassionate response and practical support helped to validate people's distress and break down barriers to seeking help, thereby reducing self-stigma.

Level 2 helped most individuals to manage their distress. Nine out of ten individuals agreed that DBI had given them the tools and skills to manage their distress. Findings also suggest that those going through DBI have been using what they learned during the intervention to help them manage their distress in the longer term.

DBI may also be contributing to suicide prevention. One in ten individual evaluation participants revealed that they may have attempted suicide or continued with suicidal thoughts if DBI had not been offered to them.

Level 2 seemed to work less well for some of those with long-term enduring mental health or addiction needs. Level 2 did not meet the needs of individuals when their expectations of what the programme offered were misaligned with the reality of a short-term, problem solving, and practical service. Feedback from Level 1 and 2 providers and individuals suggests that DBI is less appropriate for the needs of those with severe and/or enduring mental health problems and/or addictions. The desire to facilitate quick access for support, even when referral to DBI was inappropriate, may highlight gaps in existing services to provide immediate support to people with more enduring mental health problems. Notwithstanding these limitations, some individuals who repeatedly access unscheduled care appear to have positive outcomes from DBI.

Less is known about the longer-term impact of DBI on individuals and experiences seem mixed. Three months since their last contact with DBI around half of the evaluation participants had been in contact with the police, ambulance, their GP, or A&E because they were in distress. Half of the evaluation participants reported at 3 months that they were referred to other services by their DBI provider and four out of five of those reported they had taken up the service. For some individuals, this had led to re-engagement with work, as well as offering a longer-term support mechanism. Some reported feeling isolated and lost following DBI and awaiting further support.

The reach and impact of DBI suggests that it is contributing towards improved population wellbeing, including appropriately managing distress and may be contributing to preventing some suicidal behaviour.

8.2 Preparedness of practitioners to effectively deliver DBI

Level 1 practitioner training (C) generally increased DBI Level 1 practitioners' awareness, confidence and ability (M) which enabled them to provide a consistent and compassionate response.

Level 2 practitioner training (C) generally enhanced knowledge, skills and confidence (M) to enable the delivery of the DBI level 2 interventions (O).

Face-to-face training with Level 2 staff, focused delivery of practical examples and regular refresher sessions (C) encouraged engagement in training and cross-sector relationships (M) which improved the perceived value of Level 1 training (O).

Overall, most practitioners found that Level 1 and Level 2 training prepared them well to effectively implement DBI. The delivery of training varied across pilot sites and Level 1 frontline services - but face-to-face was broadly the preferred delivery mode. However, training via an online platform has the potential to reduce cost compared with face to face training. It is worth noting that the training evaluation was conducted before the COVID-19 pandemic when online activities became more normalised. It may be that practitioners' perceptions of online training have altered as a result.

However, for some the training was unnecessary in certain respects and lacking in others. A sizable minority, one in seven (15-16%) of Level 2 practitioners felt their training had not adequately prepared them with the skills or confidence required for the job. Level 2 practitioners who received informal 'on the job' training felt that they needed the formal DBI training at an earlier stage. Frontline practitioners who had not received Level 1 training but referred people to Level 2 were likely to make mistakes in the referral process. Some Level 1 practitioners felt their clinical training already gave them specialist skills beyond those in DBI training, particularly on identifying distress and responding compassionately. In addition, some encountered barriers to accessing training. Many ambulance clinicians had to complete training in their own time and others struggled to make time within their busy working day to log in to online training.

An important impact of the DBI Level 1 and 2 training was that many practitioners reported that it changed their perceptions of people in distress; this may have consequently reduced the likelihood of individuals in distress feeling stigmatised by those whom they approached for help. Around half of practitioner participants reported that following their training they were more likely to treat someone fairly because they were seeking help for their distress.

Some Level 1 practitioners suggested they would benefit from the chance to shadow a DBI Level 2 practitioner and not all received regular refreshers or buzz sessions. Level 2 training improvements suggested included: more emphasis on the impact on practitioners of dealing with distress and self-care techniques; area specific information on local services to sign-post to; more focus on the practicalities of the job, such as completing a Distress Management Action Plan; and more use of real case studies, especially on supporting people with suicidal thoughts or behaviour.

8.3 Implementation of the DBI Programme model

Embedding delivery of training within organisations' usual training/work routines (C) led to staff having enhanced engagement with DBI (M), which resulted in increased numbers of appropriate referrals (O).

Level 2 practitioner involvement in Level 1 training and feedback from Level 2 on inappropriate referrals (C) led to staff having enhanced understanding of and engagement with DBI (M), which resulted in increased numbers of appropriate referrals, increased inter-agency working and a shared commitment to the values of DBI (O).

Flexibility in delivery of the intervention (M) enabled individuals' complex and varied circumstances (C) to be addressed (O).

DBI delivery has successfully adapted, where appropriate, to different local contexts, whilst maintaining the core elements of DBI. As the delivery of DBI Level 2 services within an area increases, this will necessitate changes in staffing and other resources, such as premises. During the delivery phase, all sites were employing senior practitioners and administrative support. Referrals to DBI were largely appropriate, with ongoing work throughout the pilot to find solutions to decrease inappropriate referrals. Level 1 practitioners reported that when they received constructive feedback and support from Level 2 practitioners on referral appropriateness, they were able to streamline and be more appropriately selective in the referrals they made.

The guideline of a contact attempt for each referral within 24 hours was met. Most (five in six) individuals were left with a clear understanding of what would happen in 24 hours of their referral to Level 2. As the pilot progressed, more emphasis was placed on the importance of providing as much practical and emotional support as possible within that initial contact, including the use of the D-MaP. Successful contact was made with individuals within 24 hours in around two-thirds of cases, rising to 86% in the following days. The analysis of outcomes suggests that this was not associated with eventual outcomes for individuals.

The 14-day Level 2 intervention guideline was met for just over half of those taking up support, with length and intensity of support provided varying by pilot site. Forty-four percent of all individuals who took up support from DBI Level 2 to planned or unplanned closure received over 14 days of support. Individuals who received up to 14 days of support received, on average, 3.1 sessions, while those who received more than 14 days of support received an average of just over 5 sessions. Although a third of individuals thought the guideline of 14 days was not enough, analysis of outcomes suggests that the length and intensity of DBI Level 2 support were not associated with either change in distress or distress levels at leaving Level 2.

A key strength of DBI is its flexibility to be tailored to the individual, resulting in being appropriate to the needs of a wide range of individuals in distress who present with an array of different characteristics, life circumstances and problems. Relationship issues were the most commonly recorded contributory factor for both men and women, recorded in 48% of all referrals. Other common contributing factors included alcohol use (22%), life coping issues (21%) and money worries and unemployment (18% each). Alcohol use was recorded as a contributory factor in a higher proportion of men (29%) than women (16%). Substance misuse was also a contributory factor in a higher proportion of men (19%) than women (7%). Recorded alcohol and substance use were lowest among those referred to primary care in hours (10% and 5% respectively) and highest in A&E (35% and 23% respectively).

Generally, individuals engaged well with DBI Level 2, with some using plans and strategies developed with their DBI practitioner (including the Distress Management Action Plan) up to three months beyond the end of their Level 2 intervention. Those referred by Primary Care and mental health unscheduled care were more likely to engage with DBI Level 2 than those referred by A&E, police and the ambulance service. This may be due in part to individuals referred from emergency services having less clarity during the referral process and hence less understanding of what DBI was about, as indicated in evaluation participant feedback. The majority of those accessing DBI Level 2 were sign-posted on to follow-up services, with practice varying by pilot site. Signposting to non-statutory services (85%) was much more common than to statutory (29%). There was considerable variation between Level 2 providers in the use of signposting to statutory services. The differences in signposting practices partially reflect differences in presenting problems in the different areas.

Initiation, where the DBI practitioners set up a call or meeting with a further service was more commonly made to non-statutory services (25%) than to statutory services (11%).

8.4 Contextual factors influencing DBI implementation success

DBI Central with enthusiastic and respected change management leadership (C) enhanced stakeholder adoption and facilitated cross-sectoral planning (M), which resulted in sustained engagement and enhanced reach of the intervention (O).

The DBI Gatherings and local implementation groups (C) facilitated "partnership in action" across agencies and services (M), which increased sharing of best practices, challenges and solutions (O).

Adaptions responding to grass-root feedback (C) developed a synergy between DBI Central and pilot sites (M) which led to continuous improvement of implementation processes (O).

Contextual factors were not associated with individual outcomes. Changes in individuals' level of distress following DBI Level 2 were not associated with area deprivation, Level 1 referrer, Level 2 provider or length or intensity of the intervention.

The role of DBI Central in coordinating services and enabling open communication and information sharing was perceived as an essential component of the DBI programme. The constructive leadership of the DBI programme manager was recognised as being particularly central to its success.

DBI Gatherings and local implementation groups enabled cross-sectoral delivery of DBI. The success of these events was perceived as stemming from the 'open door' offered by DBI Central and their continuous efforts to listen to stakeholders, acknowledge where implementation was less effective, and address issues in conjunction with those delivering the service.

Local DBI implementation groups were strong contributory factors to successful implementation at a local level. These groups enable problem-solving at a local level and were reported by some of the stakeholders involved to have successfully engaged more agencies than previous inter-agency events had managed.

Champions acted as role models within services, embodying the principles and allowing the benefits to be seen by others. Where frontline existing referral systems could be used or adapted, this facilitated referrals to Level 2.

Contextual factors that impeded DBI from meeting its aims and objectives included where DBI practitioners doubted the added value of DBI and viewed DBI as potentially replacing services that are considered to be of greater use. Where Level 1 practitioners considered addressing mental health issues as being outwith their role, this also impeded implementation. A further barrier was where frontline existing referral systems could not be adapted to incorporate DBI referrals, considerably impeding the referral process.

8.5 Impact of DBI on the wider service system

The impact of DBI on the wider service system seems to be largely positive. Less is known about the overall impact on the use of unscheduled care by those in distress and the impact that DBI onward referrals have on these agencies' capacity to meet demand. Across frontline staff and Level 2 practitioners, the majority of respondents agreed that DBI provides a more effective way for services to respond to people in distress and that DBI has improved integrated working across frontline services. Practitioner feedback suggests that DBI is providing Level 1 services with an opportunity to contribute to better outcomes than before for those presenting to them in distress. DBI training highlighted gaps in some existing Level 1 practices and acted as a catalyst to developing further training in managing distress and assessing the level of risk. For most individuals, DBI is a step towards recovery from distress and connects them (often for the first time) mainly with community-based voluntary sector services and for around a third to statutory services appropriate to their needs. Feedback on the continued success of implementing the 24-hour contact requirement at Level 2 has increased trust in the programme and Level 1 practitioners' confidence that they are offering a compassionate and effective response to individuals in distress. This perpetuates DBI's use and reduces concern about accountability amongst frontline practitioners.