Enablers and barriers to trauma-informed systems, organisations and workforces: evidence review - appendices

Appendix A: Classification of intervention effectiveness

Decision-making tools (classification of intervention effectiveness and an evidence of effectiveness decision tree) were used to inform how the evidence was synthesised for this review. These tools were developed for and initially implemented within the Scottish Government report What Works to Prevent Violence Against Women: A Summary of the Evidence. They have been adapted within this report to ensure a consistent and transparent approach to classifying the effectiveness of interventions to promote positive outcomes for staff and people with lived experience of trauma involved in different systems, such as education, child welfare, or the health system.

The decision tree leads to the following six categories of effectiveness. Appendix A provides definitions for each of these evidence.

It should be noted that the inconclusive category is distinct from the no effect category as it indicates either there is insufficient evidence to make a judgement on the impact of an intervention (e.g. only pilot evaluations available) or indicates the need for further research and evidence before conclusions can be drawn on the effectiveness of an intervention.

The decision making tool below has been used to determine effectiveness ratings throughout this report on what strategies to trauma-informed approaches work to promote positive outcomes amongst those involved in different systems.

Definition of the effectiveness categories used for analysis.

Adapted from Miller (2020).

Category: Effective

Definition: Evidence that the intervention is associated with a positive impact on the outcomes of staff working within different services, systems, and organizations, and people with lived experience of trauma involved in those systems, based on a moderate or strong evidence base. Due to the complexity of causality, an 'effective' intervention should be considered one that contributed towards improved health or wellbeing rather than one that single-handedly accounts for an increase in health or wellbeing.

Category: Promising

Definition: Findings were positive but not to the extent that they constituted evidence that an intervention was 'effective', this could be:

(i) in cases where an intervention has a positive impact on an intermediate outcome, rather than in improving health and wellbeing itself

(ii) where authors noted a positive change, but expressed doubts as to whether the intervention could confidently be said to have contributed to this (e.g. due to evidence being rated as "weak" or the other factors potentially having an impact).

Category: Mixed

Definition: Findings of individual article -

(i) An individual article that finds varied impact of a single intervention across research sites, or populations.

(ii) An article examining multiple strands of an interventions that finds some were effective/promising and others not.

Findings from a number of studies-

(i) Where there have been a number of studies and the results contrast – e.g. some found positive effects and some did not.

(ii) Similarly, a body of evidence that is mostly comprised of individual articles finding a 'mixed' impact of interventions would be considered 'mixed' overall.

Category: No effect

Definition: No evidence of effect (positive or negative) of the intervention on improving health or wellbeing or includes moderate or strong evidence found the intervention had no effect on improving health or wellbeing.

Category: Negative effect/ Potentially harmful

Definition: Evidence that the intervention is associated with worse health and wellbeing outcomes (e.g. worse than at the start of the intervention, or worse than for a control group).

Category: Inconclusive

Definition: Insufficient evidence to make a judgement on impact.