National Trauma Training Programme - local delivery trials: interim evaluation

The Process Evaluation

This interim report focuses on the progress of the NTTP Delivery Trial Sites^[4] and learning from the independent process evaluation. The process evaluation was commissioned in October 2019 and ended in March 2020. The process evaluation aimed to provide learning for future expansion/roll out of the NTTP with regard to practical and effective approaches to partnership working to deliver and embed trauma training. It was to explore contextual elements within the three sites which contributed to enhanced trauma awareness and facilitated successful implementation. The more detailed aims and objectives are listed in Appendix 1.

Intended methodology

To address the evaluation aims and objectives a Theories of Change (ToC)^[5], approach was used. ToC is a participative approach that supports the articulation of an intervention’s programme theory. ToC models were developed to illustrate what sites intended to do and how and why their activities would lead to their intended outcomes. The approach sought to understand the contexts and assumptions on which activities were based. A ToC should be plausible, doable and testable and provides a framework for strengthening planning and delivery and for informing monitoring and evaluation^[6]. The planned process evaluation methods included:

• the development of a strategic ToC model for the overall Delivery Trial Sites programme (set within the wider NTTP)
• the facilitation of local ToC models at each site
• accompanying narratives and descriptors of context at each site
• support for the prioritisation of outcomes and development of a high-level data collection/monitoring and evaluation plans
• participative reflections on progress and learning at two time points
• analysis and triangulation of learning
• reporting on learning for future roll out by March 2020.

Delivery Trial Site ToCs were informed by the local funding applications and aligned to a nationally agreed template model for the overall Delivery Trial Site programme. The above activities collectively were intended to provide a means of framing local data and learning within and across sites when these became available.

There were a number of barriers that prevented the delivery of several of the above methods across the Delivery Trial Sites. Barriers included limited time-frames and staffing and workload challenges that hampered implementation in local areas. More is said about these issues in the findings/learning section below. The barriers and delays prevented full engagement with the process evaluation activities and in turn impacted on the resulting analysis and learning. Most significantly the Coronavirus (COVID-19) pandemic prevented the delivery of the planned frontline training and internal monitoring, evaluation and subsequent analysis of the impact of that training within and across sites. Table 1 shows the progress achieved against the intended process evaluation methods.

As a result of the barriers and delays the independent evaluator was unable to conduct activities 4 and 5 in Delivery Trial Sites 1 and 2 as illustrated in Table 1. This meant that for Delivery Trial Sites 1 and 2 monitoring and evaluation plans were not discussed in detail nor supported as part of the process evaluation. It also meant that follow-up reflective sessions to seek learning on barriers and facilitators to success were also not conducted in these sites. One reflective interview was conducted with a third sector organisation representative who coordinated a consortium to deliver aspects of training in one area. In Delivery Trial Site 3 one reflective focus group was conducted with key members of the implementation group (including the appointed Project Manager) and a further interview was undertaken with the initial Lead Officer who submitted Site 3’s Delivery Trial Site funding bid.