A realist evaluation of the enhanced health visiting service in NHS Ayrshire and Arran

2 Methods

Evaluation design

2.1 The evaluation was informed by realist framework. Briefly, realist evaluation is a theory-driven approach to evaluation of social and healthcare programmes, such as the refocussing of the health visiting service in NHS Ayrshire and Arran. It focuses on building, testing and refining programme theories (or logic model) by exploring the complex and dynamic interaction of the context (settings or conditions in which the programme was implemented), mechanisms (behaviours and decisions of practitioners delivering the programme) and outcomes (intended and unintended effects) (Pawson and Tilley, 1997; Pawson, 2006, Wand et al, 2010; Byng, 2011). Following the realist framework, this evaluation comprised three phases (Figure 1).

Figure 1. The realist evaluation process

Setting

2.2 NHS Ayrshire & Arran is in the mid-south west of Scotland and covers an area of 750,464 square hectares. It comprises a mix of rural and urban development. It has three local authority areas - North, South and East Ayrshire. In mid-2013, the estimated population was 372,210 (National Records of Scotland, 2015) and the total births in 2013 was 3,640 (ISD Scotland, 2014). NHS Ayrshire and Arran has about 80 health visitors and new health visitors are currently being trained.

Sample and recruitment for phases 1 and 2

2.3 Participants of this evaluation were recruited from all three local authority areas. The detail of the steps taken for sample and recruitment are provided in Appendix 1.

Phase 1. Identifying the programme theory/logic model

2.4 Eight key stakeholders (managerial staff) involved in either developing or implementing the structured, increased home visiting service in NHS Ayrshire and Arran participated in a focus group.

Phase 2. Testing the programme theory

2.5 Overall, 25 health visitors participated in Phase 2 of the evaluation. Of this, nine were from South Ayrshire, whilst eight were from North Ayrshire and East Ayrshire had six. The other two were based in Ayrshire central hospital and covered all three areas. All health visitors except six, had experience of delivering both previous and current service.

2.6 Twenty-two parents participated in the evaluation. North and East Ayrshire had eight participants each, whilst the remaining six were from South Ayrshire. All parents except five, had used both current and the previous service.

Data collection for phases 1 and 2

2.7 Details of how data were collected in phases 1 and 2 and all topic guides used are provided in appendices 2-5.

Analysis of phases 1, 2, and 3

2.8 In phase 1, the transcribed focus group data were synthesised together with programme implementation documents to develop a logic model (see figure 2) of the enhanced health visiting service in NHS Ayrshire and Arran. In phase 2, qualitative data were transcribed and then thematically analysed based on the logic model. Transcripts were coded, analysed by themes and presented in a narrative fashion. The analysis was facilitated by the software package QSR NVivo 10. In phase 3, the findings from phase 2, and the logic model from phase 1 were disassembled, enabling the analysis to focus on explaining ways in which the programme mechanisms unfolded or did not unfold in practice. This analysis was presented as context, mechanism and outcome components, which further aided understanding of the enhanced health visiting programme.

2.9 Where necessary, published data were obtained to support the interpretation of the qualitative data. Published data covered before and after implementation of the programme and provided an indication of the impact of the programme on certain outcomes, even though cause and effect relationship cannot be assumed.

Ethics approval

2.10 Ethics approval for the evaluation was granted by the University of Edinburgh Centre for Population Health Sciences ethics committee and complied with research governance procedures in NHS Ayrshire and Arran.