Refocused school nurse role: early adopter evaluation

2. Methods

Evaluation design

The evaluation used three different methods to obtain and analyse data. Firstly a realist framework was used for interviews with staff who had been involved in the early adopter sites. Secondly qualitative interviews were held with managers about the process of the evaluation and analysed thematically and thirdly quantitative data was collected and analysed to provide additional data about use of the refocused service.

A realist framework informed this evaluation, combining both qualitative and quantitative data analysis. Realist evaluation uses a theory-driven approach to evaluate healthcare programmes and public health interventions such as the new school nurse refocused programme. This approach evaluates a programme by exploring the complex interactions observed between the context (specific settings where the programme is implemented), mechanisms (participant's decisions and actions), and outcomes (intended and unintended effects) involved in the programme (Byng, 2011; Greenhalgh et al., 2009; Pawson and Tilley, 1997). The emphasis of realist evaluation is to explain how a programme works, whilst identifying features that can be used to improve a programme. In line with realist evaluation, this evaluation proceeded in three key phases:

• Developing the programme theory (data mainly from focus groups and interviews with nurse managers, informed by the logic model designed for the national refocusing of school nursing programme )
• Testing the programme theory (data mainly from interviews with school nurses and support workers and quantitative data)
• Refining the programme theory (using phase 1 and 2 findings to explain how the programme works in practice)

Data from the first 6 months of the programme was collected and analysed in order to report on patterns of referral both in and out of the school nursing service and the pathways being used for children. A template was developed that the early adopter sites could use to record the details of each child referred into the service. This included demographic data, data on the pathway and limited data on outcomes.

Setting

The school nurse programme was implemented in both Perth and Kinross, and Dumfries and Galloway which have the following demographic characteristics:

Table 1: Demographic Factors for two Early Adopter Sites

In terms of the proportion of their populations that are in the most deprived 20% ( SIMD quintile 1), Dumfries and Galloway is ranked 19 ^th and Perth and Kinross 24 ^th out of the 32 local councils. In other words both areas have lower populations of SIMD 1 (most deprived) and higher of SIMD 5 (least deprived) than many other areas and this should be borne in mind when interpreting the findings.

Sampling, recruitment and data collection

School nurses, support workers and nurse managers who had been involved in implementation and/or delivery of the school nurse programme were recruited from the two early adopter sites. The research team contacted a senior member of staff from each early adopter site and a meeting was held to discuss the evaluation procedures and to provide recruitment materials, which were then distributed to the wider school nurse teams. Potential participants were provided with information sheets and consent forms.

A total of 27 school nurses and wider health team members were interviewed from Perth and Kinross and Dumfries and Galloway (16 and 11 respectively), with an additional six managers taking part in focus groups and interviews. Within Perth and Kinross, all eligible school nurses and wider health team, including a Looked After Children's nurse, healthcare assistants, and the Young People's Health Team participated. However, in Dumfries and Galloway, six school nurses and wider health team members (one Band 6, one Band 5 and four Band 4/3) did not participate in the evaluation. Details of number of participants in this evaluation are provided in Table 2.

Table 2: Participants' characteristics

Two focus groups and three individual interviews (two participants also took part in the focus groups) with managers from both early adopter sites were conducted. The topic guide was designed to reflect some of the items in the logic model being used for the national refocusing of the school nurse programme. Briefly, it examined their assumptions of how they expected the programme to work, whilst exploring perceived challenges and benefits of implementation. The topic guide is available in appendix 1.

Qualitative in-depth interviews were undertaken with school nurses and support staff. Interviews explored some of the initial findings from the managers' data and consisted of questions relating to their experiences of delivering the programme, with a particular focus on challenges and perceived benefits. The interviews were informed by an interview topic guide which can be viewed in appendix 2. All focus groups and interviews were audio recorded and lasted approximately 30-60 minutes.

In addition to the above data and to offer further context to the evaluation, quantitative data was collected to provide an overview of the characteristics of the children who were seen by the school nurses in the first half of 2016. In order to do this, a form was designed which school nurses were requested to complete for each new referral. The form underwent several amendments during the course of the early adoption and so each area did not submit exactly the same data. The forms asked for information on age, sex and SIMD of child referred, reason for referral, pathway child was placed on and some information around outcomes, although this was limited. The form has since been standardized so that it can be used in both areas and, more widely if necessary.

Data analysis

Audio recordings of the interviews and focus groups were transcribed verbatim before being independently coded by two researchers ( LD and SM). After coding had been agreed upon, a thematic analysis was undertaken and findings reported in a narrative fashion based on context, mechanism and outcome ( CMO) configurations of realist evaluation. Analysis was conducted using the software package QSR NVivo 10. All data were anonymised to preserve participants' confidentiality.

The quantitative data was submitted in Excel but then converted to SPSS and analysed using standard statistical techniques.

Ethical approval

Ethical approval for the present evaluation was granted by the University of Edinburgh Centre for Population Health Sciences Ethics review group and complied with research governance procedures in both NHS Tayside and NHS Dumfries and Galloway.