Evaluation of the Family Nurse Partnership Programme in NHS Lothian, Scotland - Late Pregnancy and Postpartum

2 SUMMARY OF EVALUATION AIMS AND METHODS

Evaluation aims and objectives

2.1 The overall aim of the evaluation of FNP in Scotland is 'to evaluate the implementation of the programme in Scotland (Lothian), focusing on process and understanding how the programme works in the Scottish context'. In particular, it is intended to assess:

• Whether the programme is being implemented as intended (and if not, why not)
• How the programme works in Scotland (Lothian), looking in particular at:
  • How Nurses, clients and wider services respond to the programme
  • Implications for future nursing practice, and
  • What factors support or inhibit the delivery of the programme
• What the potential is for FNP to impact on short, medium and long-term outcomes relevant to Scotland.

2.2 The evaluation focuses on the experience of delivering FNP in the first Scottish site in NHS Lothian, Edinburgh, with the expectation that the learning from this will help inform decisions and practice relating to further roll-out of FNP in Scotland. It is hoped that many of the findings in this second evaluation report will be relevant to new FNP sites in both Scotland and further afield, and to Health Boards considering whether or not to apply to deliver FNP locally.

2.3 The evaluation is not a formal impact evaluation. Where possible, it reports on the evidence for the potential for FNP to impact on key outcomes for parents, children and services. However, it does not have an experimental or quasi-experimental design and it is not, therefore, possible to conclusively establish causal links between FNP and particular outcomes. The current 'Building Blocks' RCT in England (described in Sanders et al, 2011) will be able to provide such causal evidence, and is therefore likely to be of considerable importance for those with an interest in FNP in Scotland too.

2.4 Further details about the remit for the evaluation are provided in Martin et al (2011).

Monitoring and evaluation framework

2.5 The evaluation of FNP in NHS Lothian, Edinburgh is informed by a monitoring and evaluation framework, developed by Jacki Gordon in discussion with key stakeholders from Scottish Government, NHS Lothian and City of Edinburgh Council. The key questions set out at the start of each findings chapter in this report are taken from this framework. For further details, see Martin et al (2011).

Overview of research methods

2.6 The evaluation is addressing the aims set out in Chapter 1 using a range of quantitative and qualitative methods, comprising:

• Analysis of quantitative data collected by the FNP team in NHS Lothian, Edinburgh for all clients in the first FNP cohort in Scotland. A large volume of data is collected on all clients by Family Nurses and collated and provided to the external evaluation team as anomysied, aggregate figures.⁵
• Qualitative research with a smaller sub-sample of FNP families. Qualitative data compliment the quantitative data collected for all clients by providing a more nuanced account of experiences of programme, including aspects of the programme that are difficult to quantify. A panel of mothers from the first FNP cohort in NHS Lothian, Edinburgh are being interviewed at four points as they move through the programme. 15 clients were initially recruited to this panel, with the expectation that at least 10 would be willing to be re-interviewed on all four occasions. In addition, participants 'significant others' are being interviewed at around 3-6 months and around 21-24 months after joining the programme. Finally, the evaluation team plans to conduct three focus groups with clients not included in the qualitative panel over the course of the evaluation.
• Qualitative interviews with the NHS Lothian Edinburgh Family Nurse team. Family Nurses (including the Nurse Supervisor) in Edinburgh are also being interviewed on four occasions during the evaluation, roughly mirroring the timing of the client panel interviews.
• Qualitative interviews with national and local stakeholders. A series of key national (Scotland and England) and local (Lothian) stakeholders were interviewed to inform the first evaluation report (Martin et al, 2011). The National FNP Lead for Scotland is being interviewed throughout the evaluation, while it is expected that the views of more local stakeholders, probably drawn from midwifery and General Practice, will be included in the next evaluation report, in order to explore their views of FNP and its relationship with wider services.

2.7 Further detail about the research methods used by the evaluation, and the purpose and contribution of each component, are included in Martin et al (2011).

Data included in this report

2.8 This second report of the evaluation of FNP in Scotland focuses on the experience of delivering and receiving FNP in the pregnancy and post-partum period. It uses data drawn from:

• Quantitative data collected and collated by the NHS Lothian Edinburgh FNP team for all clients covering the pregnancy period and the period up to 6 weeks post-partum, and
• Qualitative data from:
  • panel clients, interviewed around 3 months after their babies were born
  • clients' 'significant others', interviewed after the clients' 3 month interviews
  • the NHS Lothian, Edinburgh Family Nurse team, interviewed in October/November 2011.

2.9 The report is also informed by an additional paired depth discussion with two further FNP clients (not included in the longitudinal panel) and ongoing interviews with the FNP National Lead for Scotland.

2.10 Of the original panel of 15 clients recruited to the evaluation, 14 were re-interviewed around 3 months after their babies were born, while one dropped out of the evaluation. The evaluation team received contact details for 13 potential 'significant others', 6 of whom were interviewed (including a combination of participants' partners and their mothers). Two of those nominated declined to participate, while five were not contactable during the fieldwork period.⁶

2.11 As discussed above, the evaluation team planned to conduct three focus groups with NHS Lothian, Edinburgh FNP clients who were not selected for the longitudinal panel interviews. The first group was scheduled for October 2011. However, although 7 clients had indicated that they would attend, on the day only two clients attended. In light of this, the evaluation team is reviewing its strategy in advance of the next group (planned for summer 2012). The two clients who were interviewed nonetheless had valuable insights about their experiences of FNP, which are included in this report alongside those of panel clients.

Reporting conventions

2.12 As discussed above, detailed information from FNP clients, their 'significant others', Family Nurses and key stakeholders were collected using a qualitative approach. Qualitative samples are generally small, and are designed to ensure a range of different views and experiences are captured. It is not appropriate given the number of interviews conducted to draw conclusions based solely on the qualitative data about the prevalence of particular views about or experiences of FNP. Given this, where possible quantifying language, such as 'all', 'most' or 'a few', is avoided when discussing qualitative findings.

2.13 It is also worth noting that interviews with clients, Family Nurses and stakeholders focused on their perceptions of FNP. These perceptions may not necessarily always completely agree with each other, or with the views of others on how the programme works. However, they each provide valuable information about how the programme is experienced from the point of view of different stakeholders.

2.14 In order to protect the anonymity of clients and Family Nurses, participants are referred to by numbers only. Where there are small number of participants in particular roles, it can be more difficult to guarantee confidentiality in reporting their views. In light of this consideration, where participants were in unique or identifiable roles, they were given the opportunity to review their transcripts and any sections of the report that summarised their views in a way that might potentially be identifiable or which quoted them directly. Any requests to remove a quote or potentially identifiable summary were always respected.

2.15 Finally, this report does not include any explicit comparisons with the implementation evaluation of FNP in England (Barnes et al, 2008, 2009 and 2011). This is because the implementation of FNP in Scotland has been informed by the experiences of FNP in England. Any comparisons may not, therefore, be entirely comparing like with like.