Evaluation of the Family Nurse Partnership Programme in NHS Lothian, Scotland: 4th Report - Toddlerhood

2 Summary of Evaluation Aims and Methods

Evaluation aims and objectives

2.1 The overall aim of the evaluation of FNP in Scotland is 'to evaluate the implementation of the programme in Scotland (Lothian), focusing on process and understanding how the programme works in the Scottish context'. In particular, it is intended to assess:

• Whether the programme is being implemented as intended (and if not, why not)
• How the programme works in Scotland (Lothian), looking in particular at:
  • How Nurses, clients and wider services respond to the programme
  • What factors support or inhibit the delivery of the programme, and
  • Implications for future nursing practice
• What the potential is for FNP to impact on short, medium and long-term outcomes relevant to Scotland.

2.2 The evaluation focuses on the experience of delivering FNP in the first Scottish site in NHS Lothian, Edinburgh to the first cohort of clients, with the expectation that the learning from this will help inform decisions and practice relating to further roll-out of FNP in Scotland.

2.3 This evaluation is not an experimental impact evaluation and cannot, therefore, conclusively establish causal links between FNP and particular outcomes. However, where possible, it reports on the evidence for the potential for FNP to impact on key outcomes for parents, children and services, drawing primarily on the accounts of clients and Family Nurses. The current 'Building Blocks' Randomised Controlled Trial in England (described in Sanders et al, 2011 and due to report final results in 2014) will be able to provide causal evidence, and is therefore likely to be of considerable importance for those with an interest in FNP in Scotland too. Further details about the remit for the evaluation are provided in Martin et al (2011).

Monitoring and evaluation framework

2.4 The evaluation of FNP in NHS Lothian, Edinburgh is informed by a monitoring and evaluation framework, developed by Jacki Gordon in discussion with key stakeholders from Scottish Government, NHS Lothian and City of Edinburgh Council. The key questions set out at the start of the findings chapters in this report are taken from this framework (see Martin et al, 2011 for full details).

Overview of methods and data included in this report

2.5 The evaluation addresses the aims set out above using a range of quantitative and qualitative methods. These are described in full in Martin et al (2011). This fourth report draws on:

• Quantitative data collected and collated by the NHS Lothian, Edinburgh FNP team for all clients covering the toddlerhood period. This data is routinely collected by Family Nurses and collated and provided to the ScotCen evaluation team as anonymised, aggregate figures.^[3] Quantitative data provides information about the extent to which FNP in NHS Lothian, Edinburgh have met the fidelity requirements of the programme with their first cohort, and provides wider context for the more qualitative findings which comprise the bulk of this report.
• Qualitative data from:
  • A smaller sub-sample of FNP clients, interviewed around 22-24 months after their babies were born. This was the fourth occasion on which this longitudinal client 'panel' was interviewed for the evaluation. Of the original panel of 15 clients recruited to the evaluation, 13 were re-interviewed at 22-24 months.^[4]
  • Clients' 'significant others' - clients were asked to nominate a 'significant other' who could speak to the research team about their views of FNP. Seven interviews with significant others (including two with the baby's father, four with the client's mother and one with a friend of the client) were carried out around 24 months after clients joined FNP.^[5]
  • The NHS Lothian, Edinburgh Family Nurse team (including the Nurse Supervisors), interviewed in early 2013.^[6] Again, this was the fourth round of interviews with the team.
  • Ongoing interviews with the FNP National Lead for Scotland.

2.6 Qualitative data explores experiences of the programme in depth, including aspects that are difficult to quantify.

2.7 The evaluation team had also planned to conduct three focus groups over the course of the evaluation with NHS Lothian, Edinburgh FNP clients who were not selected for the longitudinal panel interviews. However, the first two of these groups were not successful, with a high level of non-attendance on the day. The evaluation team attended a graduation event in March 2013 in order to gather informal feedback from a wider group of clients and to check the findings included in these reports for face validity.

Reporting conventions

2.8 As discussed above, detailed information from FNP clients, their 'significant others', Family Nurses and key stakeholders were collected using a qualitative approach. Qualitative samples are generally small, and are designed to ensure a range of different views and experiences are captured. It is not appropriate given the number of interviews conducted to draw conclusions based solely on the qualitative data about the prevalence of particular views or experiences of FNP. Given this, where possible quantifying language, such as 'all', 'most' or 'a few', is avoided when discussing qualitative findings.

2.9 It is also worth noting that interviews with clients, significant others, Family Nurses and key stakeholders focused on their perceptions of FNP. These perceptions may not necessarily always agree with each other, or with the views of others on how the programme works. However, they each provide valuable information about how the programme is experienced from the point of view of different stakeholders.

2.10 In order to protect the anonymity of clients and Family Nurses, participants are referred to by numbers only. Where participants were in unique or identifiable roles, they were given the opportunity to review their transcripts and/or any sections of the report that summarised their views in a way that might be identifiable or which quoted them directly. Any requests to remove a quote or potentially identifiable summary were always respected.

2.11 Finally, this report does not include any explicit comparisons with findings from the implementation evaluation of FNP in England (Barnes et al, 2008, 2009 and 2011). This is because the implementation of FNP in Scotland has been informed by the experiences of FNP in England. Any comparisons may not, therefore, be entirely comparing like with like.