Family Nurse Partnership evaluation: methods and process

Appendix 4: Statistical analysis plan.

With no primary outcome, equal importance will be given to each short and medium term outcome. All comparative analyses will be pre-specified and conducted on an intention to treat (ITT) basis. ITT in this study means that the analysis will include everyone who started the programme, according to their original ‘allocation’, i.e. the intervention group will be women enrolled in FNP regardless of the treatment (intervention) they received.

All analyses will compare outcomes (intervention effect) between the two groups (Cases and Controls) using multilevel regression models, to allow for clustering of outcome within NHS HB, and FNP team/cohort (where more than one team runs within a HB). Intervention effects will also be examined over time and between different geographical areas (HB and team/cohort) by fitting multilevel models and interactions (group x year). Alongside the estimate of effect, for all outcomes a 95% confidence interval (CI) and p-value will be presented. Sensitivity analyses will explore the effect of multiple comparisons and also adjustment for any hypothesized confounders of outcomes at baseline.

Binary outcomes will be modelled using a logistic model and presented as odds ratios comparing the odds of an event in a case compared with the Control. For continuous outcomes a multilevel linear model will be fitted and results presented as a difference in means (Case minus Control group). Time to event analyses (e.g. cessation of breastfeeding, time to subsequent birth) will be analysed using a proportional hazards regression model and results presented as hazard ratios. We will ascertain if the proportional hazards assumption has not been violated by inspecting the log (−log(survival)) plot and Schoenfeld residuals. Count data will be analysed using a Poisson multilevel model. If the distribution of events display signs of over dispersion (greater variance than might be expected in a Poisson distribution), then a Negative Binomial model will be used. Results will be presented as the incidence rate ratio in the case arm compared to the Control group. The impact of FNP visits (dosage of intervention) on outcomes will be explored as a sensitivity analysis. Adherence will be defined as the number of FNP visits that a Client received during their programme enrolment overall or by phase (pregnancy, infancy, toddler), dependent on outcome. 

Subgroup analyses

We will examine the effect of FNP on pre-specified outcomes by modelling interactions between FNP uptake and pre-specified maternal baseline characteristics such as ever been on the child protection register/ looked after child, substance misuse issues and child demographics such as gender. Effect sizes alongside 95% CIs and p-values will be reported. 

Sensitivity analyses

We plan to conduct several sensitivity analyses: 

• Adjustment for any imbalance in confounders (pre-exposure maternal and baby characteristics). Note that these will be assessed for the differing denominators (study populations) dependent on outcome; 
• Adjustment for multiple testing;
• The impact of FNP visits (dosage of intervention) on outcomes will be explored as a sensitivity analysis. Adherence will be defined as the number of FNP visits that a client received during FNP enrolment overall or by phase (pregnancy, infancy, toddler), depending on the outcome. We will use pregnancy phase visits for short-term outcomes such as birth weight, and visits across all phases for longer-term outcomes, to examine the impact of the fidelity of intervention delivery on effectiveness. 
• For certain outcomes such as smoking status and drug use, there may be missing data. Multiple imputation using chained equations accounting for the clustered nature of the data will be performed in addition to a complete case analysis(23). 
• For outcomes that may have variable follow-up times (such as 27-30 month health visitor review) and in outcomes that variable length of follow-up might have an impact, such as in development, we will examine the average length of time of follow-up between cases and Controls and where imbalanced, further adjust. 
• For outcomes with known constraints on data quality, such as mandatory drug and alcohol questions at booking since April 2011, analyses will be restricted to consider these quality issues for example excluding years where data is known to be of variable quality.

Intervention effects will also be examined over time (e.g. year of recruitment), site maturity (e.g. if outcomes are more improved for second cohorts within a given site) and between different geographical areas (HB and team/cohort) by fitting interactions (group x year). These analyses are essentially exploratory and will require cautious interpretation. Effect sizes alongside 95% confidence intervals and p-values will again be reported. 

How to access background or source data

The data collected for this  social research publication:

☐ are available in more detail through Scottish Neighbourhood Statistics     

☐ are available via an alternative route

☐ may be made available on request, subject to consideration of legal and ethical factors. Please contact <email address> for further information. 

☒ cannot be made available by Scottish Government for further analysis as Scottish Government is not the data controller.