Whole Family Wellbeing Funding (WFWF) Programme - year 2: process and impact evaluation - summary report

Evaluation aims and approach

Aims

In September 2023, the Scottish Government commissioned IFF Research to undertake a process and impact evaluation of the WFWF Elements 1, 2 and 3 in Year 2 (see Glossary in Annex 3 of the full report). This built on the process evaluation of Elements 1 and 2 conducted in Year 1 (Scottish Government, 2024b).

The Year 2 evaluation aimed to:

• Understand how CSPPs continued to use their WFWF allocation;
• Understand and assess the extent of achievement of the outcomes related to the four core components of WFWF based on the availability and strength of evidence, and the factors contributing to these outcomes;
• Assess the extent to which Element 3 projects align with the long-term outcomes of WFWF logic model^[3], and their delivery progress; and
• Provide evidence to inform and enable Scottish Government and CSPPs to make future improvements to WFWF policy and practice.

To fulfil these aims, IFF researchers adopted multiple methods (described below) to gather data, analyse this and assess the evidence overall.

Qualitative case studies

Five of the 30^[4] CSPPs were selected as case studies to provide coverage across a range of areas including amount of funding received; whether they participate as a Collaborative Partnership through Element 2; and rurality, among other factors. Each case study included qualitative discussions with strategic leads (senior leaders within each CSPP), local managers of WFWF activities, frontline practitioners, and children, young people and families on topics related to the aims of the evaluation listed above (see Annex 1 of the full report). These discussions were carried out between February and August 2024.

Secondary data analysis

The Scottish Government developed a template for CSPPs to provide annual reports on their WFWF plans. The template aimed to gather information about CSPPs’ WFWF activities to date, funding spent, the key challenges and successes in delivering WFWF activity and evidence of the extent of achievement of outcomes. These annual WFWF progress templates were returned by 22 of 30 CSPPs. Eleven reports from the twelve projects were also submitted, along with two documents related to progress on the Collaborative Partnerships. IFF researchers then conducted a quality assurance review, analysed this data to draw out key themes aligned to each of the research questions, and triangulated the findings with the qualitative data outlined above.

Theory based evaluation: contribution analysis

The evaluation used a contribution analysis method (see Glossary in Annex 3 of the full report). This method compares the existing WFWF logic model with the evidence collected to determine if the WFWF contributed to the observed outcomes. Contribution analysis is a rigorous approach for evaluating complex systems. Annex 5 of the full report has more detail on the contribution analysis approach taken for this evaluation.

The evaluation team assessed both the level of achievement of each outcome and the strength of the evidence base for each outcome, for each CSPP. These assessments were then combined to form overall assessments for each outcome across all CSPPs where evidence was available (see Table 1 below for the resulting overall assessment categories).

Where an outcome was considered to have been achieved through this overall assessment, this did not necessarily mean this was achieved across all CSPPs. Nor did it mean that activities associated with this outcome had stopped or can/should do so. Continued achievement of outcomes may depend on ongoing delivery. Additionally, in future years of the WFWF programme, the assessment of an outcome may change, due to changes to local delivery, contextual factors, and availability of evidence.

Table 1 Overall assessment categories

Overall assessment: Definition

Outcome achieved

Where the evaluation evidence was strong and consistent to conclude that this outcome was achieved.

Outcome partly achieved

Where the evaluation evidence was strong and consistent to conclude that there was some achievement of this outcome; or where evidence was positive about full achievement, but was weak^[5].

Outcome not achieved

Where the evaluation evidence was strong and consistent to conclude that this outcome was not achieved.

Inconclusive

Where the evaluation evidence was insufficient (including if there was no evidence) to draw a confident conclusion about the extent to which the outcome was achieved.