Whole Family Wellbeing Funding (WFWF) Programme - year 2: process and impact evaluation - summary report

Summary of evaluation findings

The Year 2 evaluation shows varied achievement of the different outcomes outlined in the WFWF logic model. CSPPs prioritised availability and access and child- and family-centred service design, with outcomes related to these core components often being partly achieved. There was less achievement of the outcomes in the whole systems approach and leadership, workforce, and culture core components. Strategic leads argued that these outcomes need more time to emerge and are harder to measure, as they are more conceptual and depend on some degree of achievement of outcomes in other core components first. Overall, the evaluation’s evidence suggests that many WFWF outcomes showed some achievement, but the extent of achievement varied between CSPPs. While the evidence was largely inconclusive for some of the outcomes, there was no evidence captured by the evaluation of these outcomes worsening. There were also no outcomes assessed as having not been achieved.

The evaluation’s key findings are summarised below, with references provided to the detailed findings within the relevant sections of full report.

How CSPPs continued to use their WFWF allocation

Most CSPPs continued to implement their initial funding plans, with greater focus in Year 2 on delivery, following exploration and set-up activity in Year 1. The funding’s flexible conditions enabled some CSPPs to carry over funds from Year 1 to Year 2, where they determined that this was required based on the local situation and plans. Other CSPPs experienced barriers to spend, including recruiting new staff and finding appropriate locations to deliver support.

In Year 2, the 12 projects related to wider Scottish government policy areas were in the early stages of delivery. They focussed on design and set-up, such as establishing governance and monitoring processes, recruiting and training staff, and finding locations for delivering support.

There was early evidence of CSPPs aligning existing local funding of family support services to ensure staffing for services can be sustained; to deliver related national priorities (such as the Promise); and improve efficiency. Key factors enabling this included collaborative working and effective leadership across the CSPP.

CSPPs welcomed Scottish Government support, particularly the newsletters, the Learning into Action Network, and having a dedicated and responsive contact at Scottish Government. However, staff capacity and mixed awareness of the support available to CSPPs limited their ability to fully benefit from it.

Section 3 of the full report provides more details on the findings above.

Outcomes and contributing factors: availability and access

The table below shows that the evaluation in Year 2 found that three of the outcomes in the availability and access core component of the WFWF logic model had been partly achieved. There was insufficient evidence to draw a confident conclusion about achievement of one outcome. A descriptive summary of the evaluation’s key findings follows this table.

Outcomes:

• Children, young people and families are aware of how to access relevant family services support
• Focus on prevention and early intervention, and children, young people and families say family support is accessible and provides early help and support where and when it suits them
• Children, young people and families say they feel positive and trusting of services

Overall assessment: Outcomes partly achieved

Outcomes:

• More children, young people and families receive whole family support through referrals or self-referrals

Overall assessment: Inconclusive

Within WFWF-related activities, families typically knew how to access relevant family support when they had already accessed or been referred to children’s services. Families’ awareness of available WFWF support was low if they had not previously accessed non-WFWF support.

Factors contributing to increased awareness of and access to family support offered through activities related to WFWF included: advice from a trusted source sharing their experience or recommendation of the support; information sharing about locally available support (for example, new webpages and new promotional materials); and having a dedicated support worker to signpost to other services and provide tailored support. Barriers limiting awareness of and access to family support included: practitioners describing services in a way which led to families deciding not to access services; waiting list times for some services; limited staff capacity to provide or refer to the right services; and case management system functionality.

Many CSPPs had initiatives in development or newly implemented to offer support to families at an early stage, to avoid crises. CSPPs improved their focus on early intervention by: identifying gaps in service provision through self-evaluation and service mapping; developing joined up ways of working between key partners to understand family needs and offer the right support earlier; and delivering dedicated support to groups of families or individuals with specific needs (for example, neurodiversity) to prevent needs escalating to crisis point.

Families interviewed who had received WFWF support expressed satisfaction with the support they accessed, a key factor in their perceptions about the availability and accessibility of support. Satisfaction was underpinned by feeling listened to, trusting the practitioners who supported them, and benefitting from the support received. Secondary analysis of annual reports, and CSPP staff interviewed, suggests more families, and more diverse families, accessed family support than in Year 1.

Section 4 of the full report provides more details on the findings above.

Outcomes and contributing factors: leadership, workforce and culture

The table below shows that the evaluation in Year 2 found that there was insufficient evidence to draw a confident conclusion about achievement of three of the outcomes in the leadership, workforce and culture core component of the WFWF logic model. There was strong and consistent evidence that two of the outcomes had been partly achieved. A descriptive summary of the evaluation’s key findings follows this table.

Outcomes

• Strategic leaders, family services managers and frontline practitioners (including third sector partners) are working more collaboratively and with adult services
• Increased holistic whole family support service capacity among CSPP partners – plans available for integrating scaled and new services

Overall assessment: Outcomes partly achieved

Outcomes

• CSPPs have clear and shared understanding of families' needs and how services are experienced across whole system
• Strategic leaders, family services managers and practitioners’ wellbeing is improved and integral to delivery of family services
• Strategic leaders, family services managers and frontline practitioners implement initiatives to develop a culture in CSPPs that encourages and empowers them to develop innovative family services solutions

Overall assessment: Inconclusive

Visible and embedded strategic leadership within CSPPs was a key enabling factor for workforce and culture outcomes. It helped promote multi-agency collaboration and innovation. All CSPPs demonstrated evidence of partners, including health, education, police, social work, and third sector, working collaboratively in planning and delivery.

New and improved ways of working in CSPPs encouraged more transparency and information sharing between partners. In some CSPPs, this contributed to improved understanding of families’ needs and how services were experienced across children and adult services.

There was some evidence of improved family support capacity, particularly around the recruitment of new or specialist roles and less duplication of work due to greater partner collaboration. However, CSPPs continued to experience recruitment and retention challenges, related to the need to rely on short-term staff contracts (due to the time-limited nature of WFWF) and limited numbers of qualified candidates for roles.

Most CSPPs did not provide evidence of WFWF activities related to workforce wellbeing or developing innovative approaches to delivering holistic family support in Year 2. The activities they prioritised sought to address other WFWF outcomes first. Evidence was limited of the impact of wellbeing and innovation initiatives in the few CSPPs that had implemented them.

Section 5 of the full report provides more details on the findings above.

Outcomes and contributing factors: children, young people and families at the centre of service design

The table below shows that the evaluation in Year 2 found that three of the outcomes in the children, young people and families at the centre of service design core component of the WFWF logic model had been partly achieved. There was insufficient evidence to draw a confident conclusion about achievement of one outcome. A descriptive summary of the evaluation’s key findings follows this table.

Outcomes

• Children, young people and families are actively, regularly and meaningfully engaged in service design
• Children, young people and families say that services are designed to be: free of stigma; rights-led; and to meet their specific needs
• Children, young people and families recognise available support is informed by them and feel their contributions influence service design

Overall assessment: Outcome partly achieved

Outcomes

• Family services managers and frontline practitioners develop engagement, feedback collection and co-design skills, and collate and analyse service design feedback

Overall assessment: Inconclusive

Family services managers and frontline practitioners developed skills in family engagement and co-design for individual services. CSPPs also demonstrated ongoing family engagement, using feedback to improve individual family support and services.

However, there was limited evidence of family engagement and co-design being used to inform broader service design (beyond individual services). Families also felt their feedback directly influenced their own support but were less aware of its impact on broader family support service design in their area. More work was needed to ‘close the feedback loop’, ensuring families see how their input shapes available support.

Families interviewed who had received WFWF support appreciated the non-judgmental, stigma-free support they received. There was less clear evidence of services being stigma-free from the secondary data. While CSPPs said this monitoring was happening, they did not share sufficient supporting evidence. This suggests CSPPs may need support with monitoring and reporting on perceptions of being stigma-free in their delivery.

Section 6 of the full report provides more details on the findings above.

Outcomes and contributing factors: whole systems approach

The table below shows that the evaluation in Year 2 found that there was insufficient evidence to draw a confident conclusion about achievement of two of the outcomes in the whole systems approach core component of the WFWF logic model. There was strong and consistent evidence that one outcome had been partly achieved. A descriptive summary of the evaluation’s key findings follows this table.

Outcomes

• CSPPs plan to shift towards non-siloed and aligned family support funding that matches scale of need

Overall assessment: Outcome partly achieved

Outcomes

• Strategic leads, service managers and frontline practitioners use analysed evidence from children, young people and families to inform multi-agency and partnership planning and service delivery
• Strategic leads set local budgets to invest in planning system change, and started to transform commissioning and procurement

Overall assessment: Inconclusive

Generally, strategic leads and service managers interviewed found whole system outcomes more conceptual and harder to robustly evidence. There was limited evidence demonstrating outcomes related to whole systems approach.

Strategic leads, service managers and frontline practitioners also viewed a whole system approach as difficult to achieve in the originally intended timescales for WFWF implementation. Interviewees expressed that this was mainly because it requires large-scale (often cultural) change that takes longer to set up and embed and requires some degree of achievement of outcomes in other WFWF core components first. Wider contextual factors, such as financial constraints across CSPPs, added to the challenges of implementing a whole system approach in the originally intended timescales for WFWF implementation.

Most CSPPs analysed, interpreted and used evidence from children, young people, and families to inform individual service delivery. However, there were fewer examples of this evidence being used to guide multi-agency planning across children’s and adult’s services, beyond individual services. CSPPs made some changes to support this in the future, including recruiting staff for dedicated data roles to analyse and interpret evidence across the partnership, and establishing or refining governance groups committed to systematically using this evidence in decision-making.

Some progress towards more aligned family support funding was evidenced, enabled by Third Sector Interface involvement; new commissioning and procurement processes; and multi-agency collaboration.

There was less evidence that CSPPs set local budgets to invest in planning system change. CSPP budget discussions typically focused on whether to continue WFWF activities based on existing evidence rather than having a whole system focus. Changes to approaches to commissioning and procuring services were underway, and CSPPs started involving a broader range of partners in decision-making around service commissioning.

Section 7 of the full report provides more details on the findings above.

Monitoring the impact of WFWF support

All CSPPs had systems to monitor and/or evaluate their WFWF delivery performance, typically at an activity or service level. Many CSPPs described improvements to the volume and quality of delivery performance and outcome data since Year 1. It was less common for CSPPs to have established systems for data analysis and change planning at a system level. The implications of these limitations of current data systems on whole systems change is explored in Section 7 of the full report.

The limited functionality and agility of data collection and monitoring systems was a barrier to CSPPs progressing with some WFWF outcomes at the pace envisioned by Scottish Government. For example, a limiting factor of some whole systems change outcomes was related to the challenges with the sharing of data and analysis across organisations. Case study strategic leads and managers identified two related issues as contributing to this. The lack of streamlined processes, such as misaligned management information systems, and not having data sharing agreements in place to enable sharing was one issue. The lack of agreement on how to share information across the CSPP, what information to share and how frequently this would be shared to ensure most value was a related issue. Many CSPPs did have plans to establish or enhance systems for intelligence gathering to inform WFWF support.

In Year 1 of WFWF delivery, CSPPs struggled with using data in strategic decision-making. This continued into Year 2, but more CSPPs were in the process of developing their analytical capacity to use data to inform service delivery. Good practice sharing at multi-agency CSPP events and meetings helped evidence-based decision making. To inform strategic decisions, in some CSPPs the events and meetings were structured and facilitated in ways that encouraged multi-agency practitioners to question and critically assess the practice or evidence shared.

Section 8 of the full report provides more details on the findings above.