Publication - Research and analysis
Whole Family Wellbeing Funding (WFWF) Programme - year 2: process and impact evaluation - full report
Full report of the year 2 process and impact evaluation of the Whole Family Wellbeing Funding (WFWF).
Annex 1: Year 2 Evaluation Framework
Key evaluation aim 1: Understand how CSPPs continued to use their WFWF allocation.
| Key research questions | Measures | Source - unless stated otherwise, all qual covers Element 1& Element 2 |
|---|---|---|
| A. How have CSPPs spent their WFWF allocation? | initial plans; recruitment; workforce training and development; multi-agency communication and working development; governance arrangements; support design; support delivery (new or expanded); performance monitoring; other | annual reports strategic leads interviews Manager interviews |
| B. How have CSPPs spent their WFWF allocation? | For Element 3 only - activities/type of support model, including relevant audiences involved | secondary analysis of Element 3 only |
| C. What changes in spending have occurred over Year 1 and 2 of the funding, and why? | activities/topics CSPPs dropped, added or revised, why who’s involved (including third sector) and in what way how were these decisions agreed what no longer funding, why | annual reports strategic leads interviews Manager interviews |
| D. What other funding streams, other than SG WFWF funds, are now funding family services? | type of funding streams (e.g. local and from where; national and from where), and their use/purpose | strategic leads interviews Manager interviews |
| E. How are CSPPs making decisions about spending/changes to spending? | what triggers changes to spend who’s involved (including third sector) and in what way how are spending changes decisions agreed | strategic leads interviews Manager interviews |
| F. What WFWF support (other than funding) has been used and what are CSPPs experiences of this support? | experiences of Khub; Learning into Action network; Element 2 support; Element 3 support; SG WFWF lead, Collective leadership support (new from Jan 2024) | Strategic leads interviews manager interviews secondary analysis - learning into action evaluation Element 3 secondary analysis |
| F. What WFWF support (other than funding) has been used and what are CSPPs experiences of this support? | SG-led National Support Team (NST) | Element 2 strategic lead and manager interviews only |
Key evaluation aim 2: Understand which outcomes related to Children, young people and families at centre of service design are evidenced, and the factors contributing to these outcomes.
| Key research questions | Measures | Source - unless stated otherwise, all qual covers Element 1 & Element 2 |
|---|---|---|
| A. To what extent has the funded activity achieved intended early outcomes related to children, young people and families at the centre of service design? | experiences of support accessed in relation to being at centre of service design - relevance, accessibility, proportionate, satisfaction | children, young people and families’ interviews |
| B. Were there any unintended outcomes? | perceptions of children, young people and families -related unintended/other outcomes achieved | children, young people and families interviews Strategic leads interviews Manager interviews Practitioner groups |
| C. How have CSPPs monitored outcomes? | experiences with setting up and monitoring these outcomes, all CSPP partners | Strategic leads interviews manager interviews |
| D. What factors contributed to the outcomes relating to children, young people and families at the centre of service design? | perceptions of children, young people and families -related factors leading to each outcome achieved: aware of how they can provide feedback; option of different, audience-appropriate ways of providing feedback | children, young people and families interviews practitioner groups |
| D. What factors contributed to the outcomes relating to children, young people and families at the centre of service design? | perceptions of family services-practitioner related factors leading to each outcome achieved: children, young people and families service feedback engagement skills (e.g. including children, young people and families co-design, non-verbal feedback collection); | Children, young people and families interviews manager interviews practitioner groups |
| D. What factors contributed to the outcomes relating to children, young people and families at the centre of service design? | perceptions of family services-manager factors leading to each outcome achieved: develop/refine systems to continuously and appropriately capture children, young people and families support feedback and use it in service design; regularly communicate to children, young people and families how feedback informed support changes | manager interviews practitioner groups |
| D. What factors contributed to the outcomes relating to children, young people and families at the centre of service design? | role and extent of involvement of third sector in achieving these outcomes | manager interviews practitioner groups |
| E. What are the differences in outcomes for Element 1 and 2 CSPPs? And what are the reasons for these? | through analysis, comparing findings from CSPPs receiving Element 1 with Element 2 | N/A |
Key evaluation aim 3: Understand which outcomes related to availability and access are evidenced, and the factors contributing to these outcomes.
| Key research questions | Measures | Source - unless stated otherwise, all qual covers Element 1 & Element 2 |
|---|---|---|
| A. To what extent has the funded activity achieved intended early outcomes related to support availability and access? | experiences of support accessed = timely, relevance, accessibility, proportionate, satisfaction, for as long as they need it | children, young people and families' interviews |
| B. Were there any unintended outcomes? | perceptions of unintended/other outcomes achieved | annual reports secondary analysis of other documents/ reports strategic leads interviews manager interviews children, young people and families interviews practitioner groups |
| C. How have CSPPs monitored outcomes? | experiences with setting up and monitoring these outcomes, all CSPP partners | Strategic leads interviews manager interviews |
| D. What factors contributed to the outcomes relating to support availability and access? | children, young people and families -related factors leading to each outcome achieved: support to give feedback if they choose to, recognise support is informed by them, feel available is relevant to their needs, know what family support services relevant to them is available, know how to access relevant family support services, all family members who want to access support do so with ease, extent of trust in practitioners | children, young people and families interviews manager interviews practitioner groups |
| D. What factors contributed to the outcomes relating to support availability and access? | family services-practitioner related factors leading to each outcome achieved: practitioners develop relevant, targeted family services promotional materials and use them; receive guidance on available family support services and using referral pathways; use referral pathways | manager interviews practitioner groups |
| D. What factors contributed to the outcomes relating to support availability and access? | family services -manager factors leading to each outcome achieved: establish relevant referral pathways into and between family services; establish infrastructure and indicators for multi-agency performance monitoring of availability and access; monitor family services availability and access and implements necessary changes | strategic leads interviews manager interviews practitioner groups |
| D. What factors contributed to the outcomes relating to support availability and access? | role and extent of involvement of third sector in achieving these outcomes | strategic leads interviews manager interviews practitioner groups |
| D. What factors contributed to the outcomes relating to support availability and access? | Above, for Element 3 only | secondary analysis of Element 3 |
| E. What are the differences in outcomes for Element 1, 2 CSPPs? And what are the reasons for these? | through analysis, comparing findings from CSPPs receiving Element 1 with Element 2 | N/A |
Key evaluation aim 4: Understand which outcomes related to whole-system approach are evidenced, and the factors contributing to these outcomes.
| Key research questions | Measures | Source - unless stated otherwise, all qual covers Element 1 & Element 2 |
|---|---|---|
| A. To what extent has the funded activity achieved intended early outcomes related to a whole systems approach to family services? | learning is shared effectively within CSPPs and between CSPPs and SG Capacity and capability to do transformation work | Element 2 strategic lead and manager interviews only |
| B. Were there any unintended outcomes? | perceptions of unintended/other outcomes achieved | annual reports secondary analysis of other documents/ reports strategic interviews manager interviews practitioner interviews |
| C. How have CSPPs monitored outcomes? | experiences with setting up and monitoring these outcomes, with all CSPP partners | strategic leads interviews manager interviews |
| D. What factors contributed to the outcomes relating to a whole systems approach to family services? | needs analysis conducted and used; systems analysis conducted and used; change planning programme; monitoring performance and communicating lessons from pilots/scaled up approaches; use of locally based multi-agency services coordinating support; monitoring outcomes, Learning into Action Network; budget decisions aligned with provision of preventative family support | strategic leads interviews manager interviews practitioner interviews |
| D. What factors contributed to the outcomes relating to a whole systems approach to family services? | role and extent of involvement of third sector in achieving these outcomes | Strategic leads interviews manager interviews practitioner groups |
| E. What are the differences in outcomes for Element 1, 2 CSPPs? And what are the reasons for these? | through analysis, comparing findings from CSPPs receiving Element 1 with Element 2 | N/A |
Key evaluation aim 5: Understand which outcomes related to leadership, workforce and culture are evidenced, and the factors contributing to these outcomes.
| Key research questions | Measures | Source - unless stated otherwise, all qual covers Element 1 & Element 2 |
|---|---|---|
| B. Were there any unintended outcomes? | perceptions of unintended/other outcomes achieved | annual reports Secondary analysis of other documents/ reports Strategic interviews Manager interviews Children, young people and families interviews Practitioner groups |
| C. How have CSPPs monitored outcomes? | experiences with setting up and monitoring these outcomes, all CSPP partners | strategic leads interviews Manager interviews |
| D. What factors contributed to the outcomes relating to leadership, workforce and culture? | recruitment practices; workforce training and development activities; streamlined, accessible and less bureaucratic reporting mechanisms | strategic leads interviews manager interviews practitioner interviews Element 2 secondary analysis |
| D. What factors contributed to the outcomes relating to leadership, workforce and culture? | Collective Leadership Series; National Team support | Element 2 secondary analysis only |
| D. What factors contributed to the outcomes relating to leadership, workforce and culture? | role and extent of involvement of third sector in achieving these outcomes | strategic leads interviews manager interviews practitioner groups |
| E. What are the differences in outcomes for Element 1, 2 CSPPs? And what are the reasons for these? | through analysis, comparing findings from CSPPs receiving Element 1 with Element 2 | N/A |
Key evaluation aim 6: Extent to which Element 3 projects align with the long-term outcomes of WFWF and their delivery progress.
| Key research questions | Measures | Source - unless stated otherwise, all qual covers Element 1 & Element 2 |
|---|---|---|
| A. What long-term Element 3 outcomes match WFWF long-term outcomes and which differ? | Element 3 stated project outcomes that match WFWF long-term outcomes Element 3 stated project outcomes that differ from WFWF long-term outcomes | Element 3 secondary analysis |
| B. What factors are supporting Element 3 project delivery and what factors are limiting Element 3 project delivery? | facilitating factors for project delivery limiting factors for project delivery | Element 3 secondary analysis |
Key evaluation aim 7: Provide evidence for policy and practice to inform future WFWF improvement.
| Key research questions | Measures | Source - unless stated otherwise, all qual covers Element 1 & Element 2 |
|---|---|---|
| A. What examples are there of good practice in whole family support? | examples of successful responses to challenges faced by CSPPs | manager interviews practitioner interviews secondary analysis |
| B. What conditions are necessary for consolidating improvements in the funded CSPPs? | perceptions of requirements for sustaining changes | strategic leads interviews manager interviews practitioner interviews Secondary analysis |
| C. What are recommendations for the remaining funding period (until 2026)? | Based on analysis of above | N/A |
Contact
Email: socialresearch@gov.scot
There is a problem
Thanks for your feedback