Early Adopter Communities: process evaluation

Appendix C: Local theory of change full narrative and causal pathways

Note: To view the images on this page at a larger size, please open the PDF that can be found under “Supporting Documents”.

This appendix provides a narrative overview of the local-level theory of change for the Early Adopter Communities (EACs).

The approach to developing each theory of change was a collaborative process, with multiple opportunities for local and national stakeholders to feed into the process. The four community-specific theories of change were drafted first (in order to inform the overarching theories). The evaluation team prepared initial draft theories of change based on information gathered in the scoping phase, which were then reviewed and modified during in-person workshops. Four community-specific workshops were held in October and November 2023 with project leads, other EAC staff, and stakeholders such as school age childcare service managers. To develop the overarching theories of change, two workshops were held with the Scottish Government, national stakeholders, and EAC project leads in December 2023 and February 2024. Revised versions were circulated among EACs and the Scottish Government, with a further opportunity to feedback via email as part of a formal review process.

Why is this intervention being run?

Scottish Government statistics estimate that 24% of children were living in relative poverty and 21% in absolute poverty (after housing costs) in 2019-22. Scottish Government recognises the benefits of childcare for both parents/carers (hereafter referred to as just 'parents' for brevity), children and young people. It also recognises the role of childcare in tackling child poverty, increasing family wellbeing and reducing the poverty-related outcomes gap. Therefore, the Scottish Government remains committed to designing and building a new system of school age childcare that provides care before and after school, all year round, and supporting parents (particularly on low incomes) to have secure and stable employment.

What is this intervention?

The EACs are community-level 'tests of change' which aim to build and deliver effective school age childcare for parents and carers within target areas in Clackmannanshire, Dundee, Glasgow and Inverclyde. This investment will enable us to work with local authorities and communities to understand what it takes to deliver local childcare systems that support families with children from the early years to the end of primary school, and the difference this can make in helping eradicate child poverty.

The programme will work innovatively to increase capacity, reduce inequality of access and minimise the cost of childcare for families. It will also improve public service delivery to make it easier for families to find and access childcare and wider support. The system will be co-designed with the people who use childcare and the people who deliver it. This will ensure that funded childcare is sustainable and meets the needs of different communities across Scotland, with a focus on place-based, people-centred approaches.

What is the overall vision?

For the EACs, the aim is that each area will provide funded childcare that meets the needs of families who need it most. This will increase wellbeing and employability among families and reduce the poverty-related outcomes gap.

The aim of the wider School Age Childcare Transformational Change Programme is that Scotland will have a system of accessible and affordable school age childcare, providing care before and after school and during the holidays for primary school children from low-income households, and that children will be able to access healthy and nutritious food as part of this offer. The childcare offer should be rights based, dynamic and support choice and growth, enabling families and communities to reach their full potential.

Inputs

For the EAC element of the School Age Childcare Transformational Change Programme to run successfully, sufficient resources (inputs) from the Scottish Government and the EAC communities are needed.

Specifically, the resources needed to input into the EACs include:

• Scottish Government funding
• Other sources of additional funding
• Programme approach principles and learning offer
• EAC project guidance
• Existing evidence & experience of what works when providing childcare/ family support
• User needs
• Local political influence
• Legislation and regulation, including:
  • The Equalities Act
  • Care Inspectorate requirements
  • The United Nations Convention on the Rights of the Child Act (UNCRC)
  • Children and Young People (Scotland) Act (2014)
  • 1995 Act
  • Public Sector Reform Act (2010)
• National and local strategies and plans, including:
  • Tackling child poverty
  • Community Planning Partnerships
  • Economic strategies
  • City region and growth deals
  • Children and young people planning partnerships
  • Local Employability Partnerships
• Workforce (in terms of both time and expertise)
• Facilities and infrastructure
• Equipment for delivery (including IT infrastructure, toys, and anything else required to put on provision)
• Training resources for staff
• People and organisations, including:
  • Families
  • Childcare / activity providers
  • Sport and leisure organisations
  • Schools
  • Colleges and Further Education
  • 3^rd Sector (including representative bodies, youth work, campaign organisations)
  • Community anchor organisations
  • Local authority leadership
  • Child services (including social work)
  • Employability
  • Health (including Local Authority health teams, the NHS, and Health and Social Care Partnerships)
  • Child Poverty Pathfinders
  • Care Inspectorate
  • SSSC

Activities

To set up the programme, several system-level activities will be undertaken. These include: initial scoping/mapping exercises (to identify local needs in relation to school-age childcare); ongoing codesign of provision with families/partners; work to align EAC provision with other local initiatives; procurement, establishment and development of childcare services and the distribution of funding; and setting up project governance structures (including internal meetings, multi-agency meetings, regulatory processes and other governance processes).

Close partnership working is also expected to be required, including stakeholder engagement to 'make the case' for the programme and collaboration between the EAC areas, for example sharing challenges, learning, or best practice. Ongoing training will be provided for providers and other local actors where appropriate. EACs will also work with partners to try to help develop the childcare workforce.

Monitoring and evaluation activities will include the collection of operational data (including, for example, user needs and data on families not engaged with services). EACs will collect and analyse monitoring data for evaluation and to further develop and improve services. EACs will also be expected to share data and learning with the Scottish Government on an ongoing basis to inform national evaluation and as part of a feedback loop.

In terms of the direct delivery for families in each EAC, these activities will include: promotion and engagement with families and/or stakeholders; establishing referral pathways; establishing application/registration processes; promoting and supporting parents to use childcare benefits they may be entitled to; and delivery of funded childcare that meets the needs of target families and maximises use of childcare benefits. Childcare will include food and a range of activities (including skills-based activities and active play), as well as the delivery of family support where appropriate.

Intended outcomes

This programme is expected to achieve a range of outcomes for target families, as well as outcomes for EAC provision at a systems-level.

Intended outcomes for thechildcare system are that childcare is more affordable, flexible and accessible for target families in EACs; there is sufficient capacity to meet needs of target families; and the system is integrated within local communities. Furthermore, the programme is aiming to achieve an improved understanding of how to deliver high quality childcare and continued improvement of EAC policy; an improved understanding of wider child poverty landscape and how childcare fits into that; stakeholder buy-in and improved partnership working; increased financial security for providers including through improved awareness and use of public funding and benefits; and an enhanced childcare workforce.

Intended outcomes forcommunities include more high-quality employment and volunteering opportunities; and families more engaged with their communities.

Intended short-term outcomes forparent/carers include more families access childcare; reduced cost of living; benefits maximisation; parents/carers prepare for/start/ sustain/increase hours of work; parents/carers train/study; more families engaged with support services; and increased parental respite.

In the short-medium term, these are expected to then lead to reduced financial pressure; income from employment maximised; progression in employment; and improved wellbeing including fewer crisis points.

Intended short-term outcomes forchildreninclude access to nutritious food; children more physically active; increased social connections; increased opportunities for learning and new experiences; and improved life skills and life opportunities for children.

In the short-medium term, these are expected to then lead toimproved relationships; improved social, emotional and behavioural development; improved health and wellbeing; and increased confidence and aspirations.

Intended impacts (in the long term)

This programme is intended to contribute to a range of impacts in the longer-term, both for children and families, and on Scottish Government school-age childcare provision at a systems-level.

The intended impacts for school-age childcare in Scotland at a systems level include a sustainable system of childcare; childcare being place-based and easily accessible to families who need it; a system that is person-centred and provides the support families need; and a system that is aligned with broader SG initiatives to address child poverty.

Intended impacts on broader society include contributing to a whole system approach to tackling child poverty; and improved gender equality.

Intended impacts on communities include public service reform and partnership working; and resilient communities and increased social capital.

Intended impacts for parent/carers include fewer families living in poverty; and sustained parental mental health and wellbeing.

Intended impacts for children include that there is sustained child physical, mental and social wellbeing; that children's poverty-related outcomes gap narrows; children who grow up in poverty are not in poverty as adults; and children become responsible citizens and effective contributors to society.

Assumptions and risks

The following assumptions were identified as underpinning the success of the EAC element of the School Age Childcare Transformational Change Programme.

• There is continuity of multi-year funding for the programme.
• There is clarity around, and a shared understanding of, the vision for the programme (including around eligibility and target groups).
• Sufficient resources are available and accessible, for example providers having enough staff and space to run EAC provision.
• There is continued demand for EAC provision among target families.
• There is buy-in and support for the programme among partners and target families.
• Strong relationships are established and maintained between EAC staff, target families and EAC partners.

However, there are also various risks that may hinder the EAC element of the School Age Childcare Transformational Change Programme from making progress and achieving its intended outcomes and impacts. These include:

• A change in Scottish Government funding priorities negatively impacts funding available for the programme.
• Local authority budget cuts reduce resources/additional funding for the programme.
• A lack of skilled staff reduces providers' ability to meet demand.
• Providers experience challenges engaging with target families, resulting in fewer families using EAC provision.
• Eligibility criteria could result in a 'postcode lottery' where families who need support are unable to access high quality school age childcare (SACC), or any SACC at all, due to where they live.
• Providers experience challenges collecting the data required for to support robust monitoring and evaluation of the programme.
• Unintended consequences for childcare sector (e.g. childminders losing families to EAC provision).
• Children spending more time at formal childcare may result in less parent/child quality time spent together.
• Rapid expansion of the SACC sector may risk de-professionalising the childcare workforce.
• Rapid expansion of the SACC sector may result in a lack of adherence to safeguarding frameworks.
• More or better SACC could lead to higher demands from employers on their employees who are parents (e.g. being less flexible with hours).
• Accessing EAC provision impacting on families' social security entitlements.

Causal Pathways

The following causal pathways are based on the standardised local Theory of Change, with some elements combining or summarising particular elements for clarity. Please note, however, that these pathways and the evidence presented here is not exhaustive.

Highlighting key causal pathways within a theory of change model is particularly beneficial when models contain a lot of detail and information, in order to outline how an intervention or programme is expected to work in practice and linking up different elements of the model.

These have been informed by the Scottish Government's prioritisation of the three high-level outcomes in the School Age Childcare Delivery Framework, and the short/medium term outcomes that were considered most aligned to these, developing causal pathways to illustrate the mechanisms underpinning how EAC inputs and activities are expected to lead to these outcomes, and the availability/feasibility of data collection.

Pathway 1: Designing a system around the needs of families

This causal pathway (illustrated in Figure A.1) hypothesises that existing evidence and experience of what works when providing childcare/family support will inform EAC project guidance, but that this will be further developed through scoping and mapping exercises, and ongoing co-design with families and partners in each EAC. Operating under the assumption that EACs have access to the necessary data and families/partners engage with the co-design process, this is expected to lead to an improved understanding of how to deliver high quality childcare in different local contexts. At the same time, EACs will collect and analyse monitoring data for evaluation purposes. Evaluation data and outputs will also contribute to this improved understanding around childcare provision. EAC project leads are expected to regularly share learning with the Scottish Government on an ongoing basis as part of a feedback loop which, in turn, is expected to improve understanding at a national level and lead to continued improvement of EAC policy. This assumes that shared learning is used as part of an iterative approach to policy development. This process of improving understanding and implementing this in future policy/design of provision should then contribute to achieving a person-centred childcare system which provides the support that families need.

Pathway 1 Evidence mapping

Previous research suggests that co-design approaches hold "great promise" for public policy. Findings from Scottish Government research into person-centred approaches in the recovery from Covid-19 suggests that co-production is a key element of creating person-centred support, and that engaging with local communities can be "instrumental" in making public services more accessible. Similarly, research undertaken to review the Scottish Social Security Charter identified the co-design process as a key strength and suggested the Charter was broadly reflective of clients' needs. These findings are reflected in the Scottish Approach to Service Design which recognises that engagement with users of public services, and participation in service design from the beginning, is key to "building services around users". More generally, a research paper on co-production and value creation (Strokosch, K., and Osborne, S.P., 2020) suggests that co-production is critical in supporting the effectiveness of public service ecosystems in tackling complex societal challenges. Notably however, research by Simpson, M. et al (2019) suggests that while co-design can be meaningful, this is underpinned by an assumption that participants not only have an opportunity to express their views, but that there is a willing audience who will listen and have the power to enact genuine influence in order to act on suggestions. It also highlights the risks of co-design initiatives being dominated by more confident or articulate participants, those more directly affected and/or who have more time to spare.

Pathway 2: Reducing child poverty through supporting parents to work

This causal pathway (illustrated in Figure A.2) hypothesises that the delivery of funded childcare that meets the needs of target families will mean that parents want to use the service and their children will be happy and able to attend. As a result of children attending the provision, parents are expected to have more time to prepare for, start, sustain and/or increase hours of work or study (assuming that parents want to and are able to pursue these things). In turn, this will help to maximise parents' income from employment, thereby reducing financial pressure on the household and contributing to fewer families living in poverty in the long-term. Furthermore, families being able to access a subsidised childcare place should lead to a reduction in the cost of living for those previously spending money on childcare. This is also expected to reduce financial pressure for families and contribute to fewer families living in poverty.

Pathway 2 Evidence mapping

This pathway is supported by previous research that demonstrates the strong link between childcare and parents' ability to work or study. For example, recent evaluations of Scottish Government-funded test-of-change childcare projects have found that these projects have contributed to parents being able to start, maintain or progress their career, engage in training, or made it easier to do so (Access to Childcare Fund (Phase 2) evaluation (2023)); (Summer holiday food, activities and childcare programme evaluation - summer 2022). These studies also found that, when childcare costs were reduced, this could reduce the cost of living for parents who were at risk or experiencing child poverty (but notably only for those who would have sought childcare elsewhere regardless). Furthermore, more generally, the out of school care parent survey (2019) found that, among parents in Scotland who did not use term-time childcare but would like to, the main reason for wanting to use childcare was to allow them/their partner to work (43%), while one in five (19%) said childcare would allow them/their partner to work more hours. An NHS Health Scotland evidence review (2015) also found evidence to support this, highlighting particular benefits for lone parents.

Pathway 3: Improved child wellbeing

This causal pathway (shown in Figure A.3) hypothesises that, through attending EAC childcare provision (which includes food and a range of activities, including skills-based activities and active play), children will have increased opportunities that will contribute to improving their health and wellbeing. Specifically, the provision is expected to provide opportunities for increased social connections, learning and new experiences, which should lead to improved social, emotional and behavioural development. Additionally, provision is expected to provide opportunities for children to be more physically active and access to nutritious food, which should lead to improved child health and wellbeing. Assuming that children continue to attend EAC provision, and that some of these benefits are retained in the longer term (e.g. new learning), these short-term wellbeing benefits are intended to contribute to sustained child physical, mental and social wellbeing in the longer term.

Pathway 3 Evidence mapping

As the Scottish Government's draft framework for out of school care (2019) acknowledges, there is a "lack of robust evidence on the impacts of out of school care/activities on children and families". However, more recent Scottish Government research into previous funded childcare projects found evidence to suggest that when children have increased social connections, opportunities for learning, opportunities to be more physically active and access to nutritious food at out of school care, that these translated into improved development and wellbeing for these children (Access to Childcare Fund (Phase 2) evaluation (2023); Summer holiday food, activities and childcare programme evaluation - summer 2022). The Care Inspectorate's Out of school care: A review of children's experiences (2019) suggests that this is also the case among other childcare settings generally (among services evaluated as 'Good' or better, children's wellbeing was found to be promoted through access to a variety of play experiences and opportunities to build relationships). The NHS Health Scotland evidence review (2015) found similar positive impacts for children, especially among those from disadvantaged backgrounds in terms of learning and development. This was due to these children benefitting from the additional "resources, learning support and further stimulating activities" available at after-school care which they otherwise were unlikely to have access to. At a UK level, research by Chanfreau et al (2016) found a link between school attainment and prosocial skills among disadvantaged children who attended after school clubs.

Pathway 4: Improved parental wellbeing

This causal pathway (shown in Figure A.4) hypothesises that the delivery of family support as part of EAC projects, and the opportunity for parental respite, ultimately contributes to sustained parental mental health and wellbeing. In relation to family support, it is expected that families will engage with this and any referrals onto further support (if necessary). This should lead to more families being engaged with support services generally. Under the assumption that support services are able to effectively meet families' support needs, this should lead to improved wellbeing for parents (and their families). Assuming that families are referred onto support before reaching crisis point, this approach should also lead to a reduced likelihood of crisis points in the future as part of an early intervention approach. A parallel pathway is that when children attend the EAC childcare provision, this is expected to provide parents with a break from their caring responsibilities and to function as respite time, particularly for those parents who are not working or those with children with additional needs. This is expected to contribute to improved wellbeing and fewer crisis points, and to lead to sustained parental mental health and wellbeing in the longer term.

Pathway 4 Evidence mapping

This pathway is supported by existing evidence from Scottish Government evaluations that acknowledge the associated benefits for parental wellbeing as a result of accessing family support via childcare projects, as well as from gaining respite from parenting (Access to Childcare Fund (Phase 2) evaluation (2023); Summer holiday food, activities and childcare programme evaluation - summer 2022). Providing holistic family support has been shown to improve wellbeing for parents (and often indirectly for children). For example, an evaluation of the Children's Centres in England (2010) found that Children's Centre service use was correlated with reductions in parent–child dysfunctional interaction, chaotic environments and parental distress.