Early Adopter Communities: process evaluation

Executive Summary

Overview of the EAC evaluation study

In October 2022, four Early Adopter Communities (EACs) were established in local areas within Clackmannanshire, Dundee, Glasgow and Inverclyde. They were created to provide an opportunity to test and refine what an effective school age childcare system looks like at the community level, with a focus on supporting families on low incomes. They have been supported by around £3.5 million in Scottish Government funding over two phases, up to March 2024. The EACs have recently received multi-year funding agreements, committing £8 million per year – £16 million in total – to support delivery until at least the end of this parliamentary term, in March 2026.

The Scottish Government commissioned Ipsos Scotland to carry out work with the EACs to inform its approach to longer-term monitoring and evaluation. This work comprised four stages:

• developing theories of change at both the community and national level
• developing a monitoring and evaluation (M&E) framework
• conducting a process evaluation on early implementation and delivery, and
• producing a preliminary evaluability assessment to inform future evaluation.

This report presents the findings of the early process evaluation for the EACs. It aims to identify what has worked well or less well during early set-up and implementation, to inform ongoing design and delivery.

This early process evaluation was qualitative in nature, based predominantly on in-depth interviews in March and May 2024. Participants included each EAC project lead, 17 EAC stakeholders, and 20 family interviews covering the experiences of 22 parents and 27 children (between the ages of five and 11) who had attended an EAC. Discussion guides were developed for each audience, shaped by the theories of change and M&E framework. The evaluation team also reviewed recent EAC monitoring reports.

Overview of EAC activities

Broadly speaking, the implementation of EACs has happened as intended. All EACs have met their commitment to offer school age childcare throughout the 2023/24 financial year, broadly in line with the activities specified in their original proposals to Scottish Government. As well as providing services for families, EACs were also expected to monitor progress, capture learning, and share this with the Scottish Government. This was achieved through EAC attendance at progress meetings, networking sessions and quarterly written reports.

All EACs were situated in communities with high levels of deprivation, and targeted families in the priority groups at higher risk of experiencing child poverty identified by the Scottish Government. However, specific delivery approaches were shaped by local circumstances. Some communities had large scale existing initiatives or a greater number of school age childcare services to build on.

The childcare offer across EACs varied. Some areas had expanded existing services while others created new ones, and a mix of registered and unregistered childcare or activity providers were used. Each EAC offered funded places at breakfast, after school, and holiday clubs (although the individual offer to families varied). In some areas there were specialist services for children with additional support needs (ASN). Each EAC involved a wide range of local partners.

While three EACs provided fully funded provision for families, Glasgow took a tapered approach based on household income, in order to reach a greater number of families.

Family support provision varied, with some EACs more involved in targeting and supporting families with greater challenges in their lives. Three of the communities had family support workers to help reach families.

Targeting and Reach

Families from the intended priority groups had attended EAC services, but challenges remained in reaching certain groups in some areas, such as minority ethnic families. Further outreach was needed to engage these families.

Attendance rates had generally been in line with EACs' expectations. However, there were concerns that some families do not use all their allocated days when they are fully funded. Where EACs monitored attendance and engaged with families, this helped to ensure that families had access to the right amount of provision to meet their needs.

There were a range of promotion and communication strategies adopted, including a mix of targeted and universal approaches. Direct communication from trusted individuals or organisations, such as schools, was considered particularly effective.

Application and registration processes were generally felt to be accessible, with EACs making efforts to simplify forms and provide support to families to attend. Parents valued the opportunity to visit childcare services with their children before or during registration to ensure a good fit. However, there were indications that some families are reluctant to share details on their child's additional support needs.

Meeting the needs of families

Delivery of services to families was broadly happening as planned. EACs had progressed each activity in the local theory of change, including co-designing provision, engaging families, and delivering funded childcare (including food) and family support. Overall, families interviewed were very positive about their experiences, indicating that EAC processes were generally working well to meet their needs.

EACs had all implemented some co-design activities with families, however the extent of this varied. In particular, there was less co-design among established partner providers. Co-design activities ranged from consulting families on their needs, to more participatory activities such as parent focus groups and child wish lists. Staff generally felt co-design had enhanced provision and gave examples of this. However, there was little recollection of co-design among parents interviewed. There were mixed views on the extent to which parents said they would have liked to have been involved.

Timings and locations were generally working well for families interviewed. Covering working hours was a key factor in this, and flexibility was appreciated by those with changing shift patterns. Local provision was also something that supported families to use EAC services.

There was evidence that the activities at EAC childcare met children's needs, with children interviewed saying that they enjoyed the activities offered and being able to spend time with their friends. The importance of giving children a choice and providing a diverse offer in order to meet different needs was a strong theme in staff interviews, although catering for the needs of different age groups had been a challenge.

EAC funding had been used to remove or reduce childcare costs for parents, which was an important factor in enabling families (who took part in this study) to attend. However, cost was a barrier to some families, who had partially-funded places, using more days.

Friendly and knowledgeable staff were very important to parents using EAC services. However, staff recruitment and retention was a significant challenge for providers, particularly for roles supporting children with ASN.

From the perspective of families with children with ASN who took part in the evaluation, needs were being met. Partnership working with specialist ASN providers and access to training had helped mitigate some challenges to accommodating children with ASN.

EACs were providing differing levels of family support. While some providers signposted to this, others provided it directly. Having dedicated staff (e.g. family support workers) was viewed positively. However, in some areas, take up of family support had not been as high as anticipated. Staff were mindful of managing families' expectations of the support they could offer.

Implementation of systems change

Scoping pre-existing childcare services and getting local stakeholders on board were important aspects of the design stage. While these had happened, staff noted that it can take time to embed change and to build trust with stakeholders.

EAC governance and partnership working appeared to be working well, especially having project leads working at a strategic level and linking with other child poverty initiatives. EACs had formed a range of partnerships at both the strategic and operational levels, including with schools, childcare providers, various local authority teams, welfare advice, early years, colleges and the training sector.

Project leads and stakeholders stressed the need for multi-year funding to retain staff and plan services effectively.

Although the focus of this report is on processes rather than impacts, it also notes early indications of some systems-level outcomes being achieved. Enablers to this included Scottish Government support; governance structures that facilitate close relationships; involving a wide range of partners; committed staff with a common goal; and regular contact between the EACs and partners.

There was agreement among staff and stakeholders that, in principle, EAC services could be adopted in other areas. Perceived enablers included support from the local authority; applying learning from the current EACs; existing stakeholder relationships and integrating provision into local areas.

However, sustainability of the EACs was thought to rely on continued Scottish Government funding. Various barriers were also mentioned, particularly ongoing workforce challenges, but also the capacity of existing provision, Care Inspectorate staffing ratios and obtaining appropriate physical spaces.

Conclusions

Delivery of services for families was generally happening as intended, shaped by co-design activities. Broadly speaking, the provision met the needs of families who took part in this research and was considered accessible, affordable and good quality by them. This was facilitated by: funded places; local venues; timing that suited parents' working patterns and staff who parents trusted to meet their children's needs. Staff cited trusting relationships as key to engaging families. At the same time, there is scope for extending reach to certain groups in some areas. For example, staff mentioned parents who were not working and minority ethnic families.

EACs have also been working as intended at the systems level. Specifically, EACs had worked with partners to achieve buy-in and further develop partnerships. Being in areas with existing child poverty initiatives meant EACs were not starting from scratch and could work with partners towards a common goal. Governance arrangements, as well as the commitment from staff across EACs and partners, had ensured that EACs were well aligned with local initiatives and able to enact systems change. This had been particularly effective in Dundee, where there are several initiatives working closely in a very small local area.