Early learning and childcare expansion programme: evaluation strategy

3. Monitoring progress on outputs

3.1. Delivery assurance

3.1.1. To monitor progress and provide assurance on the delivery of the expansion programme, the Improvement Service and Scottish Futures Trust (SFT) have been collecting and analysing data on progress of delivery across local authorities on behalf of Scottish Government.^[38] The first data was collected for the period May to September 2018.^[39] Up to the lockdown of the sector in March 2020, the Improvement Service was collecting data at the beginning and end of each academic year, while the SFT collected infrastructure data quarterly.

3.1.2. The Improvement Service is now reporting on this data for every academic term from August 2020 until April 2022.^[40] The indicators cover the outputs in Figure 1 including phasing, workforce, capacity and infrastructure:

• number of two year olds and three to five year olds accessing funded ELC, and the range of hours being accessed (600 hours or fewer; >600 hours to <900 hours; ≥900 hours to <1140 hours; full 1140 hours)
• number of two to five year olds accessing funded ELC in local authority, funded provider and childminder settings (split by age: two year olds, three and four year olds, and five year olds e.g. children whose school entry is deferred)
• base operating model and Care Inspectorate-registered physical capacity in local authority settings
• local authority staff in place, split by job type (managers, practitioners, care workers etc.)
• number of refurbishment, extension and new build projects 'complete', 'in construction' or 'in development' (collected by SFT)

3.2. Understanding changes in the ELC sector and workforce

3.2.1. The logic model underlines that the delivery of 1140 hours is dependent on having sufficient workforce in place. A skilled ELC workforce is fundamental to achieving the intermediate outcomes – in particular providing high quality ELC – and subsequently achieving the policy's aims. As noted above, the delivery assurance data collected by the Improvement Service has allowed the monitoring of capacity in terms of physical capacity and local authority staff in place. It is also important to understand how changes related to the ELC expansion itself, as well as to the ongoing impacts of the pandemic, are affecting the sector and workforce.

3.2.2. In terms of changes in the ELC sector and workforce, key aspects include:

• changes in the number and composition of services in general and those providing funded ELC places, including public, private, third sector and childminding services
• the sustainability of services
• staff demographics, staff qualifications, and turnover
• wider movement within the social care workforce
• pay and working conditions, including payment of the Real Living Wage and implementation of Fair Work practices and
• views, experiences and wellbeing of staff

3.2.3. A range of data sources are used by the ELC Directorate, working with stakeholders on an ongoing basis, to explore changes in the composition and capacity of the ELC sector and workforce, beyond the local authority workforce data collected by the Improvement Service:

• The Care Inspectorate collect information on ELC services, including the number of daycare of children services^[41] and of childminders registered, the number of services providing funded places and children registered with services.^[42]
• Scottish Social Services Council (SSSC) data provides information on the ELC workforce, including age, gender, qualifications against registration requirements and hours worked.^[43] This data can be broken down by, for example, employer type (public, private, voluntary) and job function. While the key focus is on ELC services,^[44] when considering this data it is also useful to compare ELC Services with the broader children's services and the social services workforce to identify trends.^[45]
• The ELC Census also collects workforce data on teachers, graduate staff and staff working towards graduate qualifications.^[46]
• The Scottish Childminding Association (SCMA) have been collecting information on childminding services delivering funded ELC since the start of the expansion to 600 hours.^[47] Information is gathered from local authorities on use of childminding services. It has provided an important data source on delivery of funded ELC by childminders. In addition, in the December 2020 annual returns, the Care Inspectorate incorporated new questions for childminding services about provision of funded ELC places and becoming a funded service. The new ELC Census will also provide data on children registered for funded hours with childminders (see 4.6 for more detail on the new ELC Census).
• The Financial Sustainability Health Check, published in August 2021, provided a range of detailed information on the whole childcare sector, including factors related to the move, for some services, to delivering 1140 hours.^[48] We will carry out an update of this exercise in the second half of 2022 and are looking to learn lessons to inform our ongoing evidence base on the sustainability of the sector.

3.2.4. Gaps identified in the data and analysis available include: data and analysis to better understand the movement within the daycare of children workforce (e.g. between private, third sector and local authority settings) and the wider social care workforce; movement of childminders into other parts of the childcare sector, or who leave the sector completely and why/where they go; regularly collected data on the views, experiences and wellbeing of staff (in both local authority and private and third sector settings). Work to prioritise and fill these data gaps is being taken forward by the ELC census development advisory group and as part of the development of a Childcare Workforce Strategy.

3.2.5. A programme of work has been developed to strengthen the evidence base on the sustainability of childcare providers in the private, third and childminding sector. Key elements include the Financial Sustainability Health Check, published in August 2021, and the annual collection of information on the sustainable rates set by local authorities for the delivery of funded ELC.^[49] These larger scale evidence collection exercises will be supported by regular monitoring of service registration data from the Care Inspectorate and ongoing engagement (and intelligence gathering) with providers and the sector representative bodies. This work also links closely to the economic evaluation of ELC expansion – full details of which are set out in section 6.

3.2.6. In partnership with SCMA and Care Inspectorate, Scottish Government commissioned research to explore the range of factors that may be contributing to the decline in the Scottish childminder workforce in order to identify ways to better recruit, support and retain them. The research, undertaken by Ipsos, involved a Rapid Evidence Assessment to assess existing data and trends on the childminding workforce and primary qualitative research to understand the views and experiences of current, former and prospective childminders. A report was published in April 2022^[50] and findings will be incorporated into the evaluation.

3.2.7. The ELC Census Advisory Group includes representatives from SSSC and the Care Inspectorate, alongside other stakeholders. Together the group is trying to meet the data requirements of the sector, often through considering data already collected such as the Care Inspectorate annual returns, and how this data can be further utilised. This group also advises on the changes to the ELC Census.

3.2.8. In the Financial Sustainability Health Check Survey report (see above), Scottish Government committed to the development of a new Childcare Workforce Strategy. This was in recognition of the current and forthcoming challenges facing the sector around capacity, recruitment and retention, in the context of the implementation of the ELC expansion; recovery from the pandemic; and scoping work for the delivery of the new Programme for Government commitments around wraparound childcare and extending early learning to one and two year olds. As part of the work on developing the workforce strategy, Scottish Government are working with SSSC to ensure as complete a picture of workforce information as possible and to consider what further information can be gleaned from existing sources, such as registration data.