Scottish Study of Early Learning and Childcare: ELC Leavers (Phase 2) Report - Updated 2021

Characteristics of ELC

To gather information on the characteristics of ELC settings, inspectors from the Care Inspectorate (acting as observers independent of their regulatory roles) conducted observations of 150 settings using the most recent version of the Early Childhood Environment Rating Scale (ECERS-3). The ECERS was developed in the United States by the Environment Rating Scale Institute along with the Infant / Toddler Environment Rating Scale (ITERS). Both are widely used in English speaking countries. In the United Kingdom, ECERS has been used in both the Effective Provision of Pre-School Education (EPPE) study and in the more recent Study of Early Education and Development (SEED) in England ^[32]. In Scotland, ECERS was used as long ago as 1994^[33] and has seen many applications since that time.

Both environment rating scales have a positive international reputation as a way of assessing the quality of provision in a 'snap-shot' observation and as a tool which gives researchers access to the everyday experiences of children in their educational settings. The scales have high reliability at indicator and item level when used by trained observers^[34]. Validity is also high in terms of their relationship to other ways of assessing quality and to measures of children's outcomes. Further, in conjunction with academics and the Care Inspectorate, some minor amendments were made to ensure that the ECERS-3 was reflective of the aspects of quality that are expected in Scotland (e.g. that rainfall should not prevent outdoor play).

ECERS-3 was used for a number of reasons: it centres on the experience of the child in the setting; it allows for the effect of setting quality on child outcomes to be controlled for; and it is relatively easy to administer given that only one three hour observation is required. This tool can also be used to see if particular characteristics of settings contribute to differential outcomes in children. Further, ECERS-3 is designed for use in settings where most children are aged between three and five and and as such, it was deemed suitable for use with the ELC leavers involved in Phase 2 of the SSELC.

It is important to note that these tools are not the only method of assessing setting quality in Scotland. Indeed, the Care Inspectorate ratings provide a broader measure of the quality of practice and policy within settings that have also been found to be related to children's outcomes in Scotland.

As with the Care Inspectorate inspection approach, the setting observations focussed on outcomes. However, the methodology differed in that the ECERS-3 tool was used to observe for three hours, with no consultation with setting staff and no professional dialogue or explicit feedback provided. This was because the observations were to be a snapshot to inform the SSELC and control for variation in child outcome data, rather than serving as an assessment of an individual setting's quality. During the ECERS-3 observations, observers looked at the six domains specifically for four- and five-year-olds. In contrast, during a formal inspection, Care Inspectorate inspectors consider a range of areas that impact on experiences for all children attending the setting, not only the four- and five-year-olds. The key areas covered during a formal inspection are likely to include some or all of the domain areas but can also cover other aspects of the provision to evaluate the overall quality of the setting.

The ECERS-3 scale comprises 35 items across 6 different subscales: space and furnishings; personal care routines; language and literacy; learning activities; interaction and programme structure.

• Space and furnishings includes observation of: indoor space; furnishings for care, play, and learning; room arrangement; space for play; space for privacy; display for children; and play equipment.
• Personal care routines includes observation of: meals and snacks; toileting; health practices; and safety practices.
• Language and literacy includes observation of: encouraging children to expand vocabulary and use language; encouraging children to communicate; staff use of books with children; and encouraging children's use of books and familiarity with print.
• Learning activities includes observation of: fine motor; art; music and movement; blocks; dramatic play; nature and science; maths materials, understanding of written numbers and the use of maths in daily events; appropriate use of technology; and promoting acceptance of diversity.
• Interaction includes observation of: supervision of gross motor play; individualised teaching and learning; peer interaction; staff-child interaction; and discipline.
• Programme structure includes observation of: transitions and waiting times; free play; and group play activities.

In line with ECERS-3 guidance, each subscale is scored from 1 to 7. These scores are calculated by averaging the score for each item within the subscale. Each of the 35 items are also scored from 1 to 7. These scores are calculated using the indicators contained within each individual item. Indicators are grouped under scores of 1 (inadequate), 3 (minimal), 5 (good), and 7 (excellent), with each indicator providing an example of what should be observed relevant to each score. Indicators themselves are scored as yes or no depending on whether the indicator has been observed. In some cases, observers are able to record indicators or items as not applicable; these are then excluded when calculating item or subscale scores. A score of 1 is given if any indicator grouped under 1 is scored yes. For an item to score a 7, each indicator grouped under 7 must be scored yes.

In addition to the main indicators, background data was collected during observations on the structure of the setting, including the number of children and staff present at the time of observation and whether there was freeflow^[35] access to outdoor space.

Of the 150 settings observed, nearly two-thirds (63%) had just one room for four- and five-year olds. A further 27% had two rooms, 9% had 3 rooms and 1% (two settings) had more than 3 rooms. The number of four- and five-year olds present on the day of observations ranged from 3 to 88. Children had access to outdoor space in all settings, and this was freeflow in two-thirds (65%) of cases.

Table 16 summarises scores on each of the ECERS-3 subscales. Settings scored highest on the interaction subscale, with 79% scoring 5 or above. The majority of settings also scored at this level on the personal care routines (65%), the space and furnishings (59%) and the programme structure (59%) subscales. It should be noted however, that an average score of 5 on items within a subscale still indicates room for improvement on multiple items within the scale. On two of the subscales (space and furnishings and learning sctivities) no settings scored the maximum 7, and of the other four, the highest proportion achieving the maximum, indicating 'excellent' on all items, was 10%.

* Settings' mean score for each subscale was categorised based on the highest score fully achieved e.g. if a setting scored 4.5 for the 'Space and Furnishings' subscale, they would be categorised as '4 < 5' rather than rounding up to 5. This decision was made in consultation with academic colleagues and the Care Inspectorate.

On the language and literacy subscale, only 29% of settings scored 5 or above, while overall performance was weakest on the learning activities subscale, with only 7% scoring 5 or above, and 33% scoring below 3.

Only top-level analysis of the ECERS-3 data has been conducted for this report. Future analysis will consider associations between setting characteristics and child outcomes.