Early learning and childcare - parents' views and use: survey findings 2022

Appendix A: Methodology

Aims

The 2022 ELC Parent Survey was designed to explore views on early learning and childcare (ELC) services of parents and other primary carers of children who have not yet started school. It was designed to provide up-to-date information on use and barriers to use, experience, perceptions, engagement with ELC and outcomes. Information about socio-economic characteristics, family and household circumstances, characteristics of childcare use was collected. The data collection was structured to facilitate, as far as practical, comparison with the 2018 ELC parents survey.

Overview of approach

The study sought the views of parents with children aged under six years who were not yet in school (referred to as 'pre-school children' in this report). A number of possible options for sampling this population were considered but our scoping concluded none were possible, or possible within the timeframe or budget, of the study.^[32] However, information from the Scottish Household Survey indicated that 79% of households with a child aged two to five years used some form of childcare in 2019 rising to 88% for those with a three year old^[33]. More recent data, on registrations for funded ELC among children aged three and four year olds indicate that around 97% are registered^[34].

It was concluded that accessing parents of pre-school children via childcare settings was a reasonable proposition. It was important, however, to ensure that as broad a mix of parents was contacted, so parents were also contacted via a range of intermediaries and using Scottish Household re-contact data as follows:

• Online survey with parents who have pre-school children. The survey link was primarily promoted to parents through ELC providers. Further promotion of the survey was undertaken by local authority early learning leads and other key stakeholders with a focus on reaching parents who were not using ELC or using childminders.
• Telephone interviews with parents who have pre-school children drawn from the re-contacts database of the Scottish Household Survey^[35].

The main fieldwork took place between 25 April and 29 May 2022

Sampling

The target group for the research was parents and other primary carers (referred to throughout simply as 'parents') with children aged under six who were not yet in school in Scotland. The objective was to cover as wide a mix of parents as possible. In particular, this was to include parents whose children aged two years and children 3 to 5 years who were eligible for funded ELC and who were using, at least some of, the 1,140 hours; but also to include parents of eligible children not using their funded hours, as well parents of younger children not yet entitled to funded ELC.

The sample frame used for the study was Care Inspectorate Datastore^[36]. This contained information on all registered care services operating in Scotland. The study drew on information relating to children's daycare service which included email addresses. The database information covered childcare settings in the public, voluntary and private sector, but does not include contact information for childminders.

Settings across Scotland were asked to share the survey link with the parents who use their service. It was appreciated the resulting sample would be biased towards those using ELC (funded and non-funded) and would not collect information from parents who had no children in ELC. It would also not include those who only used childminders. Distribution was therefore widened to include a range of stakeholders, including local authority early learning leads, Care and Learning Alliance, Scottish Childminding Alliance, National Parent Forum, Enquire, and minority ethnic organisations.

A target sample size was not established. The strategy was to circulate the invitation to all settings and stakeholder contacts and invite all parents to participate. This broadly replicated the approach adopted in 2018. It was noted that in 2018, a sample of 10,526 was achieved (an estimated response of around 11%).

It was assumed that the response for the 2022 survey would be somewhat lower: online survey response rates been generally declining over recent years; parents in particular have a great deal on their minds in the current climate. It is also possible that parents would have been especially motivated to respond to the 2018 survey as they were being given the opportunity to shape the new ELC offer, and there was some discontent with the level of provision at the time.

Questionnaire design

The survey questionnaire was designed in three stages: an initial draft, design workshop, final drafts.

Initial draft questionnaire: Progressive developed a draft questionnaire drawing on the 2018 survey, relevant Scottish Household Survey questions and the parent survey component of the Scottish Study of ELC for the review by the client team. The client team determined which elements of the questionnaire would benefit from further consideration at the questionnaire design workshop.

Questionnaire design workshop: A half-day session, comprising Progressive, Scottish Government and stakeholders, reviewed the overall approach and some key elements of the draft questionnaire. The client team provided in-depth feedback on the draft.

Final draft questionnaire: following the workshop, further work was undertaken to revise and refine the final questionnaires in consultation with the Scottish Government client team.

The questionnaire contained sixty questions, sixteen of which were classification questions which collected information on the respondent's gender, age, working status and socio-economic group, income, where they live, and so on. The general survey questions were mainly closed, using a mix of categorical and interval questions. The multiple choice questions typically included an 'other, please specify' option giving the respondent the opportunity to add their views/experience as required. A small number of open questions were also included. These explored how parents' use of ELC had been impacted by Covid-19 since August 2021 and their overall views on ELC impacts.

Once approved by the client team, two 'versions' of the questionnaire were produced:

• the omnibus: a short (eight questions) which covers usage, type of use by age of child, and reasons for non-use
• the core (online/telephone survey) which contained the full question set

A copy of the questionnaire can be found as a supporting document to this publication.

Data collection

Data were gathered using three methods: an omnibus survey, a full online survey and a telephone survey.

Omnibus survey of parents with children in Scotland. The overall omnibus sample is a targeted quota sample from YouGov's opt-in panel in Scotland. Each wave contains at least 1,000 responses. It was estimated that around 10%-of these responses to be from parents with children under six years. The omnibus was therefore run over two waves, to achieve a reasonable of parents of pre-school children.

The focus of the omnibus was subsequently refined to concentrate on parents of children aged three to five years, with only one question asked of all parents of pre-school age children. As a consequence, the number of valid interviews achieved across the omnibus was considerably lower than anticipated: 162 parents with children aged under six years; 104 with a child/children aged three to five years, and just 82 a child/children aged three to five years in funded ELC.

These samples were not considered sufficient to influence the development of the main survey. The data from the omnibus has however been reviewed and reported as tables within the Data Appendix. It is confirmed that the findings are broadly consistent with those from the main report.

The omnibus fieldwork took place between 23^rd and 30^th March 2022.

Telephone interviews with parents who have pre-school children drawn from the re-contacts database of the Scottish Household Survey (SHS). These respondents had previously participated in the SHS and had given permission to be re-contacted to take part in further research. Scottish Government analysists reviewed the overall sample set and only provided contacts for those who are eligible to participate in this survey – that is people who are parents of children aged under six years.

In total 271 contacts with a usable phone number were provided by Scottish Government and 78 successful telephone interviews were achieved. While it was appreciated this would be a small number of completed interviews, this was considered a useful opportunity to engage with a broad mix of parents including, potentially, those not using paid-for/funded ELC.

Online survey with parents who have pre-school children. As detailed above, the survey link was primarily promoted to parents through ELC providers. Contacts for suppliers (excluding childminders) were sourced from Care Inspectorate Datastore. This contained some 3,198 contacts with email addresses.

Progressive contacted all the suppliers on the sample frame and requested they disseminate the survey link to parents who use their ELC services, for example via email or through the social media groups (e.g. through their parents' twitter group or WhatsApp group). We provided text that could be forwarded to parents, including the survey link. The purpose of this email/text was to:

• introduce Progressive as an independent, impartial agency
• explain the purposes of the research
• reassure respondents of their anonymity and confidentiality of the survey
• outline the value of their participation
• notify respondents of the fieldwork period is
• include the survey link which was a shortened URL – such as Scottish Government Parent Survey – which was more welcoming than a standard URL
• send thanks from Scottish Government for taking part

The Scottish Government also provided a list of key stakeholders (early learning leads, parents organisations, equalities organisations), to enable the survey link to be circulated to a broad range of parents' organisations.

The survey response was monitored on a daily basis to assess the overall response and the profile of the response. A number of measures were taken to promote a strong response to the survey. These included:

• Scottish Government used their social media to promote the survey among stakeholders and more widely
• reminders were issued during the fieldwork period to prompt providers to share the survey link with parents
• local authority early learning leads were encouraged by Scottish Government to promote the survey (this work had to wait until after the local elections)
• early learning leads in authorities with disproportionately low response rates were contacted by Progressive or Scottish Government and asked to make special efforts to contact providers in their area
• the survey deadline was extended by 10 days to allow for additional responses

The main fieldwork took place between 25^th April and 29^th May 2022.

Response

The main survey generated 8,224 responses: 43 of these responses were excluded from the analysis as, on inspection, they had duplicate emails and/or telephone numbers. This resulted in 8,181 valid responses: 78 telephone responses and 8,103 online responses. This was considered a strong response to the survey, broadly in line with the 2018 response and sufficient to provide a clear indication of the use and views of parents of young children in Scotland.

We do note however that respondents to the online survey were self-selecting. We cannot therefore provide statistically precise margins of error or significance testing as the sampling type is non-probability. The margins of error outlined below should therefore be treated as indicative, based on an equivalent probability sample. The survey dataset has a margin of error ±1.1%, calculated at the 95% confidence level (market research industry standard). This means that if 50% said they were satisfied with their provider, we can be 95% confident that the result lies between 48.9% and 51.1%.

Data processing and analysis

Data analysis has been conducted using SNAP. All analysis uses weighted data (see below), except where discussing the characteristics of the sample. Tables that use unweighted data are clearly marked Tests for statistical significance have been conducted using z and t tests, and all differences discussed within the text statistically significant, calculated at the 95% confidence level^[37].

Weighting

Survey data is commonly weighted to help make the achieved sample representative of the population it was drawn from, and to help produce unbiased survey estimates.

The ELC survey produced a differential response across the country, with a proportionately lower responses from large urban areas. This can be seen on Table MA1 below, which compares the profile of children receiving funded ELC (as a proxy for the profile of families of young children) from the ELC Census against the profile of the sample achieved. The response from large urban areas Aberdeen City, Edinburgh, Fife and South Lanarkshire were lower than anticipated, even following mitigations in the field.

The sample was therefore weighted. Weights were calculated to adjust for the population totals of number of children by urban rural 6-fold classification. This has had the effect of adjusting the totals for urban local authorities upwards, and the totals for the more rural areas downwards.

Analysis workshop

A client group workshop was held to review the analysis approach. This session agreed a set of themes for the analysis, which formed the broad structure for the report.

Missing data

Input data Some work was undertaken to input postcode data solely for use with the Urban Rural variable. It was noted that of the total 8181 cases, some 626 (7.6%) cases contained either missing or insufficient information to list a postcode for analysis purposes. The following approach was adopted.

• Where parent postcode was missing, the postcode of provider was input, as a proxy as it was considered a reasonable assumption the parent address and provider address were likely to lie within the same urban rural classification.
• Where the parent postcode was missing and no provider postcode was available – for example, no postcode was attached to the provider sample or the respondent replied to the survey using a non-provider link (such as Early Years or a stakeholder), the postcode remained blank.

This approach reduced the number of missing/unrecognisable postcodes by 277 to 352 (4.3%).

It is stressed that the inputted postcode data were ONLY used to populate the Urban Rural variable. They were NOT used for other variables built from postcodes (i.e. SIMD).

Comparisons with 2018 Survey

The 2022 questionnaire was based on the 2018 questionnaire. The 2022 questionnaire was substantially revised to reflect the changes in the funded ELC provisions, current policy interests, and to reflect changes in the use of ELC over recent years, including the impact of the pandemic.

A light touch approach to comparison with the 2018 survey was therefore adopted in the report, with comparisons drawn out as appropriate across the report.

Age cohorts One point to note on the age groups used in the 2018 and 2022 report. Typically, the 2018 report refers to children aged 3-4 years as the core age group eligible for funded ELC. The 2022 refers to children aged three to five years. This reflects the time of year when each survey took place. The fieldwork for the 2018 report was undertaken in August-September 2017, at the start of the school year when eligible children were aged 3 to 4 years. The fieldwork for the 2022 survey took place in April-May 2022 at the end of the school year, extending the relevant age range to three to five years.