Scottish Study of Early Learning and Childcare: Three-year-olds (Phase 3) Report - Updated 2021

Appendix D: Regression analysis

Tables D1 to D6 show the results of logistic regression analysis of whether a child has delayed development on at least two domains of the Ages and Stages Questionnaire and of raised / high score on the Strengths and Difficulties Questionnaire total difficulties scale at Phase 3. 

Logistic regression analysis is a method of summarising the relationship between a binary 'outcome' variable and one or more 'predictor' variables. It allows us to estimate the odds of a child having a score of '1' on the outcome variable (as opposed to '0') from knowledge of their scores on the predictor variables. In the model shown in Table D1 the score of '1' on the dependent variable refers to exhibiting delayed development on two or more of the ASQ domains, while a '0' refers to exhibiting no delayed development, or delayed development on just one of the domains. 

Logistic regression allows us to consider multiple relationships at the same time and to identify those relationships between a predictor variable and the outcome variable which remain statistically significant even when we take into account other predictor variables. For those variables that do remain significant we can say that they show an independent association with the outcome variable while controlling all other factors in the model.

The first two regression models are longitudinal models. All of the predictor variables included are from Phase 1, while the outcome variables are from Phase 3. This introduces a time element allowing us to say for certain that all of the predictor variables predate the outcome – although this does not imply causality. The other four models are cross-sectional, taking all the data from Phase 3.

Tables D1 to D6 show how the odds for each category of each predictor variable compare with the odds for the reference category. An odds ratio of greater than 1 indicates that, holding all other factors constant, there is an increased likelihood of a child in that category being in the category '1' for the outcome variable compared with a child in the base category. For example, in Table D1, the odds ratio of 1.7 for the category 'Female' means that girls are more likely than boys (the base category) to exhibit development that is on schedule for at least four of the five ASQ domains (and the odds of a girl exhibiting such development are 1.7 times those for a boy, holding all other factors constant). Conversely, an odds ratio of below 1 means they have lower odds of exhibiting delayed development than respondents in the reference category.

Because data are taken from a sample, we recognise that the odds ratios are only estimates, so we also include confidence intervals around each estimate. If the survey were to be repeated, we would expect the true value to fall within these odds ratios 95 times out of 100.

Two measures of statistical significance are provided. The first is for the comparison between a particular category and the base category, while the second is for the variable as a whole. Where the independent variable has just two categories, these are the same. A significance level of 0.05 or less indicates that there is less than a 5% chance we would have found these differences between the categories just by chance if in fact no such difference exists, hence we can say that we are 95% sure there is a relationship between the predictor and outcome variables. A level of <0.001 indicates that there is a less than 0.1% chance, so we can say that we are 99.9% sure that the relationship exists. For the purposes of Tables 4 and 5, we described a level of significance of less than 0.01 as "highly significant", of between 0.01 and 0.05 as "moderately significant, and of between 0.05 and 0.10 as "marginally significant".

The Nagelkerke R-square value provided at the bottom of each model is a rough indication of the proportion of variation in the outcome variable explained by the predictor variables in the model. In the first two models this is between 0.2 and 0.25, which is fairly typical for this type of analysis, while in the subsequent models this is lower still. This means that there is a lot of variation in the data which is not explained by the variables (and nor would we expect it to be).

All models have been tested for stability through the systematic removal of variables to check for changes in odds ratios and significance of other variables, and checks on the covariation of independent variables, and all were found to be stable. Because of the small sample size for all the models, but particularly the longitudinal ones, it was not possible to include a large number of predictor variables. Instead a number of key variables were forced into each of the models. Other variables were then systematically tested to check for a significant association with the outcome variable when controlling other factors. Only tested variables that were significant at the 10% (0.10) level were included in the final models as presented in Tables C1 to C6. The variables which were forced into the models are included in the Tables irrespective of their level of significance. Forced variables for the longitudinal models were the Phase 1 score (ASQ 4+ domains on schedule or SDQ total difficulties on schedule, to match the outcome), the ITERS rating from the setting observation data, sex of the child and area deprivation of the home address. Forced variables for the cross-sectional models were sex of the child, area deprivation of the home address and whether the child has a long-term condition which may affect their development, as reported by either the keyworker or the parent. A list of all the variables considered for inclusion in the regression models is given below.