Scottish Household Survey: childcare topic report

Annex A

Statistical Significance 

Because the survey's estimates may be affected by sampling errors, apparent differences of a few percentage points between sub-samples may not reflect real differences in the population. It might be that the true values in the population are similar but the random selection of households for the survey has, by chance, produced a sample which gives a high estimate for one sub-sample and a low estimate for the other.

A difference between two areas is significant if it is so large that a difference of that size (or greater) is unlikely to have occurred purely by chance. Conventionally, significance is tested at the five per cent level, which means that a difference is considered significant if it would only have occurred once in 20 different samples. Testing significance involves comparing the difference between the two samples with the 95 per cent confidence limits for each of the two estimates.

For example, suppose the survey estimates that there are 14 per cent 'single adult households' in Stirling (±4.1 per cent), 10 per cent in Aberdeenshire (±1.7 per cent), 15 per cent in Fife (±2.0 per cent), and 24 per cent in Edinburgh (±2.5 per cent). Assuming that the estimates' values are 'exact' (i.e. that the figure underlying 10 per cent is 10.0 per cent), we can say the following: 

• The difference between Stirling and Fife is not significant because the difference between the two (one per cent) is smaller than either of the confidence limits (at least ±2.0 per cent). In general, if the difference is smaller than the larger of the two limits, it could have occurred by chance and is not significant;
• The difference between Stirling and Edinburgh is significant because the difference (10 per cent) is greater than the sum of the two confidence limits (4.1 + 2.5 = 6.6 per cent). In general, a difference that is greater than the sum of the limits is significant. 

If the difference is greater than the larger of the two confidence limits, but less than the sum of the two limits, the difference might be significant, although the test is more complex. 

Statistical sampling theory suggests that the absolute value of the difference between the two estimates |𝑝1−𝑝2 | is significant if it is greater than the square root of the sum of the squares of the limits for the two estimates, as explained by the following formula: 

|𝑝1−𝑝2 | > √[(𝐶𝐼1) 2 + (𝐶𝐼2) 2]

The difference of five per cent between Aberdeenshire and Fife is greater than the largest confidence limit (±4.1 per cent) but it is less than the sum of the two limits (4.1 per cent + 2.0 per cent = 6.1 per cent) so it might be significant. In this case 4.12 = 16.81 and 2.02 = 4 giving a total sum of 20.81. The square root of this is 4.56, lower than the sum of the two limits (6.1 per cent), which means that the difference of five per cent is significant (although only just). Similar calculations will indicate whether other pairs of estimates differ significantly. 

It should be noted that the estimates published in this report have been rounded, generally to the nearest whole number, and this can affect the apparent significance of some of the results. For example: 

• If the estimate for Aberdeenshire was 10.49 per cent (rounded to 10 per cent) and the estimate for the Fife was 14.51 per cent (rounded to 15 per cent), the difference would be calculated as 4.02 per cent rather than five per cent. This is below the calculated 'significance threshold' value of 4.56 per cent; 
• If, however, the estimate for the Lothian's was 10.51 per cent (rounded to 11 per cent) and the estimate for Fife was 15.49 per cent (rounded to 15 per cent) the difference would be calculated as 4.98 per cent rather than five per cent. This is higher than 4.56 per cent. 

For this reason, caution should be exercised where differences are on the margins of significance. In general, we would suggest that differences should only be considered significant where the difference is clearly beyond the threshold of significance.

All comparisons made within the text of this report have been tested for statistical significance.

Statutory Entitlement

The statutory entitlement for three-year-olds and two-year-olds (who qualify for the earlier offer) commences from the start of the first term after the child's 3^rd or 2^nd birthday, respectively. Education authorities also have discretionary power under section 1(1C) of the 1980 Act to provide additional ELC to any child. A number of education authorities therefore secure earlier commencement dates, including: from a child's 3^rd birthday; from the first term after their 3^rd birthday; or certain children before they are three-years-old. More information is available at: https://www.mygov.scot/childcare-costs-help/when-funded-early-learning-and-childcare-can-start/

Entitlement for Two-Year-Olds

Two-year-olds are entitled to statutory funded ELC if they meet various critiera as set out in the Children and Young People Act 2014 and the Provision of Early Learning and Childcare (Specified Children) (Scotland) Order 2014 (SSI 2014/196). Some local authorities provide discretionary funding for some two-year-olds who do not qualify for the statutory entitlement. More information is available at: https://www.mygov.scot/childcare-costs-help/funded-early-learning-and-childcare/