Access to postgraduate study - representation and destinations: discussion paper

Annex A – Confidence intervals and modelling

The error bars shown in this paper have been calculated based on approximate 95% confidence intervals. As the Destination of Leavers from Higher Education (DLHE) is a survey, a sample of the overall leaver population is collected. This means that the percentage of those progressing to postgraduate study and the percentage of those obtaining a professional role are estimates of the corresponding figures for the whole cohorts of leavers. Therefore, there is an element of uncertainty around these estimated figures, and confidence intervals allow us to understand how certain we can be that the estimates are precise.

A 95% confidence interval provides the likely extent to which an estimate may differ from the figure for the whole cohort. If two intervals do not overlap, then we can say that the difference between the groups is statistically significant (at the 95% level). For example, among Post-92 postgraduate leavers from Q1 and Q5, the confidence intervals for the percentage obtaining a professional level job do not overlap. This means we can be reasonably confident that the differences we see in the sampled respondents reflect a difference in the whole cohort of leavers. In general, where the intervals do overlap, the difference is not significant. Differences throughout have therefore generally only been commented on where the intervals clearly do not overlap. Where the sample size is small e.g. among Q1 students in Ancient universities, the interval is generally wider as we have less data to determine whether the individuals are representative of the population as a whole. This means that we are less confident in the estimate of the true figure for such groups.

The confidence interval method adopted for both progression and destinations is a Normal approximation to the Binomial distribution. For progression, the "success" parameter was taken to be those with HESA XACTIV02 variable equal to 04, 05 or 06 i.e. those in full-time work, part-time work or primarily studying and also in work. For destinations, the "success" parameter was taken to be those with a SOC 1-3 occupation. The analysis in this paper has not been weighted for non-response bias or design effect, and therefore the confidence intervals calculated do not incorporate such weighting. Further analysis (outwith the scope of this paper) was performed using logistic regression modelling. This modelling informed conclusions and determined whether SIMD was statistically significant in both progression and postgraduate destinations when controlling for other factors.