Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS): Technical Report 2015

Response rates

Overall, 264 out of an invited 382 schools participated in the survey. This equated to 1,182 out of an invited 1,756 classes and 25,304 out of a possible 31,147 pupils who participated in the survey. All pupils who completed the survey were included in the analysis of results. However, due to differences in sampling, the schools completing the RCS boost were not included in assessment of the overall response rate, discussed in more detail below. The response rate is based on a total sample of 16,911 pupils.

In the sample included in the response rate, 235 of the 351 invited schools took part in the survey, giving a school response rate of 67%. The class response rate was 61%, 822 classes out of the original sample of 1,345. Overall 16,911 pupils completed the questionnaire, exceeding the target sample of 16,000. Based on the class response forms ^[2] sent out to participating schools, this equated to a pupil response rate of 87%.

Prior to 2002, the survey was conducted across the whole of the UK and not just in Scotland. While in previous years the response rate was calculated as the product of the school response rate and the pupil response rate, this changed in 2002 when the Scottish survey became separated from the English and Welsh survey. From this point on, the overall response rate was calculated as the product of the class response rate and the pupil response rate, with the exception of 2006. The overall response rate in 2015 was 53% (Table 1 and Figure 1).

Figure 1: Response rates for SALSUS and predecessors: 2002-2015

Source SALSUS 2002-2015

N.B. The response rate in 2006 was calculated in a different way than other years of SALSUS and is therefore incomparable. For this reason, it has been excluded from the response rate time chart.

Table 1: Response rates for SALSUS and predecessors: 1982-2015

Survey year	School response rate	Class response rate	Pupil response rate	Overall response rate
1982	96%	-	90%	84%
1984	89%	-	90%	81%
1986	82%	-	91%	78%
1990	97%	-	90%	87%
1992	96%	-	90%	86%
1994	92%	-	90%	82%
1996	81%	-	90%	71%
1998	81%	-	89%	70%
2000	72%	-	90%	64%
2002	75%	73%	89%	65%
2004	72%	71%	87%	62%
2006	69%	-	82%	57%
2008	69%	65%	91%	59%
2010	75%	68%	91%	62%
2013	71%	67%	90%	60%
2015	67%	61%	87%	53%

The overall response rate has dropped since 2013. This was due to a lower than expected response rate among schools completing the survey online. The response rate for the paper sample was broadly in line with that for 2013.

Non-response

Most surveys are subject to possible bias due to non-response. Within this survey there were several possible reasons for non-response to occur: school and class non-response; pupil non-response; and item (question) non-response. The impacts of non-response bias can be addressed through the use of weights which is discussed later in further detail.

School and Class Non-Response

The extent to which school non-response leads to bias in the survey results will depend on the extent to which this leads to a systematic under-representation of schools with particular features, where those features are linked with the variables the survey measures. For example, smoking prevalence can be higher at schools with a high proportion of pupils living in areas of greater deprivation.

The overall school response rate was 67%. Table 2 presents a comparison of the sample with pupil census information to allow assessment of the existence of non-response bias. This shows that the sample was representative in terms of school denomination and whether the school was independent or not.

However, there was some under-representation of S4 pupils (46% in the sample, compared with 51% of the population) and over-representation of 13 year olds (54% in the sample, compared with 49% of the population).

Pupils in the sample were also more likely to be in rural areas than the population profile (22% in the sample, compared with 18% of the population) which could be indicative of bias.

There did not seem to be any other obvious differences between the schools that participated and those that did not (e.g. size of school). However, it is not possible to examine or quantify all potential sources of non-response bias. For example, it may be that schools that place a higher priority on substance use education may be more likely to take part. Schools that place a higher priority on substance use education may do so because it is more of problem among their pupils. In this case the survey results may be biased by over-representing pupils who use substances. Alternatively, if the education is effective, the survey results may be biased by under-representing pupils who use substances.

Table 2: Comparison of sample profile with pupil census information

	2015 Unweighted Sample %	2015 Pupil Census %
Sex
Boys	50	50
Girls	50	50
Year
S2	54	49
S4	46	51
School type
State	94	94
Independent	6	6
Denomination
Catholic	16	18
Non-denominational	84	82
Area Type
Urban	78	82
Rural	22	18

Pupil Non-Response

Pupil non-response within classes resulted from illness on the day of the survey, other absence (this could be authorised or unauthorised) or refusal (either from the pupil or the parent). In order to maximise the response from pupils and to limit any bias, teachers were asked to administer questionnaires for absent pupils at a later date. This led to a very high pupil response rate of 87%.

Item Non-Response

Item non-response is where respondents do not answer some questions. If the item non-response is systematic in any way, i.e. if there is a reason why some groups of respondents are less likely than others to answer a particular question, there is the potential for bias in the results.

The level of item non-response in the survey was generally low. Of 90 questions in the survey, just seven had levels of non-response of 10% or more, see Table 3. Therefore, item non-response is unlikely to have greatly affected the results.

The level of item non-response was considerably higher in the online mode than in the paper mode and this is discussed in detail in the Mode Effect Study Results report ^[3] . However, the differing levels of item non-response did not have a statistically significant impact on the results of the key prevalence measures (and, consequently, has not had an impact on the trends).

By far the highest level of item non-response was at the postcode question. It was high in both modes but, again, higher in the online mode than the paper mode. It was clear from the qualitative research with pupils, conducted during the online pilot, that there was considerable concern that providing their postcode (particularly in combination with their month and year of birth) would make them identifiable. This may explain the higher non-response for postcodes for the online mode. However, it is unclear why this would be more of a concern for those completing the survey online than on paper. One possibility could be a sense that electronic data is more easily manipulated and matched up with other electronic data. Although not identified in the pilot, another possibility is that pupils may be aware of warnings (from school and elsewhere) to be very careful about the types of personal data they submit online.

Table 3 Item non-response where proportions were equal to or greater than 10%

Question	Subquestion	Base	Non-response
Q46 - How much do you think your father/carer really knows about…?	How you spend your money	16,911	10%
	Where you are after school	16,911	10%
	Where you go at night	16,911	10%
	What you do with your free time	16,911	10%
Q73 - How old were you when you first?	Got drunk	16,911	19%
Q78 Do you know the postcode for your home address?		16,911	43%
Q84 - To what extent do you agree or disagree with the following statements? 'My school provides me with enough advice and support about…?	Drinking alcohol	16,911	10%
	Taking drugs	16,911	10%
	Smoking	16,911	10%
	Leading a healthy and active life	16,911	10%
Q85 - Thinking about the future, how confident do you feel about…?	Saying no to doing something that you don't want to do	16,911	10%
	Knowing where to go for information and support about substance related issues	16,911	10%
	Avoiding getting into risky situations due to alcohol	16,911	10%
	Avoiding getting into risky situations due to drugs	16,911	11%
Q88 - In the past year, how many times did you skip or skive school?		16,911	10%
Q90 Strengths and Difficulties Questionnaire	All items	16,911	11-14%