Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS): Mode Effect Study Report 2015

Findings of a mode effect study conducted during the 2015 SALSUS survey to see whether the different routes of administration resulted in different data.


4 Does mode affect responses to key substance use questions?

SALSUS data is used by Scottish Government policy makers (in alcohol, drugs and tobacco) to monitor key targets, with at least one key trend extending back to the 1980s. This means it is crucial that SALSUS can continue to deliver robust evidence that can be used for monitoring purposes and maintain the valuable, longstanding trends.

This chapter presents findings on whether or not the mode had any impact on the key substance use measures, a key element of the Mode Effect Study.

Prevalence of substance use measures - key findings

  • On almost all key substance use measures, there was no statistically significant difference between the paper and the online results
  • There were statistically significant differences for two (related) drug use measures for 15 year olds boys
  • Additional comparisons showed no wider pattern of a mode effect among 15 year old boys

Prevalence of substance use

On almost all measures, t-tests revealed there were no statistically significant differences between the paper and the online results (see Table 4.1). If there had been a mode effect, we would have expected to see statistically significant differences, in the same direction, across a range of measures.

On two of the measures (drug use in the past year and drug use ever), for one of the sub-groups (15 year old boys), there were statistically significant differences between the paper and the online results. However, these two measures are very closely linked: they are based on the same question (' when was the last time you ever used or took any of the following…?') and anyone who answered 'in the last year' was automatically included in the 'ever' figures. Almost all of the 15 year old boys who had ever taken drugs had also taken drugs in the past year, hence these two variables are essentially measuring the same thing. In this context, they should, therefore, be seen as two aspects of the same measure.

Table 4.1: Comparison of paper and online results for key substance use measures

2015 paper prevalence 2015 online prevalence Statistically significant difference
Regular smoker
13 yr old boys 1.6% 1.4% No
13 yr old girls 1.7% 1.7% No
15 yr old boys 8.1% 7.0% No
15 yr old girls 7.7% 6.8% No
Drink alcohol at least once a week
13 yr old boys 2.4% 2.2% No
13 yr old girls 2.5% 2.5% No
15 yr old boys 12.1% 11.4% No
15 yr old girls 12.5% 13.9% No
Drank alcohol in the last week
13 yr old boys 3.8% 3.8% No
13 yr old girls 4.2% 4.4% No
15 yr old boys 16.2% 14.1% No
15 yr old girls 18.7% 18.8% No
Used drugs in the last month
13 yr old boys 3.4% 2.6% No
13 yr old girls 3.2% 3.0% No
15 yr old boys 14.9% 12.3% No
15 yr old girls 9.5% 7.6% No
Used drugs in the last year
13 yr old boys 5.4% 4.3% No
13 yr old girls 4.7% 4.2% No
15 yr old boys 21.3% 16.2% Yes
15 yr old girls 14.9% 13.6% No
Ever used drugs
13 yr old boys 6.3% 5.2% No
13 yr old girls 5.6% 4.8% No
15 yr old boys 23.2% 18.8% Yes
15 yr old girls 17.1% 14.9% No

To investigate whether there was a wider mode effect trend among 15 year old boys, a number of further comparisons were carried out for an additional 12 variables.

Given that nearly 50 comparisons were conducted, it is reasonable to assume that at least some of the differences would be statistically significant. In total, only eight of the additional comparisons were significant and none were among 15 year old boys. This indicates that there is not a more widespread effect on them due to the mode.

Table 4.2 Comparisons of paper and online for additional survey measures [11]

2015 paper prevalence 2015 online prevalence Statistically significant difference
Never used e-cigarettes
13 yr old boys 84.6% 83.7% No
13 yr old girls 87.3% 86.4% No
15 yr old boys 64.8% 68.4% No
15 yr old girls 70.5% 69.9% No
Ever been drunk
13 yr old boys 42.0% 43.0% No
13 yr old girls 43.9% 52.9% Yes
15 yr old boys 63.8% 67.9% No
15 yr old girls 69.2% 72.4% No
Expects to go to University
13 yr old boys 50.9% 46.6% No
13 yr old girls 65.2% 62.8% No
15 yr old boys 46.0% 47.6% No
15 yr old girls 63.7% 64.0% No
Below median Mother's knowledge
13 yr old boys 33.3% 35.2% No
13 yr old girls 27.0% 32.0% Yes
15 yr old boys 45.8% 48.6% No
15 yr old girls 38.6% 39.9% No
Hangs out on the street at least weekly
13 yr old boys 37.5% 44.8% Yes
13 yr old girls 33.7% 40.5% Yes
15 yr old boys 36.8% 39.7% No
15 yr old girls 31.9% 34.9% No
Has two or more close friends
13 yr old boys 95.1% 95.8% No
13 yr old girls 95.1% 94.0% No
15 yr old boys 94.2% 93.7% No
15 yr old girls 93.9% 94.0% No
Spent 4 or more nights out a week with friends
13 yr old boys 45.7% 47.8% No
13 yr old girls 43.5% 45.8% No
15 yr old boys 42.3% 41.6% No
15 yr old girls 37.4% 36.5% No
Mostly has older friends
13 yr old boys 4.0% 4.5% No
13 yr old girls 3.1% 2.7% No
15 yr old boys 5.0% 4.4% No
15 yr old girls 4.2% 4.1% No
Has ever been excluded
13 yr old boys 8.4% 9.1% No
13 yr old girls 9.3% 7.9% No
15 yr old boys 13.2% 11.9% No
15 yr old girls 8.6% 7.3% No
Does not like school at all
13 yr old boys 8.1% 9.0% No
13 yr old girls 6.9% 10.2% Yes
15 yr old boys 10.3% 10.9% No
15 yr old girls 15.0% 17.9% Yes
Feels pressured by school a lot of the time
13 yr old boys 15.4% 20.9% Yes
13 yr old girls 21.2% 27.8% Yes
15 yr old boys 34.6% 36.8% No
15 yr old girls 59.5% 63.1% No
Ever truanted
13 yr old boys 34.2% 33.1% No
13 yr old girls 31.8% 33.0% No
15 yr old boys 41.9% 38.1% No
15 yr old girls 44.0% 42.8% No

Contact

Back to top