Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS): Mode Effect Study Report 2015
Findings of a mode effect study conducted during the 2015 SALSUS survey to see whether the different routes of administration resulted in different data.
4 Does mode affect responses to key substance use questions?
SALSUS data is used by Scottish Government policy makers (in alcohol, drugs and tobacco) to monitor key targets, with at least one key trend extending back to the 1980s. This means it is crucial that SALSUS can continue to deliver robust evidence that can be used for monitoring purposes and maintain the valuable, longstanding trends.
This chapter presents findings on whether or not the mode had any impact on the key substance use measures, a key element of the Mode Effect Study.
Prevalence of substance use measures - key findings
- On almost all key substance use measures, there was no statistically significant difference between the paper and the online results
- There were statistically significant differences for two (related) drug use measures for 15 year olds boys
- Additional comparisons showed no wider pattern of a mode effect among 15 year old boys
Prevalence of substance use
On almost all measures, t-tests revealed there were no statistically significant differences between the paper and the online results (see Table 4.1). If there had been a mode effect, we would have expected to see statistically significant differences, in the same direction, across a range of measures.
On two of the measures (drug use in the past year and drug use ever), for one of the sub-groups (15 year old boys), there were statistically significant differences between the paper and the online results. However, these two measures are very closely linked: they are based on the same question (' when was the last time you ever used or took any of the following…?') and anyone who answered 'in the last year' was automatically included in the 'ever' figures. Almost all of the 15 year old boys who had ever taken drugs had also taken drugs in the past year, hence these two variables are essentially measuring the same thing. In this context, they should, therefore, be seen as two aspects of the same measure.
Table 4.1: Comparison of paper and online results for key substance use measures
| 2015 paper prevalence | 2015 online prevalence | Statistically significant difference | |
|---|---|---|---|
| Regular smoker | |||
| 13 yr old boys | 1.6% | 1.4% | No |
| 13 yr old girls | 1.7% | 1.7% | No |
| 15 yr old boys | 8.1% | 7.0% | No |
| 15 yr old girls | 7.7% | 6.8% | No |
| Drink alcohol at least once a week | |||
| 13 yr old boys | 2.4% | 2.2% | No |
| 13 yr old girls | 2.5% | 2.5% | No |
| 15 yr old boys | 12.1% | 11.4% | No |
| 15 yr old girls | 12.5% | 13.9% | No |
| Drank alcohol in the last week | |||
| 13 yr old boys | 3.8% | 3.8% | No |
| 13 yr old girls | 4.2% | 4.4% | No |
| 15 yr old boys | 16.2% | 14.1% | No |
| 15 yr old girls | 18.7% | 18.8% | No |
| Used drugs in the last month | |||
| 13 yr old boys | 3.4% | 2.6% | No |
| 13 yr old girls | 3.2% | 3.0% | No |
| 15 yr old boys | 14.9% | 12.3% | No |
| 15 yr old girls | 9.5% | 7.6% | No |
| Used drugs in the last year | |||
| 13 yr old boys | 5.4% | 4.3% | No |
| 13 yr old girls | 4.7% | 4.2% | No |
| 15 yr old boys | 21.3% | 16.2% | Yes |
| 15 yr old girls | 14.9% | 13.6% | No |
| Ever used drugs | |||
| 13 yr old boys | 6.3% | 5.2% | No |
| 13 yr old girls | 5.6% | 4.8% | No |
| 15 yr old boys | 23.2% | 18.8% | Yes |
| 15 yr old girls | 17.1% | 14.9% | No |
To investigate whether there was a wider mode effect trend among 15 year old boys, a number of further comparisons were carried out for an additional 12 variables.
Given that nearly 50 comparisons were conducted, it is reasonable to assume that at least some of the differences would be statistically significant. In total, only eight of the additional comparisons were significant and none were among 15 year old boys. This indicates that there is not a more widespread effect on them due to the mode.
Table 4.2 Comparisons of paper and online for additional survey measures [11]
| 2015 paper prevalence | 2015 online prevalence | Statistically significant difference | |
|---|---|---|---|
| Never used e-cigarettes | |||
| 13 yr old boys | 84.6% | 83.7% | No |
| 13 yr old girls | 87.3% | 86.4% | No |
| 15 yr old boys | 64.8% | 68.4% | No |
| 15 yr old girls | 70.5% | 69.9% | No |
| Ever been drunk | |||
| 13 yr old boys | 42.0% | 43.0% | No |
| 13 yr old girls | 43.9% | 52.9% | Yes |
| 15 yr old boys | 63.8% | 67.9% | No |
| 15 yr old girls | 69.2% | 72.4% | No |
| Expects to go to University | |||
| 13 yr old boys | 50.9% | 46.6% | No |
| 13 yr old girls | 65.2% | 62.8% | No |
| 15 yr old boys | 46.0% | 47.6% | No |
| 15 yr old girls | 63.7% | 64.0% | No |
| Below median Mother's knowledge | |||
| 13 yr old boys | 33.3% | 35.2% | No |
| 13 yr old girls | 27.0% | 32.0% | Yes |
| 15 yr old boys | 45.8% | 48.6% | No |
| 15 yr old girls | 38.6% | 39.9% | No |
| Hangs out on the street at least weekly | |||
| 13 yr old boys | 37.5% | 44.8% | Yes |
| 13 yr old girls | 33.7% | 40.5% | Yes |
| 15 yr old boys | 36.8% | 39.7% | No |
| 15 yr old girls | 31.9% | 34.9% | No |
| Has two or more close friends | |||
| 13 yr old boys | 95.1% | 95.8% | No |
| 13 yr old girls | 95.1% | 94.0% | No |
| 15 yr old boys | 94.2% | 93.7% | No |
| 15 yr old girls | 93.9% | 94.0% | No |
| Spent 4 or more nights out a week with friends | |||
| 13 yr old boys | 45.7% | 47.8% | No |
| 13 yr old girls | 43.5% | 45.8% | No |
| 15 yr old boys | 42.3% | 41.6% | No |
| 15 yr old girls | 37.4% | 36.5% | No |
| Mostly has older friends | |||
| 13 yr old boys | 4.0% | 4.5% | No |
| 13 yr old girls | 3.1% | 2.7% | No |
| 15 yr old boys | 5.0% | 4.4% | No |
| 15 yr old girls | 4.2% | 4.1% | No |
| Has ever been excluded | |||
| 13 yr old boys | 8.4% | 9.1% | No |
| 13 yr old girls | 9.3% | 7.9% | No |
| 15 yr old boys | 13.2% | 11.9% | No |
| 15 yr old girls | 8.6% | 7.3% | No |
| Does not like school at all | |||
| 13 yr old boys | 8.1% | 9.0% | No |
| 13 yr old girls | 6.9% | 10.2% | Yes |
| 15 yr old boys | 10.3% | 10.9% | No |
| 15 yr old girls | 15.0% | 17.9% | Yes |
| Feels pressured by school a lot of the time | |||
| 13 yr old boys | 15.4% | 20.9% | Yes |
| 13 yr old girls | 21.2% | 27.8% | Yes |
| 15 yr old boys | 34.6% | 36.8% | No |
| 15 yr old girls | 59.5% | 63.1% | No |
| Ever truanted | |||
| 13 yr old boys | 34.2% | 33.1% | No |
| 13 yr old girls | 31.8% | 33.0% | No |
| 15 yr old boys | 41.9% | 38.1% | No |
| 15 yr old girls | 44.0% | 42.8% | No |
Contact
There is a problem
Thanks for your feedback