Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS) Data Over Time - Stage 1: Feasibility Report

3. Conclusions and Recommendations

3.1.The survey's objective has been consistent over the years. It has used the same method of data collection (paper self-completion among children in school), and similar sampling strategies (all designed to provide nationally representative estimates at a minimum). While the scope of the survey has increased over time, there has been a large degree of consistency across waves.

3.2.Therefore, we conclude that the data is consistent enough between years for a combined dataset to allow meaningful analysis of trends over time. Stage 2 of the research should proceed.

3.3.With regard to the change in sample design between 2000 and 2002:

• The data for 2000 and before should not be amended to bring it into line with 2002 (by stripping out the data for S1 and S3 pupils) as secondary analysts may want to compare the full S1 to S4 data between 1990 and 2000.
• Instead, this change in sample coverage should be prominent in the user guide.
• Additionally, variables combining the weights from different waves should be designed to make it easy to carry out appropriate analysis of trends over time and difficult to carry out incorrect analysis. (This would be done by setting the values in the main weighting variables to 0 for S1 and S3 pupils in the earlier waves, but including this weighting data in supplementary variables).

3.4.With regard to the later fieldwork period in the 2002 and 2004 waves (and therefore the older average age of respondents):

• The data for 2002 and 2004 cannot be amended to control for this difference. The user guide should, however, clearly highlight this. It should include a section on average age of pupils in each year group by wave if possible - to help ensure that secondary analysts are made aware of the difference.

3.5.With regard to coverage of the combined dataset:

• Questions that have been asked in at least three separate waves should be included in the combined dataset.
• Questions that have been asked in the 2010 and 2013 waves (and will be included in the 2015 wave).
• Some questions comprise a sizeable number of individual components to derive the main measures. For these, the combined data-set should only include the main measures and key sub-measures. (For example: a) total number of units of alcohol consumed in the last week and number of units for each type of alcohol - but not alcohol by type by how consumed; b) number of brands of cigarettes mentioned, but not whether mentioned each brand separately; c) Main Strengths and Difficulties Score and scores for each internal scale (but not scores for each individual question).
• Where there has been a change in a question wording, response categories or routing that is likely to affect comparability over time, we would include the data in separate variables (with related names). We will discuss these with the Scottish Government as Stage 2 progresses and fully document these in the user documentation.

3.6.We would also recommend:

• If any discrepancies are discovered between the published results and the deposited data, we would discuss the appropriate course of action with the Scottish Government.
• That the variables in the combined data are named using conventions based on the topic area. (For example, all questions relating to smoking starting S_....)

Figure 4: Summary of coverage over time of smoking related questions

Figure 5: Summary of coverage over time of alcohol related questions

Figure 6: Summary of coverage over time of drugs related questions

Figure 7: Summary of coverage over time of questions related to sport, leisure, social contact, mental well-being and school.

Figure 8: Summary of coverage of demographic information and other contextual information.

Figure 9a: Summary of methodology (1982 to 1998)

Methodology	1982	1984	1986	1990	1992	1994	1996	1998
Data available from the Data Archive	No	No	No	Yes	Yes	Yes	Yes	Yes
Who carried it out	ODPM	ODPM	ODPM	OPCS	OPCS	ONS	ONS	ONS
Target population - age/year groups included	1st to 4th form	S1 to S4	S1 to S4	S1 to S4	S1 to S4	S1 to S4	S1 to S4	S1 to S4
Target population - State and Independent schools?	Y	Y	Y	Y	Y	Y	Y	Y
Summary of sampling approach	Random sample of schools and then a sample of pupils from schools	Random sample of schools and then a sample of pupils from schools	Random sample of schools and then a sample of pupils from schools	Random sample of schools and then a sample of pupils from schools	Random sample of schools and then a sample of pupils from schools	Random sample of schools and then a sample of pupils from schools	Random sample of schools and then a sample of pupils from schools	Random sample of schools and then a sample of pupils from schools
Designed to be representative at Alcohol Drug Action Team (ADAT) level	Y	Y	Y	Y	Y	Y	Y	Y
Designed to be representative at Local Authority level	N	N	N	N	N	N	N	N
Designed to be representative at NHS Board level	N	N	N	N	N	N	N	N
Designed to be representative at NHS Board level	N	N	N	N	N	N	N	N
Fieldwork dates	Oct/Nov 1982	Oct/Nov 1984	Unclear	Oct/Nov 1990	Oct/Nov 1992	Oct/Nov 1994	Oct/Nov 1996	Oct/Nov 1998
Achieved sample in Scotland	2,287	2,798	2,710	2,681	2,917	2,725	2,397	3,538
Number of schools that took part in Scotland	69	85	81	?	96	92	82	122
School response rate	96%	89%	82%	92%	96%	92%	82%	81%
Pupil response rate	87%	91%	91%	90%	90%	90%	89%	89%
Overall response rate	84%	80%	78%	87%	86%	83%	73%	70%
Note on weighting	None	Unclear	Unclear	Unclear	Unclear	Unclear	Unclear	School year (probability of selection of each year varied across schools)

Figure 9b: Summary of methodology (2000 to 2013)

Methodology	2000	2002	2004	2006	2008	2010	2013
Data available from the Data Archive	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Who carried it out	NFER	HBSC	HBSC	BMRB	Ipsos MORI	Ipsos MORI	Ipsos MORI
Target population - age/year groups included	S1 to S4	S2+S4	S2+S4	S2+S4	S2+S4	S2+S4	S2+S4
Target population - State and independent schools?	Y	Y	Y	Y	Y	Y	Y
Summary of sampling approach	Two stage - sample of schools and then sample of pupils	School-based stratified sample, with class as the PSU	School-based, stratified sample, with class as the PSU	School-based, stratified sample, with class as the PSU	School-based, stratified sample, with class as the PSU	School-based, stratified sample, with class as the PSU	School-based stratified cluster sample, with class as the PSU
Designed to be representative at Scotland level	Y	Y	Y	Y	Y	Y	Y
Designed to be representative at Alcohol Drug Action Team (ADAT) level	N	Y	N	Y	N - though enough to monitor prevalence	Y	Y
Designed to be representative at Local Authority level	N	Y	N	Y	N - though enough to monitor prevalence	Y	Y
Designed to be representative at NHS Board level	N	Y	N	Y	N - though enough to monitor prevalence	Y	Y
Fieldwork dates	Sept-Dec, 2000	Spring Term, 2003	Spring Term, 2005	Autumn term 2006	Sept 2008 - Feb 2009	Sept 2010 - Feb 2011	Sept 2013 - Mar 2014
Achieved sample in Scotland	4,774	23,090	7,062	23,180	10,063	37,307	33,685
Number of schools that took part in Scotland	150	314	194	258	253	302	283
School response rate	72%	75%	72%	69%	69%	75%	71%
Pupil response rate	90%	89%	87%	82%	91%	91%	90%
Overall response rate	64%	65%	62%	57%	59%	62%	60%
Note on weighting	No weighting	Unclear	No weighting	Sex, Year, School type (x2), area	Sex, Year, School type (x2), area	Sex, Year, School type (x2), area	Sex, Year, School type (x2), area

How to access background or source data

The data collected for this social research publication

☒ are available via an alternative route of the UK Data Archive