Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS) Data Over Time - Stage 1: Feasibility Report

2. Feasibility

2.1.The first stage of the research was designed to establish if it is feasible to combine the individual SALSUS datasets. 'Is it feasible?' comes down to answering the question, is the data consistent enough between years to allow meaningful comparison? (And if not, could the data be amended to allow comparisons to be made in a meaningful way?)

Overview of the data series

2.2.SALSUS has been the main vehicle for providing data on smoking among secondary school children since 2002. It built upon earlier studies in the 1980s and 1990.

2.3.The first survey was undertaken in 1982, by the Social Survey Division of Office of Population Censuses and Surveys (OPCS). This survey looked primarily at smoking, among secondary school pupils across Scotland, England and Wales^[1]. In Scotland, it collected data from just under 2,300 pupils across S1 to S4.

2.4.OPCS repeated the survey in 1984 and 1986 with minor changes. Unfortunately, the data from these three surveys, carried out in the 1980s, was not deposited in the UK Data Archive. Despite extensive efforts, it has not yet been possible to trace this data^[2].

2.5.The first wave of the survey where the data is readily available is the 1990 wave. Throughout the 1990s, the survey was repeated, largely unchanged, every two years - first by OPCS and then by its successor organisation, the Office of National Statistics (ONS). During this period, questions on alcohol consumption were added in 1990 and questions on drugs were first added in 1998.

2.6.In 2000, the survey was undertaken by National Foundation for Educational Research (NFER), and then in 2002 and 2004, by the Health Behaviour in School-Aged Children (HBSC) team at the University of St Andrews.^[3] (The 2002 wave was undertaken in conjunction with the HBSC survey). This was the first wave where the sample size allowed robust estimates at various sub-Scotland levels to be provided.

2.7.Over the last nine years, four waves were carried out - in 2006, 2008, 2010 and 2013. In 2006, it was conducted by British Market Research Bureau (BMRB). The last three waves have been conducted by Ipsos MORI.

Design and methodology

2.8.Fundamentally, the core objectives of the survey - to provide robust estimates of smoking (and then also drinking and drug taking) among secondary school children in Scotland - have remained consistent. The design and administration of SALSUS has also remained largely unchanged since its inception with many of its key features consistent over time (see Figures 9a and 9b):

The data collection has always used paper self-completion surveys administered in school^[4] under 'exam' conditions.

Similar sampling strategies have been used (all designed to provide nationally representative estimates at a minimum, and selection clustered around schools or classes or both).

Both state and independent schools have been included in the target population while special schools have been excluded.

Weighting has been used in most waves to correct for any effects of non-response to ensure that the results are representative on various characteristics of the target population.

2.9.Between 1982 and 2000, there was a drop in the overall response rate (Figure 1). This was driven by a reduction in the proportion of schools agreeing to take part rather than changes in the pupil response rate. However, since 2000, the overall response rate for SALSUS has remained broadly stable at around 60%.

Figure 1: Response rates for SALSUS and predecessors: 1982-2013^[5]

2.10. The sample size has increased over time (Figure 2). Until 1998, the sample size was always between 2,000 and 3,000 interviews per wave. It increased to 3,538 in 1998 and to 4,774 in 2000.

2.11. In 2002, the sample size was increased to 23,090. This allowed robust estimates at sub-Scotland level for the first time, with results available at local authority, NHS Heath Board, and Alcohol and Drug Action Team (ADAT) levels.

2.12. After 2002, it was envisaged that every alternate wave of SALSUS would have a sample size large enough to provide robust estimates at sub-Scotland level. This pattern was followed until the large 2010 wave. There was no survey undertaken in 2012, and the 2013 wave was designed as a large rather than a small wave, with an achieved sample size of 33,685.

Figure 2: Overall sample size by year: 1982-2013^[6] (1982-2000 covers S1 to S4 pupils: 2002 onwards only covers S2 and S4 pupils).

2.13. Apart from the increase in the overall sample size, there has only been one key change to the survey methodology (plus one change related to the administration of the survey) that will have a major impact on analysis of trends over time.

2.14. In 2002, the target population changed from all pupils in S1, S2, S3 and S4 to pupils in S2 and S4 only. This is important as age plays such a key role in the incidence of smoking, drinking and drug use. Trends over time have normally been reported using incidence levels among 13 year olds (in S2) and 15 year olds (in S4) separately. The change in the target population means that the 'usable' sample size for analysis of trends will be roughly half those shown in Figure 2^[7] for the years prior to 2002.

2.15. With regard to the administration of the survey, the 2002 and 2004 waves were undertaken during the spring term. The bulk of the fieldwork for all other waves was undertaken during the autumn term. This means that the 2002 and 2004 cohorts were on average 6 months older than pupils in other waves and is again a central consideration due to the importance of age to patterns of behaviour.

2.16. To examine the potential for meaningful analysis over time, the questionnaire coverage across the different waves was compared. This is detailed in Figures 4 to 8. They show whether a question (or set of questions on a topic) have been included in a particular wave. They also indicate if there have been any changes that might affect trend analysis or the construction of a combined dataset. The key to the charts is provided below.

Figure 3: Key to questionnaire coverage charts.

2.17. Note that this was completed to provide a high level summary of the topic coverage. These will be refined and annotated during Stage 2 to help document the combined data^[10].

2.18. There are a large number of questions that have been asked in a consistent manner over a number of waves. We summarise the questionnaire coverage across the different broad topics areas very briefly below.

2.19. Smoking (see Figure 4):

A sizeable number of questions on smoking behaviour, including key trend data such as whether ever smoked, access to tobacco, and starting smoking have been asked in a consistent way since 1982 (though data only exists from 1990).

Questions on giving up smoking were added in 1990 and have been asked in each wave since then.

Additionally, various questions on the behaviour and the attitudes of other people, as well as questions relating to the respondents' knowledge/ awareness of tobacco have been asked in a number of different waves.

2.20. Drinking (see Figure 5):

Questions on drinking were first asked in 1990. These include some of the key measures that have been asked in each wave since - such as whether ever had an alcoholic drink, when last had an alcoholic drink, and usual frequency of drinking - as well as the set of questions to calculate units consumed in the last seven days^[11].

Questions on attitudes of other people to the respondent drinking alcohol have been present since 2002.

A question on access to alcohol has been asked since 1996 although this has been revised on a number of occasions. Further questions (such as whether had tried to buy from a shop, supermarket etc.) were added in 2006.

2.21. Drugs (see Figure 6):

Questions on drug use were first introduced in 1998^[12], including whether ever taken any drugs and when last took various types of drugs^[13].

The section on drugs was expanded in 2002 with questions relating to access to drugs and a number of attitude statements added.

2.22. Questions on sport and leisure participation, social contact, mental well-being and attitudes to school (see Figure 7):

Questions on social contact and activities undertaken (such as frequency of participation in various leisure activities and sports) have been asked since 2002, with a number of questions added in 2006 (for example on number of close friends).

Questions on truancy, exclusion from school have been asked since 2002, with additional questions on how much they like school and how often they feel pressured by schoolwork added in 2006.

The strengths and difficulties questionnaire was first added in 2006, and the Warwick-Edinburgh Mental Well-being Scale (WEMWBS) was added in 2010.

2.23. Demographic questions and contextual information (see Figure 8):

Key demographic questions (sex, year group etc.) have been included since the start.

Questions have been added at various times, such as questions on household composition in 1996, while others have been removed from the survey.