Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS): technical report 2018

Survey design and implementation

Fieldwork Period

The fieldwork for the survey was conducted between September 2018 and April 2019. While the majority of questionnaires were returned by January, the completion and return of questionnaires was delayed in a number of schools so the fieldwork period was extended to maximise the response rate. The fieldwork period was broadly in line with the 2015, 2013, 2010, 2008 and 2006 surveys and surveys in the series prior to 2002. However, it was earlier in the school year than the 2002 and 2004 surveys, which were conducted between February and May and, therefore, sampled slightly older pupils.

Study Design

Prior to 2015, SALSUS had historically been a paper-based survey, administered in schools. As part of the 2015 wave of SALSUS, a study was conducted to explore the feasibility of administering the survey online. This consisted of a feasibility study, an online pilot and a mode effects study.^[2] This concluded that there was no evidence of a mode effect and the data from the online and paper samples were combined for analysis and reporting of the 2015 results. This approach was also taken in 2018. Overall, 71.5% of pupils completed online and 28.5% on paper^[3].

Sample design

The sample design aims to create a subset that is as representative as possible of the population of S2 and S4 pupils in mainstream schools in Scotland. This is critical to obtaining reliable estimates of the prevalence of smoking, drinking and drug use within this population group.

The Scottish Government school database was used as the sampling frame. All state funded, grant-maintained and independent secondary schools in Scotland were included in the sampling frame. As in previous years, special schools were excluded.

The primary sampling unit (PSU) was S2 and S4 classes within each of the schools in the sampling frame. In total, 409 schools with an estimated 101,943 S2 and S4 pupils and an estimated 4,721 S2 and S4 classes were included in the sampling frame.

The sample was stratified by local authority, and within each local authority area, by school type (state school or independent school), then by year group (S2 or S4). A sampling fraction was calculated for each local authority. This varied depending on the number of pupils targeted in each local authority. The number of classes selected within each school depended upon the number of pupils, an estimate of average class size, and the 2015 local authority response rate. An average class size of 23 was used, based on the average class size surveyed in 2015. Sampled classes were split 50/50 between the two school years for the final level of stratification by year group.

Based on the assumption that the response rate would be similar to, or slightly lower than, that achieved in 2015, proportionately more schools were selected in local authorities with relatively low school response rates in 2015. For example, if we estimated that 40 classes were required from local authority A (80% school participation rate in 2015) and 40 classes were required from local authority B (60% school participation rate in 2015), we would sample 50 classes from local authority A and 67 classes from local authority B. Within the broad confines of the design, this approach maximises the precision of the survey estimates by minimising the need for post-survey corrective weighting of the local authority samples.

A spreadsheet was created, that listed every S2 and S4 class within each local authority, for example:

• school A class 1
• school A class 2
• school A class 3
• school A class 4
• school B class 1
• school B class 2
• school C class 1
• school C class 2
• school C class 3

Although the exact number of classes in each school was not known, an estimate was made based on the total number of S2 and S4 pupils in the local authority and the size of the school. A random starting point was selected, then the sampling fraction was used to select the required number of classes within each local authority. At this stage, the names of classes were not known, so the result was a list of schools which had one or more of its classes selected, and the number of S2 classes and S4 classes selected within each of these schools.

Head teachers were sent an advance letter that explained their school had been selected to take part in SALSUS. It outlined the benefits of completing online in terms of cost, environmental impact and data quality, but also noted the school could complete on paper if this was easier.

The final stage in the sampling process took place after relevant schools had agreed to take part. Interviewers telephoned schools to ascertain the number of S2 and S4 classes within the school. Classes were noted down in either numerical or alphabetical order, depending on how the school named them, e.g. 4A, 4B, 4C etc. or 2 'Ben Loyal', 2 'Ben Nevis', 2 'Suilven' etc. For each school, the Computer Assisted Telephone Interviewing program randomly selected which classes to invite to participate. This ensured that the whole selection process was purely random.

An additional three local authorities (Dumfries and Galloway, Moray and South Ayrshire) took part in the Scottish Government programme Realigning Children's Services (RCS^[4]). The RCS local authorises were in fact a census as all schools received one of two versions of the questionnaire: either one with the full set of SALSUS questions and a subset of RCS questions, or another with only the main RCS questionnaire. Sampling for the former was carried out in the same way as SALSUS in other local authorities. The remaining S2 and S4 pupils were asked to complete the RCS only questionnaire, as were all pupils in S1 and S4.

Access and consent

To obtain permission to contact schools, the Scottish Government sent an opt-out letter to the Director of Education in each local authority and to the Scottish Council of Independent Schools. The letter explained the purpose of the survey and what would be required from participating schools. No opt-outs were received.

In some areas, it is necessary to complete a research request application in order to gain permissions to conduct research in schools, in addition to writing to the Director of Education. These applications were submitted to, and approved by, the six local authorities concerned. Glasgow City Council was one of these areas and although permission was granted, this local authority did not ultimately take part in SALSUS. This was due to the fact that their own Health and Wellbeing Census was running at the same time.

Head teachers were approached by telephone to find out if they were willing for their school to take part. After initial permission had been granted, a school liaison contact was identified. This contact was responsible for listing the number of S2 and S4 classes within the school to enable random selection. They were also responsible for overseeing the administration and return of the surveys in their schools and received an instruction pack (Appendix A) to assist them. In addition, they received copies of instructions for administering the questionnaire to distribute to the teachers involved (Appendix B).

Procedures were in place to ensure that pupils who took part did so on the basis of informed consent from themselves and their parents. Around a week prior to the survey being administered both parents and pupils were sent information explaining the purpose of the survey and the topics covered in the questionnaire (Appendix C). The information explained that the pupils did not have to take part if they did not wish to. It also contained a link to the SALSUS 2018 privacy notice, which outlined why the information is collected, how it is used and the rights of participants.^[5] Parents were provided with slips to return to the school if they did not wish their child to take part. In addition, immediately before the questionnaire was completed, teachers were asked to remind pupils that they did not have to complete the questionnaire or they could refuse to answer specific questions.

Administration of the survey

The pupils completed the survey within class time, in a mixed ability class period such as Personal and Social Education (PSE) or Personal Health and Social Education (PHSE). Teachers were instructed to administer the questionnaires under 'exam' conditions to try to encourage honest answering.

To ensure confidentiality, pupils completing the paper survey were provided with sealable envelopes for their completed questionnaires. Responses were anonymous and pupils were instructed not to write their names anywhere on the questionnaire or envelope. The sealed envelopes were then handed back to the teacher. Couriers collected the boxes of completed questionnaires from the schools and delivered them to Ipsos MORI where the data were entered through scanning.

The survey was accessed via a link which required a unique log-in code. To ensure anonymity for pupils completing the online survey, randomised log-in codes were printed on pages of stickers and pupils were able to select one of the codes at random. This meant that the log-in was not linked to them by the teacher.

Schools were strongly encouraged to follow up on pupils who were absent on the day their class completed the questionnaires and to ensure that pupils who completed the questionnaire at a later date did so under the same 'exam' conditions as pupils in the main session. The main difference was that the pupils were given a reply paid envelope to return the questionnaire themselves. This was important because pupils completing the questionnaire on their own, or in a small group, would not have the same reassurance that their response was anonymous and would simply be added to a pile of other sealed envelopes. Teachers were asked to give pupils the reply paid envelope at the same time as the questionnaire and to explain at the outset that they should put it in the post themselves.