Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS) 2015: Online Feasibility Study

Executive Summary

Introduction

The Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS) is a continuation of a long established series of national surveys on smoking, drinking and drug use.

In the past the survey has always been administered on paper. However, as technology has advanced, the transition from paper to online administration is being considered for the 2015 wave of SALSUS. The move to a web-based survey is in line with other national surveys and reflects greater engagement with information technology (IT), particularly among young people.

Ipsos MORI Scotland conducted a study to explore the feasibility of administering the survey online. The results of the feasibility study will inform whether to move forward with the online methodology for SALSUS 2015, and specifically whether to conduct an online pilot (to identify any practical problems with an online methodology) and a mode effect experiment (to assess whether the change of mode has any significant impact on results).

Methods

The feasibility study comprised 5 strands:

• desk research, reviewing past examples of online research in schools
• a short postal survey of secondary headteachers
• telephone depth interviews with 9 liaison teachers from the most recent wave of SALSUS
• an online survey of all remaining liaison teachers
• telephone depth interviews with local government officers who could provide an overview of the IT capacity/technical issues for secondary schools in their area. Interviews were conducted with representatives from 31 of the 32 local authorities.

Key findings

Overall, the majority of local authorities (24 out of 32) would be able to complete SALSUS 2015 online. While 8 local authorities were classified as possibly being problematic, this was, to some extent, due to a lack of information.

The most likely means for running SALSUS 2015 will be on PCs in information and communications technology (ICT) suites, although a number of participants said that they would 'top-up' with laptops.

On balance, the results suggest that an online survey could result in a reduced response rate. In 2015, schools will be given the chance to complete on paper if they do not want to complete the survey online.

Participants highlighted a number of potential risks when completing the survey online including timetabling issues, a lack of computers and software compatibility. However, a number of options to overcome these obstacles were also mentioned.

Generally, liaison teachers thought that moving the survey online would be more enjoyable for pupils. Only a small proportion thought it would have a negative impact on pupils' honesty or concentration.

The current IT infrastructure could not support a full, nationally representative, 3-way (paper/PCs in ICT suites/tablets) mode experiment. However, a 2-way mode experiment (paper versus PCs in ICT suites) would be possible.

While there are some moves towards the use of mobile devices, such as tablets, schools appear to be retaining ICT suites for the near future. Therefore, it will be worthwhile to test this mode for future waves of SALSUS.

Important factors for consideration during an online pilot

As the mode effect study will focus on PCs in ICT suites, the online pilot should do the same. However, as a number of schools would be administering SALSUS online in classrooms, we would recommend including at least some of these schools in the pilot.

The online pilot should focus on the following areas:

• testing logistical issues (including timetabling, accommodating all pupils in a PSE class and establishing exam conditions)
• testing software issues (including software compatibility, connectivity and access to survey links)
• assessing pupil reaction to the survey.

Recommendations

The key recommendation is that the SALSUS 2015 electronic trial should move past break-point one and an online pilot should be conducted because it does appear feasible to conduct SALSUS online in 2015.

If the pilot goes well, and the trial progresses past break-point two, the mode effect experiment should be a two-way mode experiment comparing paper completion in classrooms to online completion on PCs in ICT suites.