Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS): Mode Effect Study Report 2015

7 What else has been learned?

In general, among those schools that had at least one class who completed the survey using the online administration, the process ran smoothly and appeared to be straightforward. The lower response rate for the online survey, however, suggests that it is a less appealing methodology from some schools' perspectives. Overall only a small number of schools raised any issues regarding the online administration. These are described in this chapter.

The survey link

The most common problems were associated with the survey link. There were two main issues.

Firstly, some schools lost their note of the survey link. The link was provided to schools on a single paper document. This meant that if it was misplaced and the school no longer had a record of it, they had to get in touch with the survey team before they could proceed with the survey (this also applied to the test log-ins we provided for liaison teachers to check the survey in advance of administration). While this did not prevent any schools from participating, it did cause them unnecessary extra effort.

This is easily fixed, and in future waves of the research we would recommend including the survey link on multiple documents (class teacher instructions, liaison teacher instructions etc.), in addition to providing it electronically by email.

Secondly, in a number of schools, some pupils had problems entering the survey link into their web browser. When someone begins to type in the survey link https://www.ipsos-mori.com/SALSUS2015 into the web browser, the browser bar suggestions give the Ipsos MORI website. Pupils were clicking on the suggested website, rather than typing in the full survey link. Alternatively, they entered the full web address but then hit 'search' (circled in red) instead of 'go to' (circled in green). This took them to a results page, which again linked to the main Ipsos MORI website - the survey would not show up in a search engine results list as it is hosted on a secure server and is not visible to the public.

Figure 7.1 Example of survey link in web browser

In some schools, this was prevented as the class teacher had uploaded the survey link to all of the computers in the ICT suite in advance rather than individual pupils typing in the survey address. In future waves, we would suggest that schools did this, where possible. In addition, it would be important to cover this issue in the class teacher survey instructions so that they know how to deal with the problem if it occurs. Furthermore, we would suggest greater exploration of which websites appear more frequently in the search results with a view to putting a redirection notice on relevant Ipsos MORI web pages. However, this option would require further investigation.

Other technical problems

As previously noted, very few schools contacted Ipsos MORI with technical problems. However, there was evidence from the class response sheets, that a very small number of pupils (n=26) could not complete the survey due to technical problems. It is difficult to know exactly what these issues were from the descriptions provided. The indications are that they were related to the schools' IT systems rather than the survey itself (e.g. internet problems, computers crashing etc.).