Scottish Household Survey User Engagement Report 2020

4. User engagement methods

Methods

• Questionnaires
• One-to-one consultations
• Interactive testing
• Feedback banner

The Scottish Household Survey team used a wide range of different methods to engage with its users. Views on the data explorer as well as the new reporting format were collected both quantitatively and qualitatively. Below follows a detailed outline of all methods used throughout the whole process.

4.1 Quantitative data collection

In total, three questionnaires were designed and distributed at different points in time between the 26^th of June 2019 and the 15^th of October 2020 to collect views on user needs and evaluations of the data explorer as well as the new annual report format in 2020.

An initial questionnaire (Annex A) was sent out between the 26^th of June 2019 and the 5^th of August 2019 to gather users’ needs for the new data dissemination tool, later known as the Data Explorer. On the first day of the survey, e-mail invitations were sent out to 92 individuals. 30 respondents took part in the survey.

Once a prototype of the Data Explorer with 2018 survey data had been developed by the 28^th of May 2020, another questionnaire (Annex B) was sent out to gather feedback on the beta version for improvement. The survey went live on the 28^th May 2020. On the first day of the survey, e-mail invitations were sent out to 87 individuals, followed by an e-mail reminder one week later. An invitation to take part in the survey was also sent out via a ScotStat e-mail. The survey received 13 responses.

Lastly, between the 15^th of September 2020 and the 15^th of October 2020, users were invited to fill out an evaluation questionnaire (Annex C) of the 2020 reporting format of all new products, including the 2019 data explorer and four different reports. On the first day of the survey, e-mail invitations were sent out to 261 individuals, followed by an e-mail reminder one week later. An invitation to take part was also sent out via a ScotStat e-mail. 18 respondents took part in the survey.

4.2 Qualitative data collection

During the development phase of the data explorer, one-to-one meetings face to face, over e-mail and over the phone were carried out. This ‘open-ended feedback’ allowed a more in-depth understanding of users’ views on the data explorer prototype. 34 SHS data users and survey data dissemination experts gave a combination of written and verbal feedback on the data explorer beta version between the 21^st of February 2020 and the 10^th of July 2020.

4.3 Interactive testing workshops

16 individuals – representing the Scottish Government, as well as friends and school children - were invited to take part of a live demonstration and test the data explorer prototype version on the 21^st of February 2020. The testing involved a presentation of the dashboard followed by an online quiz [https://veriksson94.shinyapps.io/shsapptutorial/] for the users to answer by using the data explorer. This testing workshop allowed for the users to interactively engage with the new product, and the Scottish Household Survey team to assess how easy and accessible the tool was.

4.4 On-going feedback requested through feedback banner

We continuously welcome feedback of the data explorer through a feedback banner on the data explorer, signposting users to the Scottish Household Survey mailbox.