Scottish Household Survey 2021: methodology and fieldwork outcomes

Data collection methods and instruments

Computer Aided Personal Interviewing (CAPI)

In common with many other large-scale government surveys, the SHS social interview is carried out using Computer Aided Personal Interviewing (CAPI). This offers a number of important advantages over traditional pen-and-paper interviewing for a survey of this kind.

CAPI programming is integral to ensuring high quality data. The main parameters of the data are defined within the programme, such as; the acceptable range of responses to a question; the acceptable relationships between questions – or, the routing; and the relationships between responses given at different questions.

Social survey questionnaire structure

The social questionnaire is in two parts:

• Household – Information is collected about the composition and characteristics of the household from the Highest Income Householder or their spouse/partner. This allows the SHS to produce information representative of Scottish households. A 'random school child' is also selected, and the household respondent answers questions about the school that child attends and how the child travels there.
• Random adult – Information is collected focusing on the attitudes and experiences of a randomly selected adult member of the household. This element also covers the 'Travel Diary' component which asks about travel behaviours on the day previous to that of the interview day. This produces data representative of the Scottish adult population resident in private households.

The questionnaire has been designed to give flexibility in terms of topic, frequency and geography. For example, questions asked of the full sample and asked on an annual basis would be able to provide local authority level estimates on an annual basis. Similarly, questions might be asked of 1/3 of the sample on a biennial basis (i.e. asked every second year). Such questions would provide national level estimates.

The 2021 questionnaire is provided as a supporting document to this publication.

Physical survey

At the end of the household component of the survey, the household respondent is asked if they would be willing to have the follow-up component 'physical survey' of the dwelling arranged. These are conducted by professional surveyors through a visual inspection of the dwelling, to assess the condition, design and energy efficiency of the home. Physical survey appointments are generally made for between 7 and 14 days after the interview date.

Survey fieldwork

The social survey fieldwork for the 2021 sample began in April 2021 and completed in March 2022. The physical survey fieldwork began in May 2021 and completed in April 2022.

All interviews were undertaken remotely by telephone or video. There was no travel by interviewers.

Households were sent a letter and leaflet outlining the purpose of the survey and the importance of participation. Participants were directed to an online portal where they could log in using a unique reference and then submit their name and contact details.

In the first half of the fieldwork, telephone matching was carried out, matching names and telephone numbers to addresses using publicly available sources, such as the electoral register and the telephone directory. Matching was successful for 34% of addresses. Analysis carried out during 2021, comparing the 2019 and 2020 results for variables we would expect to be relatively stable, found that the estimates from the 2020 telephone matched sample generally were substantially further from the 2019 figures than those from the 2020 opt-in sample. Therefore, telematching was not used for the second half of the 2021 fieldwork.

With no interviewer travel allowed, gaining consent for interview came either from respondents opting-in on receipt of the advance materials, or in response to an approach by telephone. After the initial mail-out, addresses where a phone number had been obtained were followed up by a telephone call. Interviewers made at least 8 attempts to call each telephone number.

For those where we were unable to obtain a telephone number, two reminders were sent after the initial mail-out, a postcard reminder followed by a letter reminder. Towards the end of the fieldwork year, a fourth reminder mail-out was sent to 35,000 of the addresses that did not opt-in in response to first three approaches.

To encourage participation, respondents were given a conditional incentive of £20 for completing the interview.

The tote bag incentive experiment that was running in the first half of the 2020 wave was restarted in 2021. Tote bags were included with the first letter to half of the sample of addresses for which Ipsos was unable to obtain a telephone number. The results of this experiment are available on the supporting documents page.