Survey response

Introduction

The final number of social survey interviews achieved was 10,543. This is above the target of 10,450 interviews and represents an overall response rate of 46%.

Survey response is an important indicator of survey quality as non-response can introduce bias into survey estimates. Standardised outcome codes (based on an updated version of those published in Lynn et al (2001)) for survey fieldwork were applied across the SHS, SHeS and SCJS. The outcome codes paper includes guidance on the appropriate categorisation of interview outcomes. This will allow consistent reporting of fieldwork performance and effective comparison between the performance of the surveys.

Scotland level

The response rate of 46% was lower lower than pre-pandemic years, which exhibited a long-term mean adjusted response rate of 67% (1999/2000 – 2019). Adjusted response rates (response rates where sample is net of unreachable addresses such as unoccupied houses) were not possible to calculate for 2020 and 2021, therefore unadjusted response rates for 2019, 2022 and 2023 are presented alongside them in Table 3 for comparison.

This drop in response rate could be attributed to remaining effects of the COVID19 pandemic. All large population-based surveys in Scotland have been affected by the recent pandemic in terms of recent lower response rates. However, this downward trend had already been identified pre-2020. The mean response rate from 2015 to 2019 was 64%, 3 percentage points below the long-term average. Response rates will be continually monitored as a part of assessing survey quality.

It should be noted that from 2012 the calculation was changed slightly and a portion of the addresses of unknown eligibility are now considered to be eligible, whereas previously they would all have been classed as ineligible.

Table 3: Overall response rates by year

Local authority performance

Table 4: Response rate by local authority

The conversion from household interview to random adult completion was 92% in 2023.

Monitoring and reducing the respondent burden

The Code of Practice for Statistics states that "Statistics producers should be transparent in their approach to monitoring and reducing the burden on those providing their information, and on those involved in collecting, recording and supplying data." Furthermore it states that "The burden imposed should be proportionate to the benefits arising from the use of the statistics."

The following steps are/have been taken to reduce respondent burden.

• addresses selected for any of the surveys (SHS, SHeS, SCJS) are removed from the sample frame for a minimum of 4 years so that they cannot be re-sampled for another survey.
• samples sizes are no greater than required to obtain robust local authority estimates.
• the size of the SHS questionnaire was not allowed to increase following the SHS 2018-21 questionnaire review. Where there was a strong policy need for new questions, the space/time required for these new questions was created by dropping existing questions or moving them to a biennial basis.
• the SHS social interview is carried out using Computer Aided Personal Interviewing (CAPI). Routing is built into the CAPI script which ensures that respondents are only asked questions that are relevant to them, e.g. respondents are only asked questions on mortgage repayments if they have a mortgage.

Interviewer briefings are held annually. These events provide interviewers with an opportunity to feedback on improvements that could be made to the questionnaire to assist them conducting interviews and reduce the burden on respondents.

Physical survey

Just over half of the SHS sample (58%) was assigned to the physical survey module. For completion of the physical survey, respondents had to agree to make an appointment for a surveyor to make a follow-up visit and to complete the appointment. 

There was a target of at least 80 completed physical surveys for each local authority along with a target of 3,004 surveys for Scotland. The results of continuing lower than estimated levels of conversion from household survey to physical survey meant that there were fewer than 80 physical survey responses for 10 local authorities in 2023. The 3,153 surveys achieved for Scotland in 2023 exceeded the target of 3,004.

Table 5: Physical survey response rate