Long term survey strategy: summary report and framework to support decision-making

Context and scope

The Scottish Government currently commissions three main large-scale cross-sectional general population surveys – the Scottish Household Survey (SHS – incorporating the Scottish House Conditions Survey), Scottish Crime and Justice Survey (SCJS) and Scottish Health Survey (SHeS). Data from these surveys underpin key measures in the National Performance Framework and Single Outcome Agreements between the Scottish Government and local authorities. They are extensively used by both the Scottish Government, other public sector bodies and wider stakeholders to support policy development and analysis. The outputs from the surveys are Accredited Official Statistics, meaning they are produced to standards set out in the Code of Practice for Official Statistics and must be robust, transparent and relevant.

All three surveys are currently conducted primarily face-to-face, although each was forced to adopt alternative approaches during the Covid-19 pandemic. However, starting pre-pandemic and accelerating since, greater consideration has been given to alternative designs for social surveys, with a number of large-scale government surveys in England, Wales and elsewhere moving away from purely face-to-face designs. Pressure on public sector budgets has been an important contributing factor to this trend, since, all other things being equal, face-to-face interviews are more expensive than interviews by other modes. Declining response rates across face-to-face surveys and increasing internet access making online approaches potentially more feasible have also been considerations.

The Scottish Government commissioned this research in order to explore the potential benefits and risks of mixed mode approaches (i.e. collecting data via another mode, such as online or telephone, or by a combination of modes) in the specific context of the three main Scottish Government general population surveys. The research is intended to provide information to inform the Scottish Government's next iteration of its Long Term Survey Strategy, which sets out the vision and plans for Scottish Government population surveys. It focuses particularly on the three cross-sectional surveys above – other key surveys, such as the longitudinal Growing up in Scotland survey, were not in scope for this research, and some of the findings may be less relevant given its longitudinal design. While the report synthesises evidence on the issues, risks and potential benefits of moving from a face-to-face to a mixed mode design to inform decision-making, it does not make any recommendations about what specific mode designs the Scottish Government should or should not consider for its surveys; decisions on this are for the Scottish Government itself.