Scotland's People Annual Report: Results from 2012 Scottish Household Survey

1 Background to the survey

Introduction

The Scottish Household Survey (SHS) is a continuous survey based on a sample of the general population in private residences in Scotland. The survey started in 1999 and up to 2011 followed a fairly consistent survey design. From 2012 onwards, the survey was substantially redesigned to include elements of the Scottish House Condition Survey^[1] (SHCS) including the follow-up Physical Survey component. The survey is now run through a consortium led by Ipsos MORI.

The SHS is designed to provide reliable and up-to-date information on the composition, characteristics, attitudes and behaviour of Scottish households and individuals, both nationally and at a sub-national level and to examine the physical condition of Scotland's homes. It covers a wide range of topics to allow links to be made between different policy areas. The specific aims of the survey are:

• Meet central and local Government needs for priority policy relevant data across a broad range of topics (including needs for continuing time-series of data collected by the SHS and SHCS previously);
• Be understandable and useful to stakeholders and so lead to a high level of buy-in and use of the SHS;
• Have built in flexibility to respond to different data needs regarding geography and frequency (e.g. to provide some data annually at Local Authority level, and some biennially at national level), and changes to these requirements over time;
• Align with other surveys and data vehicles (in particular the Scottish Health Survey and Scottish Crime and Justice Survey);
• Produce high quality data in accordance with the Code of Practice for Official Statistics^[2] so as to provide data that is suitable for the production of National Statistics publications in a cost effective way.;
• To permit disaggregation of information both geographically and in terms of population sub-groups (such as families with children or households in the social rented sector);
• To allow the relationships between social variables within households to be examined. This will support cross-analysis on a range of issues;
• To allow detailed follow-up surveys of sub-samples from the main survey sample, if required.

The Annual Report

SHS results have been reported in a series of Annual Reports between 1999 and 2011. The annual report is designed to act as an introduction to the survey and to present and interpret some of the key policy-relevant results. Findings from the Physical Survey component and other house condition information will be published through a separate SHCS 2012 Main Findings publication scheduled for release in December 2013.

A comprehensive range of web tables to accompany this report is provided on the SHS website.^[3] A series of Local Authority Tables are also available via the SHS website. Similar information is also published through the SHS Interactive Mapping tool which presents time-series analysis from the survey for key measures across all local authorities in Scotland^[4]. The user is able to select the measure of interest including ones such as neighbourhood rating, car availability and provision of unpaid care. All relevant analysis for the measure selected is displayed as a time-series in an accompanying chart along with thematic map showing differences in estimates between all local authorities. Relevant metadata is also built in to the system to offer contextual information, as well as additional notes on the questions or variables used in the survey.

Whilst this release focuses on a number of key results, the SHS Project Team can be contacted with any additional analysis requests or enquiries.^[5]

Structure of the Annual Report

At the start of each chapter introductory paragraphs draw on key policy documents to set the results that follow into the policy context for the topic it covers. In most of the chapters, the introduction draws on the Scottish Budget Spending Review 2007.^[6] This document highlights the current Government's overall purpose of increasing sustainable economic growth, and five strategic objectives, which are designed to ensure the purpose is delivered - as defined by the National Performance Framework. The framework was updated in December 2011. The objectives that are most relevant to the subject of a chapter, as defined in the spending review, are identified. The five objectives are:

Wealthier and Fairer - Enable businesses and people to increase their wealth and more people to share fairly in that wealth.

Smarter - Expand opportunities for Scots to succeed from nurture through to life long learning ensuring higher and more widely shared achievements.

Healthier - Help people to sustain and improve their health, especially in disadvantaged communities, ensuring better, local and faster access to health care.

Safer and Stronger - Help local communities to flourish, becoming stronger, safer place to live, offering improved opportunities and a better quality of life.

Greener - Improve Scotland's natural and built environment and the sustainable use and enjoyment of it.

Additional policy documents, including more detailed strategies on particular policy areas, are drawn on as appropriate and are referenced in the text.

The SHS is the source of information on eight of the 50 national indicators in the Government's National Performance Framework.^[7] The two transport indicators^[8] will be reported on separately by Transport Scotland within their Transport and Travel in Scotland, 2012 report^[9] and in the SHS Travel Diary 2012 Results publication^[10]. This report provides estimates for the remaining six national indicators:

Improve people's perceptions of their neighbourhood (Chapter 4);

Widen use of the Internet (Chapter 9);

Reduce the percentage of the adult population who smoke (Chapter 10);

Improve people's perceptions of the quality of public services (Chapter 11);

Improve the responsiveness of public services (Chapter 11);

Increase cultural engagement (Chapter 13).

The results are presented in the 12 main chapters covering: household composition; housing; neighbourhoods and communities; economic activity; finance; education; transport; internet; health and caring; local services; volunteering; culture and sport.

Guidance on using the information in the report and a glossary with detailed definitions of some of the key terms are included as annexes. Additional annexes present results on the main classificatory variables used in this report and provide guidance on assessing confidence intervals and the statistical significance of the results.

Additional SHS Reporting

Full details of the survey will also be made available through the companion Technical Reports. There are two parts to the technical reporting detailing the methodology and fieldwork outcomes^[11] and, separately, the questionnaire^[12] used.

A number of other Scottish Government publications covering previous years are also available. A comprehensive listing of all publications is available from the SHS website.^[13]

Comparability with other sources

In some cases the SHS is not the official source of statistics on a particular topic: such as income, employment or housing. The interview collects information on these topics to select the data of particular groups for further analysis or for use as background variables when analysing other topics. The results are included in order to set the context for, and aid interpretation of, the remaining chapters. Where results are not the official source, this is indicated in the chapter introduction.

The Scottish Government conducts several major population surveys that are used to inform the policy debate in Scotland, and in some instances the surveys can be complimentary. The Long Term Strategy for Population Surveys in Scotland 2009-2019, of which the SHS is a central element, is designed to improve the way population surveys are run and to increase the availability and use of survey data, both at a national and local level. A topic guide is available through this workstream setting out an inventory of sources.^[14]

There are also a number of GB or UK surveys that include a Scottish dimension. The Integrated Household Survey^[15] (IHS) is a composite survey combining questions asked in a number of Office for National Statistics GB-wide social surveys. The IHS is currently designated as "experimental statistics" so while the results should be considered with some care, in some instances the IHS may be particularly useful for making cross-GB comparisons. Please contact the Survey Methodology and Co-ordination team (0131 244 3339) if you have any queries.

Survey Design

As noted earlier, from January 2012 a new SHS went in to the field which had a substantially restructured sample design and integrated the previous SHCS. The new survey uses a fully unclustered core and modular structure with some questions being asked of the full sample and others of a one-third sub-sample. The overall sample size has reduced from around 14,000 household interviews to about 11,000 though improvements in efficiency of the survey design mean it will be possible to attain local authority estimates on an annual basis where analysis permits. While the overall sample size of the survey has reduced, the survey design improvements has meant that the precision of estimates have not been affected significantly.

Diagram 1.1 provides a visual representation of how the core and modular design is structured within each year and how this rotates and replicates across to subsequent years also. This includes a "core" set of 20 questions which have been designed to be asked in consistent ways with other surveys, such as age and gender. The subsequent "modules" of questions have been designed to be flexible in terms of topic, frequency and geography. For example, questions asked of the "full" sample and asked on an "annual" basis would be able to provide local authority level on an annual basis. Similarly, questions might only be asked of "1/3" of the sample on a "biennial" basis (i.e. asked every second year). Such questions could only get national level estimates.

Diagram 1.1: Representation of multi-year core and modular design

The survey questionnaire itself is structured in three main parts.

The household reference person, who is the Highest Income Householder (HIH) or their spouse/partner completes part 1 of the interview ('Household'). Details of all members of the household, including children, are collected during the household interview. This includes questions related to the composition and characteristics of the household, and involves capturing basic demographic information of all members of the household, such as gender, age and economic situation at this stage, as well detailed information on dwelling characteristics as captured through the old SHCS. The topics covered in the Household section of the survey are presented in Diagram 1.2.

Diagram 1.2: Topics covered in SHS 2012 Household component

Household Composition	People living in household, basic demographics
Accommodation	Tenure, Property type, Number of rooms
Household Services	Number of bedrooms, Internet access, Food waste/recycling
Driving and Transport	Cars, Fuel spend, Bicycles
Young People	Schools and travel, Play, Safety, Activities
Health and Disability	Disability and type, Caring, Noise
Housing	Aspirations, Repairs, Satisfaction, Water supply
Heating and Energy	Room types, Heating controls, Regimes, Costs, Suitability, Resilience in emergencies, Types, Smoke alarms
Condensation and Damp	Problems
Housing and Health	Adaptations, Services
Household Employment	Householder details
Household Income	Householder/Spouse paid/self-employed/other jobs, Benefits, Other sources
Household Finances	Bank, Savings, Credits, Insurance, Managing financially
Mortgages and Rent	Initial buy, Current, Service charge, Rent costs

Subsequently a child is selected from all household members under 16 (the 'Random Child') and the household respondent is asked questions about childcare for that child. A child who is at school is also selected (the 'Random School Child')^[16] and the household respondent answers questions about the school that child attends and the journey they make to go there. Once the composition of the household has been established, one of the adults in the household is randomly selected by the computer to complete part 2 ('Random Adult').^[17] This covers the behavioural and attitudinal type questions, such as satisfaction with local services, and captures further demographic information on the random adult. This element also covers the "Travel Diary" component which asks about travel behaviours on the day previous to that of the interview day. In all households with a single adult the same person completes both parts, but as the number of adults in the household increases, the probability of the random adult being the same as the household respondent declines.^[18] The topics covered in the Random Adult section of the survey are presented in Diagram 1.3.

Diagram 1.3: Topics covered in SHS 2012 Random Adult component

Adult Characteristics	Demographics, Country of birth and date of entry
Accommodation	Current/previous tenure, Homelessness
Neighbourhoods and Communities	Rating, Belonging, Police, Greenspace, Anti-social Behaviour, Feeling safe, Discrimination, Neighbours
Education and Training	Education
Internet	Use, Methods, Public sector, Non-users
Travel and Transport	Licence, Car dependency, Park and rides, Travel to work/education, Congestion, Car Sharing, Company car, Air travel, Bicycle, Walking, Buses, Trains, Mode changes, Ferry, Crime on public transport, Young Scot card, Journey planning, Accidents, Travel Diary
Services	Council, Public, Cost, Convenience, Outdoors use
Culture and Sport	Attendance and participation in Culture, Participation in Sport, Olympics
Health and Disability	General health, Concessionary travel, Smoking now/previously
Adult Employment	Economic activity

At the end of the Household component of the survey, the HIH is asked if they would be willing to have the follow-up component "Physical Survey" of the dwelling arranged. Such surveys are conducted by professional surveyors through a visual inspection of the dwelling. The surveyor will assess the condition, design and energy efficiency of the home, with much of their time spent surveying the outside, but they will ask to see all the rooms inside. Results from the Physical Survey will be reported on separately later in the year.

Further information on the SHS Questionnaire can be found via the relevant technical report on the SHS website.^[19]

Sampling

The sample for the SHS 2012 was designed by the Scottish Government. The sample design was coordinated with the sample designs for the Scottish Health Survey (SHeS) and the Scottish Crime and Justice Survey (SCJS) as part of a survey efficiency project and to allow the samples of the three surveys to be pooled for further analysis.^[20]

The sample for the survey meets a number of criteria. It is designed to provide nationally representative samples of private households and of the adult population in private households. This is achieved by splitting the interview between a household respondent and an adult selected at random from the permanent residents of the household.

The SHS sample has been designed to allow annual publication of results at Scotland level and for local authorities. To meet these requirements the target sample size for Scotland was 10,678 household interviews with a minimum local authority target of 258 (West Lothian). From 2012 onwards the physical survey of the Scottish House Condition Survey (SHCS) has been incorporated into the SHS. A subsample of the main sample has been allocated to the physical survey, which has a required sample size of 3,004 for Scotland and a minimum of 80 for each local authority.

The sample design, like the one used from 1999 onwards, uses a multi-stage stratified design though from 2012 moved to a fully unclustered design. In order to provide annual local authority results without specifying an excessive overall sample size, the sample was disproportionately stratified (smaller local authorities have a higher sample proportion relative to their populations than the larger local authorities). To deliver the required local authority precision the minimum effective sample size for each local authority was set at 250. For local authorities where an effective size sample of 250 would have decreased estimate precision by more than 25% from the previous sweep of the survey the target effective sample size was increased such that the decrease in precision was less than 25%.

The SHS sample is selected from the small user Postcode Address File (PAF) for Scotland, expanded to take account of addresses which might only be listed once but actually contain multiple dwellings, such as tenement blocks and multi-storey flats. Although the small user PAF excludes many institutional addresses such as student halls of residence or nurses' homes, there are no geographical exclusions from the survey, which covers all parts of Scotland, including the Highlands and Islands.

The main features of the design are:

• First stage, disproportionate stratification by local authority;
• Within each local authority, second stage systematic random sampling was used to select the addresses from the sample frame with the addresses ordered by urban-rural classification, Scottish Index of Multiple Deprivation (SIMD) rank and postcode;
• Once the overall sample was selected systematic random sampling was used to select the subsample for the physical survey.

As the samples for the SHS, SHeS and SCJS are all being selected by the Scottish Government from 2012 onwards, addresses selected for any of the surveys are removed from the sample frame so that they cannot be re-sampled for another survey. This will help to reduce respondent burden and facilitate the development of the pooled sample. The addresses are removed from the sample frame for a minimum of 4 years.

Response Rates

Survey response is an important indicator of survey quality as non-response can introduce bias into survey estimates. After excluding addresses that were outwith the scope of the survey^[21], the overall response rate for this sweep of the survey was 67.2% (10,644 achieved sample). This is slightly below the long-term (1999 to 2011) average response rate for the SHS of 67.9%. However, it should be noted that the calculation has changed slightly for 2012 as a portion of the addresses of unknown eligibility are considered to be eligible (addresses of unknown eligibility have been allocated as eligible and ineligible proportional to the levels of eligibility for the remainder of the sample) whereas previously they would all have been classed as ineligible.

There was significant variation in response between local authorities. Four local authorities had response rates below 60% in 2012 (East Renfrewshire, Aberdeen City, Glasgow City and Midlothian). The level for Glasgow is consistent with the long-term average response rate for the local authority of approximately 60%. The other three local authorities had long-term average response rates of over 64%. While variations in response rates from year-to-year are expected for local authorities due to relatively small sample sizes, these particular areas will be closely monitored in future SHS sweeps.

The conversion from household interview to random adult completion was almost 93% for 2012, this was an increase from 90% in 2011. However, it should be noted that two of the local authorities (Midlothian and East Renfrewshire) which had household response rates below 60% also had random adult conversion rates of below 90%. This double effect led to a relatively low number of completed random adult interviews in East Renfrewshire and Midlothian.

Further information on response rates and other such information is available in the accompanying SHS 2012 Methodology and Fieldwork Outcomes report^[22].

Weighting

Like the sampling, the weighting was undertaken by the Scottish Government rather than the survey contractor (as had previously been the case), but the methodology applied was largely consistent with that from previous sweeps of the survey. The procedures for the implementation of the weighting methodology were developed by the Scottish Government working with the Methodology Advisory Service at the Office for National Statistics.^[23]

Weighting procedures for survey data are required to correct for unequal probabilities of selection and variations in response rates from different groups. The weighting procedures for the SHS incorporate a selection weighting stage to address the unequal selection probabilities and calibration weighting to correct for non-response bias. Calibration weighting derives weights such that the weighted survey totals match known population totals. For the 2012 SHS the population totals used were the National Records of Scotland's (NRS) "Mid-2011 Population Estimates Scotland" and for households the NRS "Estimates of Households and Dwellings in Scotland, 2011" were used.^[24] To undertake the calibration weighting the ReGenesees Package for R was used and within this to execute the calibration a linear distance function was implemented.

Three weights were derived for the main section of the 2012 SHS; a household weight; random adult weight; and a random schoolchild weight. Further weights were required for analysis of the travel diary and physical survey sections which are not covered in this report.

The household weights were derived in three steps. The address selection weights were calculated to compensate for unequal probabilities of selection of addresses in different survey strata. For the SHS there were 32 strata - one for each local authority. The stratum selection weights were then used as entry weights for a calibration stage, so that the weighted total of all members of responding households matched NRS population totals for age bands and sex within each local authority. A final household adjustment was then applied so that the weighted number of households from the sample matched the NRS local authority household estimates.

A similar approach was taken to derive the random adult weights. Again, a stratum selection weight was created based on comparing the percentage of responding random adults to the percentage of adults within each stratum (local authority), this time using NRS mid-year population estimates. The probability that of an adult within a household being selected for the random adult interview was inversely proportional to the number of adults within a household - i.e. in a single adult household the only adult resident must be sampled but in a three adult household each adult only has a one in three chance of being selected. To correct for this unequal probability of selection an adult selection weight equal to the number of adults in the household was applied. Finally, the weighted total of responding random adults were adjusted to match the NRS population totals for age bands and sex within each local authority.

The random child and the random school child are dealt with in a similar way - weighting to correct for disproportionate sampling is undertaken and then residual weights, aligning the sample profile with official estimates, are calculated. The only exception to this is that in the case of the random school child, there are no official estimates of the age/sex profile of school children within each local authority. Therefore, the population of schoolchildren was estimated using the survey data by applying the household grossing weight to calculate the total number of pupils in each local authority by age group. The selection weights were then combined and applied to the data before the calibration was run to match the random schoolchild totals to the target populations by age group and local authority.