Scottish Surveys Core Questions 2014

11 Technical Notes

This chapter provides additional background on the methodology and reporting conventions of the SSCQ and its constituent surveys:

• source surveys and core questions (section 11.1)
• weighting (section 11.2)
• confidence interval calculations (section 11.3)
• statistical disclosure control (section 11.4)
• presentation of data on country of birth (section 11.5), ethnic group (11.7), religion (11.6) and mental wellbeing (11.9)
• the age standardisation process (11.10) and statistical tests used in this analysis (11.11)

11.1 Source surveys and core questions

Results from the three large-scale Scottish Government population surveys are published separately as National Statistics:

• Scottish Crime and Justice Survey (SCJS)
  www.gov.scot/Topics/Statistics/Browse/Crime-Justice/crime-and-justice-survey
• Scottish Health Survey (SHeS)
  www.gov.scot/Topics/Statistics/Browse/Health/scottish-health-survey
• Scottish Household Survey (SHS)
  www.gov.scot/shs

Further information on Population Surveys in Scotland can be found here: www.gov.scot/Topics/Statistics/About/Surveys

Since the beginning of 2012 each of the surveys has included a set of 20 core questions that provide information on the composition, characteristics and attitudes of Scottish households and adults across a number of topic areas including equality characteristics, housing, employment and perceptions of health and crime. Responses on these questions from all three surveys have been pooled to provide the Scottish Surveys Core Questions (SSCQ) dataset with a sample size in excess of 20,000 responses.

Full details of the harmonised questions are available on the Scottish Government website ^[51] and questionnaires are provided on the websites of each of the individual surveys.

The first set of pooled response tables for the year 2012 were published as data under development here: www.gov.scot/Topics/Statistics/About/Surveys/PooledSample2012

Following further consultation and methodological development, the 2013 dataset was published as Official Statistics in December 2015. The website contains further information and supplementary tables to this main report.

www.gov.scot/Topics/Statistics/About/Surveys/SSCQ

The SSCQ 2014 dataset was pooled from the first and second quarter year of the Scottish Crime and Justice Survey 2014/15 and all four quarters each of the Scottish Health Survey 2014 and the Scottish Household Survey 2014. Responses from adults aged 16 and over were included.

Due to the different sampling nature of each srvey, which is necessary to meet their primary aims, the number of respondents varies between different SSCQ questions. The questions were hence batched into three groups: household questions, individual questions and crime questions, and three different sets of weights calculated to ensure representative results. Sampling, weighting and pooled sample numbers are described separetely for each survey below.

11.1.1 Scottish Crime and Justice Survey (SCJS) technical notes

Sampling, survey response and weighting are described in full in the SCJS2014/15 technical report: http://www.gov.scot/Topics/Statistics/Browse/Crime-Justice/crime-and-justice-survey/publications/scjs2014-15technicalreport

Briefly, the survey consists of a simple random sample, designed to achieve a minimum effective sample size of 750 interviews in the eight Police Force Areas (PFAs), enabling robust analysis at this level. One random adult per household was interviewed and asked all SSCQ and SCJS questions.

The response rate was 63.8%, resulting in 11,472 interviews during the 2014/15 financial year field work. Of these, 5695 interviews (completed in the second two quarters of the financial year) were pooled into the SSCQ 2014 dataset. The subset was re-weighted to be in itself representative of the adult Scottish population distribution, as described for the SCJS publication.

Roughly half of the SCJS sample is incorporated into the SSCQ. This enables a consistent sample size in each SSCQ publication for 2013-15, while the SCJS is produced biennially on a financial year basis. From 2016/17 onwards, the SCJS will be produced annually, which will enable data pooled on a calendar year basis.

11.1.2 Scottish Health Survey (SHeS) technical notes

Sampling, survey response and weighting are described in full in the SHeS 2014 technical report: www.gov.scot/Publications/2014/12/6634/downloads

The SHeS sample is clustered in each calendar year and unclustered over four years. All adults and up to two children in each household are eligible for interview. Only one adult in each household was asked the crime and household questions, to remain in line with the SCJS sampling procedure. The SHeS sample is boosted by participating health boards. It is further boosted to interview children in further households. These households were excluded from the SSCQ dataset as equality questions were not asked.

The response rate was 62.2%, and 3,011 households were interviewed in the main and health board boost samples. 4,659 resulting adult interviews were pooled into the SSCQ 2014 dataset. Of these, 3,005 were asked the crime questions. The subset of households (excluding the child boost), and adult respondents were re-weighted to be representative of the Scottish private household and population distribution, as described for the SHeS publication.

11.1.3 Scottish Household Survey (SHS) technical notes

Sampling, survey response and weighting are described in full in the SHS 2014 technical report: http://www.gov.scot/Topics/Statistics/16002/PublicationMethodology/Methodology14

The SHS consists of a simple random sample with a target minimum effective sample size of 250 per local authority. The SSCQ household questions are asked of the highest income householder or their spouse/partner, and one adult is randomly selected to answer the individual and crime questions, in line with the other two surveys.

67.0% of eligible households responded, leading to 10,634 household interviews. The response for the random adult interview was 61.7%, yielding 9,799 interviews. Weighting is fully described in the SHS technical report.

11.2 Weighting

Datasets from the three source surveys were combined into three new SSCQ datasets: SSCQ household variables (20,743 responses), SSCQ individual variables (20,153 responses) and SSCQ crime variables (19,395 responses), see Table 89.

Each variable response category in each of the surveys carries a different design effect. If we were solely seeking the most efficient estimate for each variable separately then separate scale factors could be derived for each one. However, this would restrict the use and understanding of the dataset. Rather, for each constituent survey dataset the design effects were estimated for each response category and then the median design effect over all response categories for all variables was used as the representative design effect of that survey. These design effects were then used along with the sample sizes to calculate the effective sample sizes (neff) and scaling factors for combining the three datasets.

Table 89: Numbers of sample and effective sample pooled from the source surveys

	SCJS		SHeS		SHS		SSCQ
	sample	neff	sample	neff	sample	neff	sample	neff
Household responses [52]	5,695	4,858	3,011	2,037	10,633	8,987	19,339	15,882
Individual responses [53]	5,695	4,019	4,659	2,375	9,799	6,832	20,153	13,226
Crime responses [54]	5,695	3,968	3,005	1,447	9,799	6,566	18,499	11,980

To combine the data the scale factors were applied to the grossing weights for the individual surveys (described in section 11.1). The neff of each survey contribution formed the basis for the scaling factors:

survey A weight scaling factor = neff (surveyA) / (sum of three survey neffs).

The weights were then re-scaled to be proportionate to effective sample size contribution of each survey and used as pre-weights.

The three pooled SSCQ datasets were then weighted again to be representative of National Records of Scotland population estimates ^[55] .

11.3 Confidence Interval Calculations

All three of the source surveys are stratified to ensure sufficient sample sizes in the smaller local authority areas. In addition, SHeS is clustered in each annual fieldwork period and, while this effect cancels out over each four-year period, it must be accounted for in producing annual results.

Confidence intervals have therefore been calculated using a method to account for stratification and clustering and the resulting design effects (surveyfreq in SAS). This method is used to compare estimates of all quantities provided by SSCQ. Confidence intervals across all subgroup estimates are provided in the accompanying supplementary tables. ^[56]

Confidence intervals are plotted on all charts and figures in this report. If the intervals do not overlap then there is a significant difference between two points, but if they do overlap it does not necessarily mean there is no significant difference. ^[57] In the report text the term "significant" refers to "statistically significant" differences.

A comparison of estimates of key variables across the three constituent surveys and the SSCQ are provided in Annex B.

11.4 Statistical Disclosure Control

All estimates based on a single respondent and displayed in main and supplementary tables have been denoted with '*' to safeguard the confidentiality of respondents with rare characteristics.

For individual variables crossed with individual variables (e.g. Ethnic group by Religion), further cells with zero or low respondent numbers in the same row and column as the single response have also been suppressed with '*' to ensure confidentiality. For household and geographic variables, only one further cell in the same row was suppressed, as these cross-tabulations are not transposed.

Cells with true zero counts are denoted with '.' throughout, unless denoted '*' as part of disclosure control.

11.5 Presentation of Data on Country of Birth

The country of birth of respondents is collected in detail by the pooled surveys. Reporting on the data in this way would uniquely identify, and therefore potentially disclose the identity of, participants from a range of countries. For this reason, country of birth has been presented as follows:

• Scotland: Respondents who specifically list "Scotland" as their country of birth
• Rest of UK: England, Northern Ireland, Wales, Great Britain/United Kingdom (Not Otherwise Specified)
• Rest of EU: Austria, Belgium, Bulgaria, Croatia, Cyprus (European Union), Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Latvia, , Lithuania, Luxembourg, Malta, Netherlands, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden
• Rest of World: All other responses (excluding refusals)

11.6 Presentation of Data on Religion

Table 90: Grouping of religion in the SSCQ 2014

Base Collection Categories	Sample	Super Groups	Sample
None	8413	None	8413
Church of Scotland	6539	Church of Scotland	6539
Roman Catholic	2861	Roman Catholic	2861
Other Christian	1670	Other Christian	1670
Muslim	191	Muslim	191
Buddhist	56	Other	350
Sikh	24
Jewish	39
Hindu	63
Pagan	24
Another religion	144

11.7 Presentation of Data on Ethnic Group

Table 91: Grouping of ethnic group in the SSCQ 2014

Base Collection Categories	Sample	Super Groups	Sample
A - WHITE - White Scottish	15918	White: Scottish	15918
A - WHITE - Other British	2744	White: Other British	2744
A - WHITE - Polish	262	White: Polish	262
A - WHITE - Irish	148	White: Other	665
A - WHITE - Gypsy/Traveller	3
A - WHITE - Any other white ethnic group	514
C - ASIAN, ASIAN SCOTTISH OR ASIAN BRITISH - Pakistani, Pakistani Scottish or Pakistani British	109	Asian	340
C - ASIAN, ASIAN SCOTTISH OR ASIAN BRITISH - Indian, Indian Scottish or Indian British	94
C - ASIAN, ASIAN SCOTTISH OR ASIAN BRITISH - Bangladeshi, Bangladeshi Scottish or Bangladeshi British	23
C - ASIAN, ASIAN SCOTTISH OR ASIAN BRITISH - Chinese, Chinese Scottish or Chinese British	68
C - ASIAN, ASIAN SCOTTISH OR ASIAN BRITISH - Other Asian, "Asian" Scottish or "Asian" British	46
B - MIXED OR MULTIPLE ETHNIC GROUP - Any mixed or multiple ethnic groups	28	All other ethnic groups	206
D - AFRICAN - African, African Scottish or African British	63
D - AFRICAN - Other African background	26
E - CARIBBEAN OR BLACK - Caribbean, Caribbean Scottish or Caribbean British	12
E - CARIBBEAN OR BLACK - Black, Black Scottish or Black British	4
E - CARIBBEAN OR BLACK - Other Caribbean or Black background	2
F - OTHER ETHNIC GROUP - Arab, Arab Scottish or Arab British	14
F - OTHER ETHNIC GROUP - Other	57

11.8 Provision of unpaid care

Since 2012 there have been some changes to the collection of data about the provision of unpaid caring. The wording of the question was altered in quarter 2 of the 2014 collection period to the following:

Apart from anything you do as part of paid employment, do you look after, or give any regular help or support to family members, friends, neighbours or others because of either long-term physical, mental ill-health, disability; or problems related to old age?

In the Scottish Household Survey, this question was also moved from being asked of the household reference person in the household survey to the being asked of the random adult.

This change part way through the collection period has different effects depending on the source survey.

• The SCJS runs from Q2 2014 to Q1 2015. This methodological change therefore has no effect on SCJS data. The SCJS question wording was harmonised before fieldwork began.
• In the SHS and SHeS the harmonised question was not asked until quarter 2; around one quarter of responses are missing as a result of this change.

For this reason the sample size for the care indicator is somewhat smaller at 16,518 cases (compared to the sample of 20,153 for most individual-level questions).

To counteract any additional bias as a result of this loss of sample, a specific weight for this question has been calculated and is used throughout chapter 1.5.

11.9 SWEMWBS Scoring

Peaks at multiples of seven in Figure 28 are produced by column effects, where respondents are more likely to place answers down a column giving the same response for each question.

Figure 28: SWEMWBS unconverted and unweighted response totals, SSCQ 2014

Unlike the full WEMWBS metric, SWEMWBS scores undergo a metric conversion to correct somewhat for this effect and produce a distribution that is closer to normal. This conversion follows the schema in Table 92. After transformation, the distribution of scores is approximately normal and the boundary effect at the scale maximum of 35 is reduced, as shown in Figure 29 and Figure 30.

Table 92: SWEMWBS Scoring - Metric Conversion ^[58]

Raw Score	Metric Score	Raw Score	Metric Score	Raw Score	Metric Score
7	7	17	16.88	27	24.11
8	9.51	18	17.43	28	25.03
9	11.25	19	17.98	29	26.02
10	12.4	20	18.59	30	27.03
11	13.33	21	19.25	31	28.13
12	14.08	22	19.98	32	29.31
13	14.75	23	20.73	33	30.7
14	15.32	24	21.54	34	32.55
15	15.84	25	22.35	35	35
16	16.36	26	23.21

Figure 29: Unweighted distribution of the sum of SWEMWBS question scores

Figure 30: Unweighted distribution of SWEMWBS scores after metric conversion

11.10 Age Standardisation

Age standardisation has been used in order to enable groups to be compared after adjusting for the effects of any differences in their age and sex distributions.

When different sub-groups are compared in respect of a variable on which age has an important influence, any differences in age distributions between these sub-groups are likely to affect the observed differences in the proportions of interest.

Age standardisation was carried out, using the direct standardisation method. The standard population to which the age distribution of sub-groups was adjusted was the mid-2014 population estimates for Scotland. All age standardisation has been undertaken separately within each sex.

The age-standardised proportion p' was calculated as follows, where pi is the age specific proportion in age group i and Ni is the standard population size in age group i:

Therefore p' can be viewed as a weighted mean of p _i using the weights N _i.

Age standardisation was carried out using the age groups: 16-24, 25-34, 35-44, 45-54, 55-64, 65-74 and 75 and over broken down by sex.

The variance of the standardised proportion can be estimated by:

The populations used for age standardisation are the same as those used for weighting. See the associated Weighting Base tables for details. ^[59]

11.11 Statistical Tests

Statistical tests are used throughout this publication to determine where apparent differences are statistically significant.

For most indicators the variable type is binary, i.e. each case is either a "yes" or "no" with respect to the indicator text at the beginning of each chapter. For that reason a logistic regression model is used to determine whether differences between subgroup categories are statistically significant. Testing is relative to a reference category which is always the most populated subgroup in the domain. This is performed using proc surveylogistic in SAS to account for the complex design of SSCQ.

A similar technique is used to determine changes over time. Data year is coded as a continuous integer variable.

• Change "from 2013" excludes data from 2012 and regresses year against the indicator variable overall or within subgroup domains or geographical areas.
• Change "from 2012" retains all data years (i.e. not testing 2012 against 2014) and indicates whether a trend exists over the longer time base.

The adjusted chi-squared statistics and odds ratio confidence limits are examined to determine whether a change over time is statistically significant. The requirement for 95% confidence requires p-values to be below 0.05. Odds ratio confidence intervals, which indicate the strength of the signal, are required to exclude the value of 1 (either to lie above or below equal odds) with the same 95% confidence bounds.

In the few cases where these two indicators disagree (i.e. where the odds ratio interval includes the value of 1 but the p-value is below 0.05, or p-value exceed 0.05 but the signal is strong) are taken not to be statistically significant.

SWEMWBS is the only continuous indicator variable in SSCQ. A regressions analysis is implemented using SAS proc surveyreg to account for the complex survey design. Testing is relative to a reference category which is always the most populated subgroup in the domain.

Between subnational geographies, any formal testing is produced using contrasts to compare the area in question with the combined total of all other areas. An example of the contrast matrix for health board is provided in Table 93.

Table 93: Contrast matrix for testing health board areas against residual national average

'Ayrshire & Arran'	0.91	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07
'Borders'	-0.07	0.91	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07
'Dumfries and Galloway'	-0.07	-0.07	0.91	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07
'Fife'	-0.07	-0.07	-0.07	0.91	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07
'Forth Valley'	-0.07	-0.07	-0.07	-0.07	0.91	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07
'Grampian'	-0.07	-0.07	-0.07	-0.07	-0.07	0.91	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07
'Greater Glasgow & Clyde'	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	0.91	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07
'Highland'	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	0.91	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07
'Lanarkshire'	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	0.91	-0.07	-0.07	-0.07	-0.07	-0.07
'Lothian'	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	0.91	-0.07	-0.07	-0.07	-0.07
'Orkney'	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	0.91	-0.07	-0.07	-0.07
'Shetland'	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	0.91	-0.07	-0.07
'Tayside'	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	0.91	-0.07
'Western Isles'	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	-0.07	0.91