Scottish Social Attitudes Survey 2014: Public Attitudes to Sectarianism in Scotland

Annex B -Technical Details of the Survey

The Scottish Social Attitudes series

1. The Scottish Social Attitudes (SSA) survey was launched by ScotCen Social Research in 1999, following the advent of devolution. Based on annual rounds of interviews of between 1,200 to 1,500 people drawn using probability sampling (based on a stratified, clustered sample)^[27], it aims to facilitate the study of public opinion and inform the development of public policy in Scotland. In this it has similar objectives to the British Social Attitudes (BSA) survey, which was launched by ScotCen's parent organisation, NatCen Social Research in 1983. While BSA interviews people in Scotland, these are usually too few in any one year to permit separate analysis of public opinion in Scotland (see http://www.bsa-31.natcen.ac.uk/ for more details of the BSA survey).

2. SSA has been conducted annually each year since 1999, with the exception of 2008. The survey has a modular structure. In any one year it typically contains four or five modules, each containing 40 questions. Funding for its first two years came from the Economic and Social Research Council, while from 2001 onwards different bodies have funded individual modules each year. These bodies have included the Economic and Social Research Council (ESRC), the Scottish Government, the Equality and Human Rights Commission, and various charitable and grant awarding bodies, such as the Nuffield Foundation and Leverhulme Trust.

The 2014 survey

3. The 2014 survey contained modules of questions on:

• Dementia - funded by the Life Changes Trust and Joseph Rowntree Foundation.
• Sectarianism - funded by the Scottish Government
• Violence Against Women - funded by the Scottish Government
• Attitudes to policing - funded by ScotCen and the Scottish Institute for Policing Research
• Scottish independence - funded by the ESRC and Edinburgh University.

4. Findings from the modules funded by the Scottish Government will be available in reports published on their website (www.scotland.gov.uk). Separate programmes of dissemination are planned for each of the other modules. This technical annex covers the methodological details of the survey as well as further discussion of the analysis techniques used in this report.

Sample design

5. The survey is designed to yield a representative sample of adults aged 18 or over, living in Scotland. The sample frame is the Postcode Address File (PAF), a list of postal delivery points compiled by the Post Office. The detailed procedure for selecting the 2014 sample was as follows:

i. 104 postcode sectors were selected from a list of all postal sectors in Scotland, with probability proportional to the number of addresses in each sector for addresses in urban areas and a probability of twice the address count for sectors in rural areas (i.e. the last 3 categories in the Scottish Government's 6 fold urban-rural classification). Prior to selection the sectors were stratified by Scottish Government urban-rural classification^[28], region and percentage of household heads recorded as being in non-manual occupations (SEG 1-6 and 13, taken from the 2011 Census).

ii. 30 addresses were selected at random from each of these 104 postcode sectors

iii. Interviewers called at each selected address and identified its eligibility for the survey. Where more than one dwelling unit was present at an address, all dwelling units were listed systematically and one was selected at random using a computer generated random selection table. In all eligible dwelling units with more than one adult aged 18 or over, interviewers had to carry out a random selection of one adult using a similar procedure.

Response rates

6. The Scottish Social Attitudes survey involves a face-to-face interview with respondents and a self-completion section (completed using Computer Assisted Personal Interviewing). The numbers completing each stage in 2014 are shown in Table 1.

Table B.1: 2014 Scottish Social Attitudes survey response

	No.	% of 'eligible' (in scope) sample
Addresses issued	3,120
Vacant, derelict and other out of scope[1]	341	11
Achievable or 'in scope'	2779
Unknown eligibility[2]	21	1
Interview achieved	1,501	54
Self-completion completed	1,427	51
Interview not achieved
Refused[3]	883	32
Non-contact[4]	185	7
Other non-response[5]	168	6

Notes to table

1 This includes empty / derelict addresses, holiday homes, businesses and institutions, and addresses that had been demolished.
2 'Unknown eligibility' includes cases where the address could not be located, where it could not be determined if an address was residential and where it could not be determined if an address was occupied or not.
3 Refusals include: refusals prior to selection of an individual; refusals to the office; refusal by the selected person; 'proxy' refusals made by someone on behalf of the respondent; and broken appointments after which a respondent could not be re-contacted.
4 Non-contacts comprise households where no one was contacted after at least 6 calls and those where the selected person could not be contacted.
5 'Other non-response' includes people who were ill at home or in hospital during the survey period, people who were physically or mentally unable to participate and people in which a language barrier made recruitment too difficult (despite translation and interpreting services being offered).

Sample size for previous years

7. The table below shows the achieved sample size for the full SSA sample (all respondents) for all previous years.

Table B.2: Scottish Social Attitudes survey sample size by year

Survey year	Achieved sample size
1999	1482
2000	1663
2001	1605
2002	1665
2003	1508
2004	1637
2005	1549
2006	1594
2007	1508
2009	1482
2010	1495
2011	1197
2012	1229
2013	1497
2014	1501

Weighting

8. All percentages cited in this report are based on weighted data. The weights applied to the SSA 2014 data are intended to correct for three potential sources of bias in the sample:

• Differential selection probabilities
• Deliberate over-sampling of rural areas
• Non-response

9. Data were weighted to take account of the fact that not all households or individuals have the same probability of selection for the survey. For example, adults living in large households have a lower selection probability than adults who live alone. Weighting was also used to correct the over-sampling of rural addresses. Differences between responding and non-responding households were taken into account using information from the census about the area of the address as well as interviewer observations about participating and non-participating addresses. Finally, the weights were adjusted to ensure that the weighted data matched the age-sex profile of the Scottish population (based on 2013 mid-year estimates from the General Register Office for Scotland).

Fieldwork

10. Fieldwork for the 2014 survey ran between May and August 2014, with 83% of interviews completed by the end of June and 93% by the end of July. An advance postcard, followed by an advance letter, was sent to all sampled addresses and followed up by a personal visit from a ScotCen interviewer. Interviewers were required to make a minimum of 6 calls at different times of the day (including at least one evening and one weekend call) in order to try and contact respondents. All interviewers attended a one day briefing conference prior to starting work on the study.

11. Interviews were conducted using face-to-face computer-assisted interviewing (a process which involves the use of a laptop computer, with questions appearing on screen and interviewers directly entering respondents' answers into the computer). All respondents were asked to fill in a self-completion questionnaire using the interviewer's laptop. If the respondent preferred, the questions could be read out by the interviewer. Table 1 (above) summarises the response rate and the numbers completing the self-completion section in 2014.

Analysis variables

12. Most of the analysis variables are taken directly from the questionnaire and are self-explanatory.

Scottish Index of Multiple Deprivation (SIMD)

13. The Scottish Index of Multiple Deprivation (SIMD)^[29] 2009 measures the level of deprivation across Scotland - from the least deprived to the most deprived areas. It is based on 38 indicators in seven domains of: income, employment, health, education skills and training, housing, geographic access and crime. SIMD 2009 is presented at data zone level, enabling small pockets of deprivation to be identified. The data zones are ranked from most deprived (1) to least deprived (6,505) on the overall SIMD 2009 and on each of the individual domains. The result is a comprehensive picture of relative area deprivation across Scotland.

14. The analysis in this report used a variable created from SIMD data indicating the level of deprivation of the data zone in which the respondent lived in quintiles, from most to least deprived.^[30]

Region of Scotland

15. For the purpose of analysis, Scotland was split into four regions, north, east, west and south, although in some cases analysis is only presented for the west of Scotland and the rest of Scotland. The four regions were defined as follows:

• North - Aberdeen City, Aberdeenshire, Argyll and Bute, Highland, Moray, Orkney, Shetland, and the Western Isles;
• East - Angus, Clackmannanshire, Dundee, East Lothian, Edinburgh, Falkirk, Fife, Midlothian, Perth and Kinross, Stirling, and West Lothian;
• West - East Ayrshire, East Dunbartonshire, East Renfrewshire, Glasgow, Inverclyde, North Ayrshire, North Lanarkshire, Renfrewshire, South Ayrshire, and South Lanarkshire;
• South - Borders, and Dumfries and Galloway.

Analysis techniques

Significance testing

16. Where this report discusses differences between two percentages (either across time, or between two different groups of people within a single year), this difference is significant at the 95% level or above, unless otherwise stated. Differences between two years were tested using standard z-tests, taking account of complex standard errors arising from the sample design. Differences between groups within a given year were tested using logistic regression analysis, which shows the factors and categories that are significantly (and independently) related to the dependent variable (see below for further detail). This analysis was done in PASW 18, using the CS logistic function to take account of the sample design in calculations.

Regression analysis

17. Regression analysis aims to summarise the relationship between a 'dependent' variable and one or more 'independent' explanatory variables. It shows how well we can estimate a respondent's score on the dependent variable from knowledge of their scores on the independent variables. This technique takes into account relationships between the different independent variables (for example, between education and income, or social class and housing tenure). Regression is often undertaken to support a claim that the phenomena measured by the independent variables cause the phenomenon measured by the dependent variable. However, the causal ordering, if any, between the variables cannot be verified or falsified by the technique. Causality can only be inferred through special experimental designs or through assumptions made by the analyst.

18. All regression analysis assumes that the relationship between the dependent and each of the independent variables takes a particular form. This report was informed by logistic regression analysis - a method that summarises the relationship between a binary 'dependent' variable (one that takes the values '0' or '1') and one or more 'independent' explanatory variables. The tables in this annex show how the odds ratios for each category in significant explanatory variables compare to the odds ratio for the reference category (always taken to be 1.00).

19. Taking the model shown in Table B.5 (below), the dependent variable is agreeing that 'I am more comfortable around people with similar religious beliefs to my own' - or for those with no religion - 'I am more comfortable around people with no religious beliefs'. If the respondent either agreed or strongly agreed, the dependent variable takes a value of 1. If not, it takes a value of 0. If the respondent didn't know, refused to answer, or did not complete the self-completion, they were omitted from the analysis. An odds ratio of above 1 means that, compared with respondents in the reference category, respondents in that category have higher odds of agreeing that they are more comfortable around people with similar religious beliefs to themselves (or no religious beliefs if they have none). Conversely, an odds ratio of below 1 means they have lower odds of saying this than respondents in the reference category. The 95% confidence intervals for these odds ratios are also important. Where the confidence interval does not include 1, this category is significantly different from the reference category. If we look at religious identity in Model 2, we can see that Protestants had an odds ratio of 0.23, indicating that they have lower odds compared with those of no religious beliefs (who were the reference category). The 95% confidence interval (0.15-0.35) does not include 1, indicating this difference is significant.

20. The significance of each independent variable is indicated by 'P'. A p-value of 0.05 or less indicates that there is less than a 5% chance we would have found these differences between the categories just by chance if in fact no such difference exists, while a p-value of 0.01 or less indicates that there is a less than 1% chance. P-values of 0.05 or less are generally considered to indicate that the difference is highly statistically significant, while a p-value of 0.06 to 0.10 may be considered marginally significant.

21. The models below show the final model for each variable, which was produced using the Complex Survey command (CS Logistic) in PASW 18. CS Logistic models can account for complex sample designs (in particular, the effects of clustering and associated weighting) when calculating odds ratios and determining significance. The models shown below include only those variables found to be significant after the regression models were run using CS logistic. A number of other variables were removed from the models before these final models were produced. This was for reasons of multicollinearity and for parsimony. Models with too many variables can sometimes be misleading, particularly if there is a strong association between the independent variables. Religious identity, for example, was strongly associated with both religious belonging and family religion. The statistical modelling process does not work correctly if more than one of these variables is included in the model. It is a matter of judgement as to which is the most appropriate to include. In this case, it was judged that religious identity should be included.

Regression models

Table B.3: Factors associated with thinking that there is either a great deal / quite a lot / some' prejudice against Protestants in Scotland

Dependent variable encoding 1 = Those thinking there is 'A great deal / Quite a lot / some' prejudice against Protestants in Scotland 0 = Those thinking there is 'Not very much / None at all' prejudice against Protestants in Scotland	Odds ratio	95% confidence interval
Region (p=0.00)
Not West (reference)	1.00
West	1.86	1.39-2.47
SIMD (p=0.011)
Less deprived 80% (reference)	1.00
Most deprived 20%	1.42	1.08-1.85
Family ties to Northern Ireland or Republic of Ireland (p=0.08)
Lack of family ties to Northern Ireland or Republic of Ireland (reference)	1.00
Family ties to Northern Ireland or Republic of Ireland	1.29	0.96-1.74
Religion / lack of religion important to identity (p=0.043)
Disagree (reference)	1.00
Agree/strongly agree	1.29	0.97-1.72
Neither agree nor disagree (+DK)	1.59	1.09-2.30

Nagelkerke R^[2] = 0.072
Other factors included in the final model but which were not significant after other factors were accounted for were: age; sex; highest level of education and religious identity.
Additional modelling also looked at religious upbringing, church attendance, how religious they consider themselves to be, football club support or social ties to Catholics/Protestants but none of these factors were significant.
Excludes cases where the respondent answered "don't know" to the dependent variable, or where no answer was provided.

Table B.4: Factors associated with thinking that there is either 'a great deal / quite a lot / some' prejudice against Catholics in Scotland

Dependent variable encoding 1 = Those thinking there is 'a great deal / Quite a lot / some' prejudice against Catholics in Scotland 0 = Those thinking there is 'Not very much / None at all' prejudice against Protestants in Scotland	Odds ratio	95% confidence interval
Region (p=0.00)
Not West (reference)	1.00
West	1.70	1.34-2.38
Family ties to Northern Ireland or Republic of Ireland (p=0.010)
Lack of family ties with Northern Ireland or Republic of Ireland (reference)	1.00
Family ties with Northern Ireland or Republic of Ireland	1.38	0.98-1.75
Religion / lack of religion important to identity (p=0.016)
Disagree (reference)	1.00
Agree/strongly agree	1.32	0.95-1.83
Neither agree nor disagree (+DK)	1.72	1.19-2.48

Nagelkerke R^[2] = 0.062
Other factors included in the final model but which were not significant after other factors were accounted for were: age; sex; area deprivation; highest level of education; religious identity.
Additional modelling also looked at religious upbringing, church attendance, how religious they consider themselves to be, football club support or social ties to Catholics/Protestants but none of these factors were significant.
Excludes cases where the respondent answered "don't know" to the dependent variable, or where no answer was provided.

Table B.5: Factors associated with agreeing that 'I am more comfortable around people with similar religious beliefs to my own' - or for those with no religion - 'I am more comfortable around people with no religious beliefs'

Dependent variable encoding 1 = Agreeing 'I am more comfortable around people with similar religious beliefs to my own' - or for those with no religion - 'I am more comfortable around people with no religious beliefs' 0 = Neither agree nor disagree / disagree	Odds ratio	95% confidence interval
Gender (p<0.001)
Male (reference)	1.00
Female	0.54	0.39-0.74
Region (p=0.059)
Not West (reference)	1.00
West	1.41	0.99-2.01
SIMD (p=0.046)
Less deprived 80% (reference)	1.00
Most deprived 20%	0.61	0.38-0.99
Religious identity (p<0.001)
No religion (reference)	1.00
Protestant	0.23	0.15-0.35
Catholic	0.15	0.08-0.29
Other Christian	0.35	0.22-0.57
Other non-Christian	0.66	0.23-1.90
Religious attendance (p=0.020)
No religion (reference)	1.00
At least once a month	2.54	1.40-4.62
At least once a year but less than once a month (+DK)	1.29	0.60-2.78
Less often or never	1.29	0.81-2.06
Religion / lack of religion important to identity (p<0.001)
Disagree (reference)	1.00
Strongly agree	3.26	1.92-5.53
Agree	1.89	1.17-3.05
Neither agree nor disagree (+DK)	1.21	0.75-1.93

Nagelkerke R^[2] = 0.177
Other factors included in model but which were not significant after other factors were accounted for were: age; highest level of education; and family connections with Ireland.
Excludes cases where the respondent answered "don't know" to the dependent variable, or where no answer was provided.

Table B.6: Factors associated with having often or occasionally thought twice about telling someone about your religion / having no religious beliefs because of concern about what they might think.

Dependent variable encoding 1 = Those saying they have often/occasionally thought twice about telling someone about your religion / having no religious beliefs because of concern about what they might think 0 = Those saying that they have never thought twice about telling someone about your religion / having no religious beliefs because of concern about what they might think	Odds ratio	95% confidence interval
Gender (p=0.046)
Male (reference)	1.00
Female	0.67	0.45-0.99
Age (p=0.013)
18-24 (reference)	1.00
25-39	0.46	0.22-0.95
40-64	0.42	0.23-0.77
65 and over	0.26	0.12-0.57
Religious identity(p=0.003)
Protestant(reference)	1.00
Catholic	3.21	1.80-5.70
Other Christian	1.37	0.69-2.71
Other non-Christian	2.28	0.72-7.13
No religion	1.94	0.98-3.83
Religion / lack of religion important to identity (p=0.009)
Disagree (reference)	1.00
Agree/strongly agree	2.10	1.28-3.44
Neither agree nor disagree (+DK)	1.31	0.71-2.43
Family ties to Northern Ireland or Republic of Ireland (p=0.004)
Lack of family ties with Northern Ireland or Republic of Ireland (reference)	1.00
Family ties with Northern Ireland or Republic of Ireland	1.75	1.25-2.54

Nagelkerke R^[2] = 0.148
Other factors included in model but which were not significant after other factors were accounted for were: area deprivation; highest level of education; region.
Additional modelling also looked at religious upbringing, church attendance, how religious they consider themselves to be, football club support or social ties to Catholics/Protestants but none of these factors were significant.
Excludes cases where the respondent answered "don't know" to the dependent variable, or where no answer was provided.