Public attitudes towards fireworks in Scotland: representative omnibus survey

Method and sample

Research method

The research was conducted using a self-completion omnibus survey of the target audience, namely a representative sample of the Scottish population. The omnibus was delivered online using Progressive’s Scottish Opinion Omnibus, in partnership with YouGov. 

The key benefits of using the online omnibus approach were: 

• Fast turnaround of fieldwork to comply with timetable of project;
• Highly cost effective way to reach a large representative sample of the Scottish population;
• Survey reaches people across Scotland, including those in remote and rural areas; and
• Self-completion method means that potential social bias in terms of not wishing to verbalise perceived negative or not ‘politically correct’ attitudes is less likely to be an issue.

The possible limitation of this approach was that the survey was completed by YouGov’s online panel, which is made up of people who volunteer and are paid for the surveys they complete. As a result, the panel may under-represent certain sectors of the population, e.g. older people in lower socio-economic groups. 

However, we took great care to mitigate any impact of these limitations. The YouGov panel is made up of people of all ages and social grades in order to ensure that it is representative of the Scottish population, and does not show any of the biases that have historically been associated with internet usage. A complex weighting matrix was also applied to any data generated by the omnibus to ensure that it is accurately representative of the Scottish population in terms of demographics and geographical spread across the Scottish regions.

The survey questionnaire was developed by the project team at the SG in collaboration with Progressive, who recommended minor amends based on our experience of running online omnibus projects. YouGov scripted and hosted the survey for online completion. Fieldwork took place between the 9^th May 2019 and the 14^th May 2019.

A full technical appendix for this research is available in Appendix A.

Sample

The target audience was a representative sample of the Scottish population. 

Respondents were invited from YouGov’s opt-in panel of over 30,000 Scottish adults. The survey used targeted quota sampling to select which participants were invited, as this allowed for demographic targets to be reached efficiently. 

Emails were sent to panellists selected at random from the base sample. The email invited them to take part in the survey and provided a generic survey link. Once the panel member clicked on the link, they were sent to the survey. Due to the nature of the sampling (sometimes known as router sampling) and volume of panellist traffic, non-response basis is not something that affects omnibus sampling.

For all projects that are required to be representative of the Scottish population, data is weighted once fieldwork has ended. The sampling and weighting scheme used key demographics including gender, age, socio-economic grouping, location, and education to map out a sample frame that accurately represents the adult population of Scotland. The proportions for each quota group are based on Office of National Statistics (ONS) census data and the Labour Force Survey.

A total sample of 1,002 was achieved.

Confidence levels

An overall sample size of 1,002 provides a dataset with a margin of error of between ± 0.62% and ±3.1%, calculated at the 95% confidence level (market research industry standard). This is a robust sample for analysis.

Analysis and reporting

Data analysis

Sub-group analysis was conducted by the following groups, and significant differences reported where applicable:

• Gender
• Socio-economic grouping^[2] (SEG)
• Age group
• Scottish region
• Scottish Government 2 fold Urban Rural Classification
• Scottish Index of Multiple Deprivation (SIMD)^[3]

The Scottish Government 2 fold Urban Rural Classification and Scottish Index of Multiple Deprivation were based on respondents’ postcodes. Please note that respondents were not obliged to provide postcodes and therefore bases for these classifications are slightly lower than the total base.

Open-ended analysis

Open-ended coding was completed by our in-house team of experienced coding specialists. This involved coders creating an analytical framework used to identify themes and common ideas. This was created by a group of research specialists reading through the open-ended responses and noting down themes and ideas that were commonly expressed. Once the basic framework was created, we analysed all the scripts and tagged each of the comments against identified themes. This determined the frequency of occurrence. As new themes occurred, we added these to the framework.

Reporting conventions

Throughout this report significant differences in the data are noted where they occur, to the 95% confidence level. Only significant differences are reported, and the word ‘significant’ refers to statistical significance.

Responses under 3% have not been included in the open-ended tables.