'Your Scotland, Your Referendum': An Analysis of Consultation Responses

3 APPROACH TO THE ANALYSIS

3.1 In undertaking an analysis of consultation responses, it is important to bear in mind that a consultation is not a vote; neither is it a population survey. It is generally the case that individuals who have a keen interest in a subject - and the capacity to respond - are more likely to participate in a consultation than those who do not. Therefore, even in cases where the response to a consultation has been very large, the findings of that consultation cannot be assumed to be representative of the views of the wider population.

3.2 Because of this, the main approach to consultation analysis is generally qualitative in nature.

Qualitative analysis

3.3 The main purpose of the 'Your Scotland, Your Referendum' consultation was to gather the range and depth of views that individuals and groups have on the set of issues presented in the consultation paper. Therefore, the paper asked open questions ('What are your views on X?') - rather than asking closed (tick-box) agree / disagree questions. This allowed respondents to record their views in full, rather than simply indicating whether they agreed with the proposed approach or not.

3.4 Thus, the main aim of the analysis was to identify (qualitatively) the key themes, as well as the full range of issues, raised by respondents in their comments on each question.

3.5 Separate qualitative analytical frameworks were developed for each of the nine questions, with a separate code created to cover each of the main themes arising from the responses to those questions. For example, in relation to Question 2 (What are your views on the proposed timetable and voting arrangements?), two of the main themes relating to the first part of the question were that: (i) sufficient time was needed to allow for debate prior to the referendum and (ii) a delay in holding the referendum could have an impact on the Scottish economy. Both of these main themes, and other main themes identified in the analysis, were given separate codes in the analytical framework for Question 2.

3.6 Coding was carried out on a question-by-question basis. Every comment was considered, and one or more codes were recorded as appropriate.

Quantitative analysis

3.7 Although the primary approach to the analysis was qualitative, in order to provide some context for the qualitative analysis, the Scottish Government decided to explore, through quantitative analysis, the broad balance of opinion specifically in relation to the following issues:

• The wording of the referendum question (the first part of Question 1)
• The proposed timetable (the first part of Question 2)
• The inclusion of a second question in the referendum (the first part of Question 3)
• Voting on a Saturday (the first part of Question 6)
• Extending the franchise to 16 and 17 year old young people (Question 7).

3.8 Note that, in relation to Questions 1, 2, 3 and 6, no attempt was made to explore (in quantitative terms) the balance of opinion on the ballot paper, the voting arrangements, the voting system that could be used, or other ways of making voting easier.

3.9 In considering the results of the quantitative analysis, it must be remembered that respondents were not specifically asked whether they agreed or disagreed with the Scottish Government's proposals. Therefore, many of the respondents who took part in this consultation did not explicitly indicate whether they agreed or disagreed. Rather, their responses were discursive and, in some cases, ambiguous or inconclusive in relation to their agreement or disagreement with the proposals.

3.10 Information on the quantitative coding frameworks, and tables showing the results of quantitative analysis are included in Annex 3.