A Consultation on Proposals for a Lobbying Transparency Bill: Analysis of Written Responses

2. APPROACH TO ANALYSIS

2.1. This chapter describes the approach taken to the analysis and presentation of the data. The method involved the following elements:

• Design of an Access database and verbatim input of the material.
• Quantitative analysis and presentation of the data (where appropriate).
• Qualitative analysis and presentation of the data.
• Preparation of this report.

Database design and input of the material

2.2. An Access database was designed, containing a field for each question. There was also an additional field for any other observations made at any point in a response which, while relevant, did not relate directly to any of the questions posed in the consultation.

2.3. Each respondent was given a code and a number, which identified their category (within those agreed with the Scottish Government and used in Table 1 in Chapter 1). These codes allowed any patterns of responses, and areas of agreement or disagreement to be highlighted by type of respondent, as well as providing a clear audit trail for each point made.

2.4. The responses were transferred verbatim to the database at the appropriate question. Any responses which did not follow the questions asked, or were in "non-standard" format were read carefully to identify which of the questions were being addressed by the material. The material was put into a separate field, but the content was analysed along with the rest of the information for the relevant questions.

Quantitative analysis and presentation of the data

2.5. The consultation required a small amount of quantitative analysis, and some limited quantitative summary of qualitative data in terms of patterns of views (although most of the analysis was qualitative and is discussed later in this chapter).

2.6. The main quantitative information was extracted from the database in the first instance. A numerical count of respondents by category was carried out, as well as an initial count of those who responded directly to each question.

2.7. Given the qualitative nature of the data, however, some respondents made relevant comments at other points in their responses. Additionally, as noted in para 1.14, some did not follow the question format. In order to avoid overlaps and repetition in the report, the material in these responses was considered at the most relevant questions, alongside the comments from those who addressed each question directly.

2.8. The number of these respondents was added to the overall count for each question. This gave the total number of respondents who addressed a question at some point in their response (giving a more meaningful indication of the level of response to each issue). It should be noted, however, that this was reliant on some subjective judgement, and cannot be taken to be an exact figure.

2.9. As noted in Chapter 1, a number of questions asked respondents whether they agreed with a particular suggestion, or invited them to choose between two options. These were "open" questions, however, insofar as they did not contain a "tick box" for the answer. This left respondents to indicate their preference in the text of their response. This meant that respondents did not always give a straightforward "yes" or "no" answer (or similar), although some did.

2.10. In order to determine the pattern of views at these questions, some subjective judgement was needed in order to identify whether each respondent had stated or implied a clear view, or whether this was impossible to ascertain from their comments. This was undertaken for each of the questions of this type. The numbers who appeared to be expressing a clear view were counted. Again, this should not be taken as an exact figure.

2.11. The quantitative data presented includes the number of respondents overall, and the number / proportions of different types of respondents overall (Chapter 1, Table 1). The broad proportion of respondents who answered each question is also given (although not as an exact figure, for the reasons given at para 2.8). The general balance of views expressed at the "do you agree" questions (and similar questions offering a "choice") is also given, again in broad terms (for the reasons given at para 2.10).

Qualitative analysis and presentation of the data

2.12. The qualitative analysis involved working systematically through the detailed comments made at each question. To facilitate this, a two column Word table was generated for each of the qualitative questions, with one column containing the respondent identifier, and the other containing the verbatim material from each respondent (with a separate row for each).

2.13. This material was then analysed to identify the themes and sub-themes at each question. This allowed the detailed arguments to be identified, along with any suggestions made. It also allowed any contradictions or anomalies to be highlighted. The actual themes for each question were generated from the data received.

2.14. The detailed comments made at each question (in the fully "open" questions and where respondents were asked to give the reasons for their answer, or provide their more general views) generated a large amount of qualitative material.

2.15. It was inappropriate to attempt to quantify these detailed views for a number of reasons, including that:

• Many points were made at different questions. To avoid repetition, some judgement was required to group and present these at the most relevant question.
• Some responses represented the views of a number of individuals or organisations.
• The focus was on the identification of the range and detail of views, rather than a "weighing" of responses.
• The respondents were self-selecting and it is not possible to generalise from these findings. The consultation is not a survey of the population.

2.16. The qualitative nature of the information is reflected in the terms used to present this in the report (e.g. "a small number"; "a few"; "several"; "many"; etc.), highlighting the overall themes and the range and depth of views. It should also be recognised that some detailed points were made by small numbers of respondents, but it was important that these views were reflected in the analysis.

2.17. The material was examined for any clear variations in overall patterns of views by respondent type. Where this was the case, these are highlighted in the report, although it would be inappropriate to list the types of respondents identifying each theme in each case, and would make the report difficult to read. It should also be noted that the numbers of respondents in some of the categories were very small, making it inappropriate to identify any patterns in these cases.

2.18. The wording used in the presentation of the qualitative material sometimes follows the wording of a response closely. This ensures that respondents' intended messages are reflected (although not presented as a "quote"). Quotations have not been used, as this might imply that the view of one respondent carried more weight than another.

2.19. The term "respondent" refers to one response, even if this represents the views of more than one contributor.

The findings chapters

2.20. The remainder of the report presents the findings.

2.21. For each question, the report provides an outline of the key issue which the question set out to explore, as well as a brief summary of the material contained in the consultation document.

2.22. The level of response to the question is identified, along with a summary of respondents' overall views, giving the balance of views (where relevant).

2.23. The detailed views expressed at the question are then summarised, and the themes and sub-themes highlighted.

The full responses

2.24. This report cannot provide a compendium of the consultation material, nor can it present every individual point made at each sub-theme, as there was a large volume of detailed information. It does, however, summarise the overall themes and issues raised, even where specific issues were raised by only a small number of respondents. The full text of the non-confidential responses can be viewed on the Scottish Government website^[8].