Cleaner Air for Scotland – air quality public attitudes and behaviour: final report

2 Methodology

This chapter presents our methodological framework, including the peer-reviewed and supplementary sources used, the selection of search terms, the search limits and parameters, evaluation of search results and thematic analysis of the findings.

2.1 Overall approach

The objectives outlined in Chapter 1 were addressed by conducting a review of available scientific and grey literature, including policy documents, research project reports. Figure 2-1 depicts the methodological approach.

2.1.1 Peer-reviewed sources

UWE Bristol has access to more than 40,000 journals, many of which are online, and a wide range of databases containing peer-reviewed journals and other relevant sources and resources across a broad array of subjects. Previous experience suggests a targeted approach focused on one or two key databases can yield cost-effective yet comprehensive results. To that end, we focused on the Scopus database and the ENDS Report.

2.1.2 Supplementary sources

While peer-reviewed literature provides the most scientifically robust and rigorous source, it was recognised that studies on public attitudes and behaviour change related to clean air policies in Scotland may not necessarily have been published in academic journals. In addition to the database searches therefore, a review of grey literature and other sources was also undertaken. Research projects identified from the CORDIS database and other relevant documents selected from organisations’ and institutional websites including EU institutions’ websites, Eurocities, C40, SEPA, Scottish Government, DEFRA, The Behavioural Insights Units OECD, Global Action Plan, and the UK Committee on Climate Change, were included in the search.

2.1.3 Search terms

Search terms were standardised and recorded to facilitate transparency and reproducibility, and were relevant to the objectives as set out in the tender specifications.

The key subject areas of relevance to this review are:

• Public attitudes, knowledge and awareness/ concerns and behaviours relating to air pollution in Scotland
• Air pollution and public engagement approaches in Scotland, UK and internationally

Search terms were therefore included that reflect these key themes. The use of appropriate search terms is vital to conducting a successful and useful review and therefore expertise was sought from across the project team in consultation with the Scottish Government to agree an appropriate search strategy.

Search terms were themed according to the core criteria:

• Search 1 (Public Attitudes to air pollution in Scotland): Air pollution (focused terms and related issues) + Scotland + Public + (Attitudes OR Knowledge OR Awareness OR Concerns OR Behaviour) (TITLE-ABS-KEY search)
• Search 2 (Air pollution and public engagement approaches): Air pollution (focused terms) + Public + (Attitudes OR Knowledge OR Awareness OR Concerns OR Behaviours) + Approaches (TITLE-ABS-KEY search)

Search 1 presents a relatively narrow search of literature relevant to Scotland and hence the intention was to broaden the scope of air pollution theme terms to incorporate wider issues, including climate change, sustainability and place-making. By contrast, Search 2 focused more narrowly on air pollution specific terms as it was expected that this search would generate a much larger number of relevant results and hence requiring a need to limit these in line with the project scope, resource and timeframe. Both searches shared search terms relating to public and attitudes, knowledge, awareness, concerns and behaviours, given that this was relevant to both, however, Search 2 also included terms representing methodological approaches used in public engagement. Detailed search terms used for the Scopus search are provided in Appendix A. Further refinement of the search terms was required for the ENDS Report (Appendix B) and CORDIS database (Appendix C) searches due to inherent limitations of their search functionalities.

2.1.4 Search limits/parameters

Searches of the Scopus and ENDS databases were carried out on the 6^th and 7^th April 2020 respectively. Searches were limited to scientific literature published since 2015, to ensure only the most up-to-date evidence was included, but earlier literature was also included where relevant. Search terms for Scopus, run as Title and Abstract searches. Results from the Scopus search were stored and classified according to the Search 1 and 2 categories using Mendeley reference management software.

Searches of the CORDIS database were also carried out on the 6^th April 2020. Searches were limited to EU projects funded since 2015. Search terms for the CORDIS search primarily focused on Search 2. The search was extended beyond CORDIS using the same search terms, but also expanded to include additional sources relevant to Search 1 where identified. Results from the supplementary source searches were recorded in an Excel spreadsheet.

Both searches were limited to English language documents.

2.1.5 Evaluation of search results

The level of significance attributed to the search results were subject to a risk assessment derived from the ‘Six categories Quality Assurance Model’ in which higher scoring literature carry more weight. The categorisations are based on the Quality Assurance statement reported (Table 2-1).

For Scopus, Search 1 returned 6732 papers whilst search 2 returned 1066. These were screened by title and abstract for relevance to search 1 and 2 criteria. Search 1 subsequently returned a shortlist of 219 relevant documents whilst search 2 returned 75 relevant documents.

Shortlisted Scopus results were extracted to an Excel spreadsheet and assigned scores based on their Field-weighted citation impact (FWCI )^[9] and SCImago Journal Rank (SJR )^[10] (Scopus research metrics^[11]), before a further, more in-depth screen of each paper content was undertaken for relevance. Finally, a detailed review of each remaining paper was undertaken.

For the Scotland-specific Search 1 shortlist, key findings, themes, robustness of evidence base and outstanding gaps in evidence were extracted where possible. For Search 2, behaviour change theory/methodology, the results/effectiveness and limitations, and applicability in different contexts and recommendations for public engagement were extracted where possible. If information relevant to Search 2 was identified in Search 1, it was also recorded.

The ENDS database search returned 26 articles were for Search 1, of which, following a more detailed screen of the Title (no abstract available), 6 were shortlisted. Search 2 returned 17 articles, of which 12 had some relevance. Following a more detailed review of the content of Search 1 articles, no articles of relevance were recorded. Search 2 recorded 3 articles of relevance for the grey literature search and were subsequently screened for grey literature relevance.

The supplementary source search returned initially 227 results. The content of the projects’ deliverables and the retrieved documents were screened for relevance to Search 1 and 2 criteria. A detailed screening of titles, abstracts, and content of deliverables or report returned 42 relevant documents. These were filtered to 41 results and expanded to 45 following inclusion of subsequently identified sources. Relevant results included research projects (22), reports (11), toolkits, news items, a poll, campaign, conference paper, consultation responses and an advice paper, from various sources, including academic institutions, consultancies, government departments and agencies, local mayors, charities and NGOs. Shortlisted documents were assigned scores based on a scale from 1 to 3 (based on Table 2-1). Details of the specific behaviour change theory/methodology, and their documented results/effectiveness and limitations were extracted from the sources and entered into the search results spreadsheet. Further information on their applicability in different contexts and recommendations for public engagement were also extracted where possible.

Once all relevant records from all sources were effectively extracted, the results were thematically analysed by topic (e.g. air, transport, domestic energy), behaviour change theory/methodology (e.g. Theory of Planned Behaviour) and/or public engagement approach (e.g. citizen science), by reviewing the extracted material in the Excel spreadsheet and creating a separate column by which the records were then filtered. This was further refined as the synthesis of the literature was undertaken.