Scotland's Climate Assembly - process, impact and assembly member experience: research report

Executive Summary

About this report

This report presents findings from independent research into Scotland’s Climate Assembly.

The Climate Change (Emissions Reduction Targets) (Scotland) Act 2019 required Scottish Ministers to establish a citizens’ assembly on climate change, to function independently from the Scottish Government.

Citizens’ assemblies bring together a group of individuals, recruited through random and stratified selection to broadly represent the wider population with respect to key demographics, and in some assemblies, views on the issue under consideration. The assembly deliberates on information provided by experts, which leads to the production of a set of recommendations with the aim to inform decision making.

Scotland’s Climate Assembly was organised and delivered by a Stewarding Group, Secretariat, Design and Facilitation Team, and Evidence Group. The Assembly was conducted entirely online, due to the Covid-19 pandemic.

The Assembly had 106 members^[1]. These were individuals randomly selected to broadly represent the wider population of Scotland with respect to key demographics and climate attitudes.

The Assembly met online over seven weekends between November 2020 and March 2021 to deliberate on information provided by experts and produce a set of recommendations, addressing the question: How should Scotland change to tackle the climate emergency in an effective and fair way?

For three of the seven weekends, the Assembly members were split into three separate topic streams: Diet, Land Use & Lifestyle; Homes & Communities; and Travel & Work.

The Assembly Report containing the recommendations was laid in the Scottish Parliament on 23 June 2021. The Act required Scottish Ministers to publish a statement within 6 months of receiving the report setting out how they intend to respond to the recommendations. This response was published on 16 December 2021^[2]. An eighth Assembly weekend meeting was held in February 2022 to discuss the Scottish Government response. For the purposes of this research, Weekend 8 is regarded as a follow-up meeting and not part of the main Assembly.

This research report covers:

• Assembly process including organisation, remit and evidence.
• Assembly member experience including participation, learning and climate attitudes.
• Assembly member and public support for the Assembly and its recommendations.
• impact of the Assembly on climate change debate and policy in Scotland.
• outcomes for members including climate action and civic attitudes and participation.

The report concludes with discussion of key factors affecting the quality of the Assembly, and the impact of the Assembly on government policy and public debate in Scotland. It identifies key considerations for future assemblies: citizens’ assemblies in general, online assemblies, assemblies in Scotland, and climate assemblies. Finally, key areas for further research are identified, for this Assembly and more generally.

Research methodology

The research presented in this report has been conducted by Scottish Government Social Researchers working in collaboration with an academic researcher from Newcastle University.

This report addresses the following research objectives:

a) to evaluate the success of the Assembly as a deliberative process^[3], identifying key factors affecting its quality.

b) to contribute to cumulative learning in Scottish Government about the effective use of citizens’ assemblies and other deliberative approaches, in the specific context of the manifesto commitment to holding citizens’ assemblies.

c) to contribute to international evidence and learning on use of deliberative approaches for engaging citizens in government policy development, particularly on climate change, and in the specific context of Scotland.

The research used a mixed methods approach to analyse and integrate a range of different types of data. Data sources analysed for this report include:

• online member surveys, conducted prior to Assembly starting and after each Weekend meeting.
• non-participant observation of Assembly small group discussions.
• audio recordings of Assembly small group discussions.
• qualitative semi-structured interviews and a qualitative survey with 18 people involved in organising and delivering the Assembly, including members of the Secretariat, Design and Facilitation Team, Stewarding Group and Evidence Group. These interviews were conducted after the Assembly ended.
• population survey by Deltapoll with representative sample of 1917 adults in Scotland, conducted 29 July-14 August 2021.
• evidence presentations from the Assembly.
• the Assembly report.
• secondary data on climate and civic attitudes.
• Scottish Government policy documents.
• media coverage.

The member surveys were completed by around two thirds of members^[4]. Therefore, the results presented in this report should be regarded as indicative only of the views of all Assembly members.

Member quotes are included for illustrative purposes and to provide nuance. The quantitative survey findings are also supplemented with qualitative vignettes of four members’ experiences, charting different types of journeys through the Assembly. Pseudonyms have been used for the vignettes, and quotes have been anonymised. Where data exists, members’ views have been compared with results of the Deltapoll population survey and other secondary data.

In this report, interviewees and survey respondents from the Stewarding Group, the Secretariat, the Design and Facilitation Team and the Evidence Group are collectively referred to ‘Organising members’ to protect anonymity.

Throughout the report, a simple broad scale is used to describe the rough proportion of research participants giving a certain view or response:

all = everyone in the sample

most / a majority = more than half of the sample

some / a minority = less than half but more than ‘a few’

a few = two to five (depending on sample size)

one = one person

strong support = 75% and above

majority support = 50-75%

Further details of the methodology including limitations can be found in Appendix 1. Data tables for the quantitative data in this report are published alongside the report.

Key findings

On the whole, the research finds that the Assembly was well organised and delivered, with several innovative features including:

• involvement of Children’s Parliament in a parallel process that at times interlinked with the Assembly, and the inclusion of their Calls to Action in the Assembly Report.
• presenting Assembly members with scenarios of possible futures that depicted different worldviews and routes that could be taken to address climate change, and that showed how change can happen at different levels and paces.
• more measures were used to integrate the work of different topic streams than in previous climate assemblies.
• creation and promotion of a Civic Charter expressing support for the Assembly and its recommendations, which has been signed by organisations and individuals.
• continuation of the Secretariat after the main Assembly period, to organise public engagement activities and engagement with government officials and ministers and members of Parliament.
• organising a further Assembly meeting to discuss the Scottish Government response to the Assembly recommendations, including a Ministerial Q&A.

The Assembly members received a range of relevant information on climate change and its mitigation, and to a lesser extent information on adaptation. Member survey data suggests that on the whole members’ learning about climate change and their support for particular climate actions increased over the course of the Assembly, as did their concern about climate change as an urgent issue.

Survey data indicates there is strong support from Assembly members and majority support from the Scottish public for the Assembly goals and recommendations, and the statements of ambition. The Assembly has also featured in news media coverage of climate change.

Whilst there is evidence that the Assembly has made a contribution to climate change debate and policy in Scotland, more research is needed over the longer term to assess the extent of its impact with regards to influencing specific changes in policy and debate in the coming months and years.

However, there were also limitations to the Assembly and some aspects that worked less well. In the following sections, key lessons and considerations for future assemblies are identified. This learning can make an important contribution to improving outcomes of Scottish Government policy, both in relation to climate change and to participatory and deliberative democracy.

The Scottish Government has stated its commitment to improving outcomes and improving people’s experiences in numerous policy spaces in recent years. This is rooted in a long-standing commitment to, for example, the Scottish Approach to Service Design^[5], the outcomes focus of the National Performance Framework^[6] and the principles of Public Service Reform^[7] which emphasise the need to be in constant dialogue with Scotland’s people: listening, engaging and responding, and building on the principle that everyone is entitled to have the opportunity to shape Scotland’s shared future.

Assembly organisation

The organisers successfully delivered an online citizens’ assembly that produced a range of recommendations to address climate change, during difficult circumstances due to the Covid-19 pandemic. A majority of Assembly members were satisfied with the organisation, communication and support they received.

Early relational work between the Secretariat and the Stewarding Group was helpful in building a good relationship, with the Stewarding Group providing a useful steer for the Secretariat. There were also positive relationships between the Secretariat and the Design Team although there were differences in views about the extent to which the Assembly had been co-designed. The Evidence Group had less time to build relationships and trust before the start of the Assembly, which likely contributed to a challenging start with their decision making process, although this improved over the course of the Assembly.

The Assembly was delivered within a short period of time due to requirements of the Climate Change Act and the Scottish Parliamentary pre-election period, further Covid-related legislation notwithstanding^[8]. Whilst this tight schedule generated a sense of momentum, it created pressures for all aspects of planning and delivery, which were experienced as challenging or stressful by some, as well as pressures for members in digesting large volumes of evidence in short periods between weekends. Many other factors affecting the quality of the Assembly likely stem from this issue.

There was some uncertainty and lack of clarity regarding roles, relationships and decision making authority of the various groups involved in organising and delivering the Assembly, including the relationship between the Secretariat and the Scottish Government.

Late design decisions before many of the Assembly weekends, whilst enabling responsiveness to emerging issues, limited possibilities for oversight by the Stewarding Group and impacted on the ability of others to adequately plan and deliver. There was also a reliance on Evidence Group members voluntarily contributing more time than contracted or agreed, which not all were able to do, thereby exacerbating inequalities in contributions.

Assembly remit

The use of a deliberative process for developing the Assembly question by the Stewarding Group worked well. However, there were differences in views about how to operationalise the Assembly question, in terms of what it meant for the evidence and tasks set to members. These differences mainly related to: how closely the Assembly should align with policy needs with respect to the Climate Change Act and its targets, the extent to which the Assembly should explore the reasons why climate change is deemed an emergency, how strategic, and how radical an approach it should take or allow. These differences indicate a diversity of views amongst the various groups involved in organising and delivering the Assembly (the Stewarding Group, Secretariat, Design and Facilitation Team and Evidence Group), which is positive from a deliberative perspective, although the differences were not resolved. There were also differences in views about the extent to which the Assembly dealt effectively with the systemic nature of climate change.

The final remit was very broad, making it a challenge to address in the time available, despite having seven weekends. Indeed, many members felt the sessions were rushed with insufficient deliberation time to develop their recommendations.

To satisfactorily address the broad question, the Assembly was divided into topic streams for three of the seven weekends, which enabled members in each stream to go into more depth on that topic. The topics were chosen by the organisers rather than the Assembly members. The material from all of the streams was made available to all Assembly members, but as time for reviewing the material was not built in to the sessions, they would have had to review it in their own time. This introduces elements of inequality, as some members would not have time or skills to do so in isolation, and it risks perpetuating the learning inequalities that citizens’ assemblies are designed to overcome. The topic stream structure meant that Assembly members did not all engage with the same evidence, and also made interconnections between topics more difficult to understand.

Measures were put in place to integrate the topic streams and to share learning and deliberation across the streams, which enabled more co-ordination of the recommendations than in other climate assemblies, for example Climate Assembly UK^[9]. These measures involved mixed stream groups in Weekends 6 and 7, which helped members to understand the recommendations made in other streams and why they were proposed, although not to the same extent as the recommendations from their own stream. There was an element of members having to trust that members in other streams had followed a good process to arrive at their recommendations, with some more comfortable about relying on trust than others. The mixed stream groups were therefore not sufficient to ensure that all Assembly members had gained enough knowledge and understanding to fully endorse recommendations from the other streams. The broad remit also led to a large number of recommendations being made by the Assembly, with differences in views about whether a large number of recommendations is problematic.

As the Assembly remit did not include a critical review of all existing and planned Scottish Government climate policy, members were not necessarily aware of all that was already in process and consequently developed some recommendations that broadly matched existing or planned policy.

Online format

Running an entire Assembly online is extremely difficult, but the challenge was met effectively and the organisers were well prepared for the online digital format. There was a good level of technical support for members, both in advance and during the Assembly, although connectivity issues impacted at times on some members’ ability to participate. Video presentations enabled review of the presentations in advance of them being broadcast to Assembly members, allowed for re-watching by members, as well as allowing the evidence to be easily made publicly available. Other advantages of the online format included availability of experts and accessibility for those members for whom attending in person would have been problematic.

However, there were disadvantages. Shorter sessions (due to concerns of online fatigue) meant less time for evidence and deliberation, it was also more difficult to cater for a range of learning styles, and more difficult to both facilitate and monitor the performance of facilitators.

Whilst a sense of community amongst the members was generated, there were different opinions about whether more social activities would have enhanced this further. Previous studies have found that sense of community and socialisation can elevate the quality of deliberation. However, as this research has found, with a topic stream structure, a strong overall assembly identity may increase the risk of insufficient critique of outputs between topic streams.

Assembly design

Most members found the small group, mixed stream and plenary sessions helpful for their learning, and were satisfied with the balance of open discussions to task-based discussions.

However, there were difficulties in completing tasks within session time, and work was often rushed and at times overran into breaks. There were also difficulties in balancing time for evidence with time for deliberation in the time available. Indeed, many members did not feel there had been enough time to develop and finalise the recommendations, even if most agreed their views were reflected in the final outputs.

Overall, there were good elements to the facilitation, particularly given the considerable challenges of facilitating online and working with multiple tools and documents. As a result, members largely felt included and respected, with ample opportunity to express their views, although there were at times issues with one or more members dominating their small group, affecting others’ participation.

A key feature of deliberation is members putting forward ideas and suggestions about what should be done (referred to as ‘demands’). This requires particular facilitation techniques. Within this Assembly, techniques employed were more in keeping with fostering dialogue than deliberation. This may have contributed to the low level of demands that were made in the sample of group discussions analysed. When members did make demands, they mostly included a justification. However, only around a third of justifications made an explicit link between the demand and the reason. Such a link is an indicator of quality in deliberation. Most justifications were in service of the general interest or common good, which is also an important deliberative norm.

There were instances when facilitators inaccurately recorded members’ contributions, and these mistakes were not always picked up. Members were not always enabled to write down their views or ask questions to experts themselves.

Good pastoral support for members during the Assembly was provided, although more staff resource would have been helpful due to the emotional labour involved. The process and tasks were generally explained well to members. However, it was not always clear to members how their work had been collated and consolidated between weekends. Members had some opportunity to influence the Assembly process, and most members felt their views were reflected in the various outputs over the course of the Assembly, suggesting an overall high sense of ownership of the process and outputs.

Evidence

The evidence provided was generally good quality in terms of content, with high production values. However, peer review processes could have been better and more consistent.

Survey and observational data suggests that Assembly members understood and engaged well with the evidence, although there could have been better use of resources such as evidence summaries to help members keep track of key points. Some members felt overwhelmed at times by the volume of information. Although the interactions between experts and members was limited, both groups found it useful.

There were differing views on the extent to which diversity and balance of evidence had been achieved. Climate impacts, adaptation and resilience were under-represented in the evidence relative to mitigation, with this imbalance also reflected in the Assembly recommendations. Some interviewees also thought that the severity of the climate crisis may not have been sufficiently conveyed to members, particularly in the first two weekends.

Deliberative process

The findings indicate that, on the whole, the Assembly was successful as a deliberative process:

• the Assembly brought together a diverse set of people from Scotland, with differing views on climate change.
• the members were enabled to have meaningful discussions about how Scotland should tackle the climate emergency in an effective and fair way.
• the Assembly was successfully run online with generally high levels of member engagement.
• demands made by members in a sample of small group discussions about what should be done were frequently justified with reasons relating to the common good.

Assembly impact

This section provides an overview of public support for the Assembly and its outputs, and an assessment of the impact of the assembly on climate policy and debate in Scotland, based on data available to date. Key outcomes for members are also summarised.

Public support

Survey data indicates there is majority support from the Scottish public for the Assembly goals and recommendations, and the statements of ambition.

According to Implicit Response Testing, some statements of ambition from the Assembly appear to have higher emotional resonance with the public than others. The message that everyone having a shared role in taking action is likely to have high emotional connection whereas there may be some resistance to messages that make a direct and specific demand for cultural change at various levels, including personal change. There also seems to be some doubt about Scotland’s capacity to be a climate change pioneer.

Both the Citizens’ Assembly of Scotland and the Deltapoll population surveys find that a majority of the public appear to support fundamentally changing Scotland’s economic model to tackle climate change, and think that new ways of engaging people in decision making are needed and that citizens’ assemblies are worthwhile. These are tentative findings, hence further research on these aspects is needed.

Impact on Scottish Government policy

The Scottish Government response is comprehensive, but the way it is written makes it difficult to identify exactly what impact the Assembly has had on policy, and no evidence of Assembly impact was found in other policy documents analysed. It is generally unclear in the government response how change will be implemented at the scale and urgency emphasised in the Assembly Report’s Statement of Ambition.

With an overall lack of specific timescales and measureable objectives in both the Assembly recommendations and the government response, comparing recommended to existing or planned action is open to interpretation. A third of recommendations appear to broadly match existing or planned policy, with around a fifth being explored by government in some way albeit with no commitment to implementing. Over a third of recommendations include policy that will not be taken forward. Whilst 14 recommendations relate to UK Government reserved matters, the Scottish Government committed to contacting the UK Government about these, and has done so^[10].

Member survey results indicate that between the end of the main Assembly period and after receiving the government response, there was a decline in members’ confidence in the Scottish Government taking the Assembly seriously. There also appears to be some misalignment in views and expectations, between the Scottish Government and Assembly members, as to what constitutes an appropriate government response.

There were many features of the Assembly that were designed to optimise its impact. The Secretariat remained in post beyond the end of the Assembly enabling them to promote the Assembly recommendations within the Scottish Government. The Civic Charter has mobilised some stakeholders and civil society groups to support the Assembly recommendations. Weekend 8 brought the Assembly members and Government Ministers into discussion together, via a Q&A session, over the government response to the Assembly.

Impact on climate change debate

The analysis of online news media found that coverage was dominated by three Scottish-based outlets, which was also the case with the Citizens’ Assembly of Scotland. There were higher levels of coverage when the final Assembly report was published and during COP26. The Scottish Government’s response has received little media coverage to date, though it may yet increase. To date, the coverage has been primarily positive or neutral (40% and 41% of all coverage respectively). The analysis shows that the Assembly has contributed to the climate change debate in the online written news media. However, in the absence of a counterfactual, the way in which it has changed the debate is unknown.

Outcomes for members

Evidence suggests the most popular climate actions for Assembly members since taking part in the Assembly are: reducing amount of meat and dairy in diet, reducing overall consumption, and reducing domestic energy use. Other popular actions include: discussing climate change and politics with friends and family, and making consumer decisions based on associated climate impacts. Many members feel more confident to engage in political decision-making as a result of being involved with the Assembly, and agree that taking part in the Assembly has made them want to be more involved in other aspects of government decision making.

Key considerations for future assemblies

Considerations for citizens’ assemblies

This Assembly has enabled valuable public deliberation over an important public policy issue. Key considerations include:

• extent to which future citizens’ assemblies should give members more say on the Assembly remit, evidence, decision-making process and report drafting. In this Assembly there were some measures in place to enable the members to have some control of the process. However, they could have been afforded more opportunities to shape the agenda in accordance with the Arrangements for the Administration and Operation of Citizens’ Assembly of Scotland: Scotland’s Climate Assembly, which states that “within the remit of the legislation, and with expert support, members of the assembly (once in place) will be able to shape the assembly’s agenda.”^[11]
• how much time should be made available for building relationships between organising/delivery groups in advance of key decision making to improve communication and collaboration, and to make best use of experts’ time.
• how to schedule Assembly weekends so it is (a) not overwhelming for members, (b) ensures that there is a manageable workload for organising and delivering the Assembly (including evidence provision), whilst (c) still ensuring there is momentum to the process, and (d) that information provided is not forgotten by Assembly members.
• including pathways to impact that were adopted in this Assembly e.g. the continuation of the Secretariat beyond the duration of the Assembly to organise public engagement activities, the Civic Charter, COP26 related events, and occasions that bring Government ministers and Assembly members into discussion together.
• how governments formally respond to citizens’ assemblies with clarity and precision, for example making use of a ‘you said, we did’ format.
• how scrutiny of government action in response to assembly recommendations should take place.
• designing a research programme that allows for assessment of impact over the longer term, and that embeds data collection from members into the Assembly programme.

Considerations for citizens’ assemblies in Scotland

This research on Scotland’s Climate Assembly indicates public support for more assemblies to be organised in Scotland that enable the public to contribute to policy issues and debate in a balanced and informed manner.

Consideration could be given to institutionalising the use of citizens’ assemblies by establishing rules and regulations for their instigation, governance, and Government response; clarifying issues of independence and accountability; and providing an office space and a dedicated budget to fund citizens’ assemblies and to fund the implementation of key recommendations. This could have a number of benefits including:

• mitigating the negative consequences of introducing legislative requirements to conduct a citizens’ assembly within a limited set time period.
• enabling clear rules and resources for the governance of citizens’ assemblies so there is a clear division of labour, but also sufficient resources to staff the assembly organisation and provision of evidence adequately.
• providing adequate resource within government to respond to and implement assembly recommendations.
• aligning stakeholders’ expectations about the nature of the Government response, and what constitutes an appropriate response.
• supporting cumulative learning, as experience is sustained across assemblies.
• promoting scrutiny of government actions.

The institutionalisation of citizens’ assemblies requires that the organisation, practice, and culture of government is open to heeding assembly recommendations and learning lessons from the Climate Assembly and other deliberative processes to support improvement^[12]. Additionally, for Scotland, investment and training is needed to increase the facilitation capacity for deliberative processes.

A further consideration is how to handle topics involving matters reserved to UK Government.

Considerations for online assemblies

The research shows that citizens’ assemblies can be successfully held entirely online. However, there are some key considerations for future online assemblies.

• how to provide evidence in a variety of formats to meet different learning style needs.
• how to ensure there are frequent opportunities for assembly members and experts to discuss the issues together.
• if shorter sessions are used for online assemblies, due to the greater intensity of digital participation, how to ensure there is still sufficient time for deliberation.
• innovation is required to enable quality control and review of the facilitation to occur in online sessions. For example, small group discussions could be recorded and reviewed at a later date.
• provision of social opportunities to enable assembly members to bond and build trust in ways that provide a bedrock to deliberation and decision-making, and that support equal participation in a manner that does not exclude some members or hinder critical exploration. With a topic stream structure, a strong assembly identity may also increase the risk of insufficient critique of outputs from other streams. Further consideration needs to be given as to whether social opportunities are an optional or required part of participation, and whether these social activities are included in the gift payments to members. Social opportunities increase the workload for members and could be a barrier to participation or create difficulties for some members e.g. those with caring responsibilities or who work unsocial hours.

Considerations for climate assemblies

The research indicates that citizens’ assemblies are a good format to enable the public to engage in climate change debate. Considerations for future climate assemblies relate to the remit, evidence and Assembly member wellbeing.

Remit of Assembly

• climate change is vast and complex, and is not a ‘problem’ that can be ‘solved’. However, Assembly members could be empowered to refine the remit and consider the areas of climate change that are of greatest importance to them or of highest societal priority in mitigating emissions and adapting to climate impacts.
• how closely the assembly question, evidence and deliberation should be aligned with policy needs (e.g. legislation and targets), or should have a more open remit.
• how to ensure the systemic nature of climate change is designed in to the process in a way that does not over-complicate the learning experience for members.
• the implications of how the assembly question is framed for the number and scope of the resulting recommendations, and whether to restrict the number of recommendations produced.
• whether to include a critical review of existing policy to avoid Assembly members spending time developing recommendations for policies that already exist or are planned, of which they were unaware.
• as climate change is a long term phenomenon with potential for fast or unexpected developments due to feedbacks and tipping points, the policy context will also change. A one-off Assembly is unlikely to be sufficient, and consideration will need to be given to the shelf-life of recommendations and whether further or ongoing assemblies are required. A more permanent arrangement would enable more responsive governance.

Evidence

• how to ensure evidence relating to climate impacts, adaptation and resilience is as effectively communicated to members as mitigation.
• how the severity, scale and urgency of the climate crisis can be adequately conveyed to assembly members.
• understanding that there are no “neutral” ways of communicating about climate change, as all communication involves framing. A perception that scientific evidence is objective, values-free and emotion-free can lead to bias in selection of evidence. Care therefore needs to be taken to understand the implications of choices.
• development of a robust and consistent peer review process for the provision of evidence, with a shared understanding of what constitutes balance and bias.
• whether and how much to include children’s views. Whilst this was welcomed by some members of this Assembly, others thought the time would have been better spent on deliberation of evidence.

Assembly member wellbeing

• how the emotional wellbeing of members is monitored and supported as they learn about the severity of the crisis, as well as deal with the nature of the Government response.

Further research

Key areas for further research on Scotland’s Climate Assembly, and for citizens’ and climate assemblies in general, include:

• analysis of impact on policy of Scotland’s Climate Assembly over the longer term.
• assess the impact of the Civic Charter on the signatories’ organisational policies and practices, or on other parties’ policies.
• analysis of broadcast and social media coverage of Scotland’s Climate Assembly for further evidence of impact on debate.
• longitudinal research on the relationship between Assembly members and Scottish Government.
• involvement of Assembly members in further political activity, including in relation to deliberative or participatory democracy.
• climate mitigation and adaptation behaviours of Assembly members.
• comparative analysis of Scotland’s Climate Assembly with other climate assemblies to enhance understanding of the contribution they can make to climate governance at local and national levels.
• how the organisation, practice and culture of the Scottish Government, and governments and parliaments in general, need to be reformed to enable an empowering role for citizens’ assemblies.
• comparative analysis of existing attempts to institutionalise deliberative mini-publics such as citizens’ assemblies.

Information about climate assemblies including research publications can be found at the Knowledge Network on Climate Assemblies website https://knoca.eu/.