Public participation in policymaking: exploring and understanding impact

How is impact conceptualised in academic and stakeholder accounts?

The following sections explore how impact is conceptualised within academic literature and stakeholder accounts. First, it will detail some of the typologies which have been developed for better understanding the forms of impact which can arise from public participation, then it will detail the reflections of participatory practitioners and experts when asked to detail and describe impact. Finally, it details a number of 'key insights' which arise from the review of literature and the interview data related to how impact is thought about and understood in the context of public participation processes.

Why do we need to think about impact?

As participatory and deliberative democracy, and its many different iterations, have gained increasing interest from governments worldwide, so too have efforts increased to understand how public participation processes are experienced and unfold in the different contexts in which they take place. In recent history, there has been a recognised, and valid, focus on process design (Bussu, Bua, et al., 2022). In recognition of the volume of participatory possibilities available and the fact that 'there is no canonical form or institution of direct public participation in democratic contexts' (Fung, 2006, p.66), lots of research and practice attention has focused on understanding the intricacies of how participation occurs and processes are designed (Elstub & McLaverty, 2014; Fung et al., 2003; OECD, 2020; Smith, 2009).

This has led to a strong focus on 'what's going on in the room' and perhaps less focus on what happens 'out of the room' when the outputs of a participatory process interact with the social and political systems in which they are taking place (Russell, 2017). This means we now have a highly valued wealth of knowledge on the institutional design of participation processes but what some have described as a 'significant lacuna in our understanding of the impact of public participation on decision-making' (Font et al., 2018, p.615). Indeed, there are some common critiques laid at the doors of public participation, particularly in the context of policy impact. These include claims that participation processes perform a tokenistic function (Fung, 2015; Smith, 2009), with regular doubts cast as to whether the decision-making of institutions and their representatives is meaningfully influenced by citizen participation (Rolfe, 2016). This means that considering impact, in all its forms, but particularly in the context of policy decision-making, is a contemporary concern for many participatory practitioners and theorists (Demski & Capstick, 2022).

Impact typologies

As demonstrated in the introduction, the range of potential benefits from public participation processes are diverse and far reaching. As such, across the literature related to public participation, there are a range of different approaches to and definitions of impact. This section considers three typologies which detail different ways that impact can be thought about and understood. Across these typologies, and indeed the wider literature, there are some clear headline themes in how impact is conceptualised:

• central to many descriptions of public participation is the aspiration that the process will be 'empowered' in some way (Bussu, Golan, et al., 2022; Fung, 2004), and this often means processes which are directly linked to policy or legislative change (Bua, 2019; Bussu, Golan, et al., 2022; Russell, 2017; Smith, 2009);
• despite these common aspirations, there is widespread recognition that linear, direct impact on policy outcomes can be both difficult to achieve and evidence (Bua, 2019; Dean, 2019; Goodin & Dryzek, 2006; Papadopoulos, 2012);
• indeed, across the typologies detailed below, there is consensus that impact must be thought of as multifaceted and occurring across a variety of different spheres, be that public, political/ policy or civil society (Demski & Capstick, 2022; Goodin & Dryzek, 2006; Russell, 2017).

In their analysis of the 'macro-impacts' of mini-publics (deliberative exercises which seek a representative sample of a particular population, Elstub & Escobar, 2019) spanning across a range of different countries and topics, Goodin & Dryzek (2006) aim to trace the multiple and varied 'pathways available for impact' which can arise from these processes. They identify eight potential paths, which include, but also reach beyond, direct policy change:

Table 2: pathways to impact (Goodin & Dryzek, 2006), with additional examples

1. Actually making policy: when a process is 'formally empowered' in a decision-making process, for example where the recommendation (s) of a mini-public are put to the wider public in a referendum. Although considered 'rare' (Goodin & Dryzek, 2006), key examples include British Colombia's Citizen's Assembly on Electoral Reform (Carty et al., 2008) and the more recent Citizens' Assembly work in Ireland (Farrell et al., 2019).

2. Taken up in the policy process: the more usual case of mini-publics where recommendations are made from a process with no 'formal agreement' that they will be acted upon. Organisers typically seek a 'hard or soft' guarantee that the process will inform decision- making. Two examples of particular interest here are the Scottish Citizens' Assemblies (Andrews et al., 2022; Elstub et al., 2022), both of which had a response from Government but no formal guarantee about the fate of the recommendations.

3. Informing public debates: when the deliberations and decisions of a mini-public are communicated to and influence broader public spheres. An example of Deliberative Polling (Fishkin, 2018) is provided in Australia, where an exercise on genetically modified foods stimulated, previously minimal, public debate on the issue (Goodin & Dryzek, 2006).

4. Shaping policy by market testing: where mini-publics or other participation processes provide decision-makers with the opportunity to market-test the acceptability of their policy proposals with the general public. An example of this is an AmericaSpeaks exercise 'Listening to the City', where rebuilding plans in Manhattan were resolutely rejected and subsequently redrawn on the basis of public input (Goodin & Dryzek, 2006).

5. Legitimating policy: when mini- publics 'lend legitimacy to particular policy recommendations' thanks to public participation in the process. Cites the use of Citizens' Juries in the UK, which have been used to resolve tricky issues, particularly at a local level. A pertinent example is the case of the Leicestershire Hospitals, where initial objection to the relocating of certain services from a local hospital was upheld, whilst the broader plans for centralising services supported by a citizens' jury (Parkinson, 2006).

6. Confidence/ constituency building: This relates to the psychological/ sociological consequences which can arise from mini-publics, at an individual level, where confidence, skills and knowledge are increased, but also at a more collective level where groups of people become mobilized around a particular issue. An example of this is the Reconnecting Communities and Schools project in the USA, where an advisory group filed a lawsuit as a collective to prevent loss of funds for their school (Goodin & Dryzek, 2006).

7. Popular oversight: where mini- publics, and broader participation processes, provide a mechanism of public oversight and accountability for the decision-making of officials, often via direct interaction and dialogue with decision-makers themselves. A well- known example is Chicago Community Policing, where street-level police officers and local communities problem- solved and prioritised action together, facilitated by short feedback loops which allowed community members to consistently query decision-making and performance (Fung, 2004).

8. Resisting co-option: This pathway is specific to exercises, such as mini- publics, which provide opportunity for diverse perspectives and high quality deliberation. The central discursive element of deliberative processes is thought to open up a space where genuine opposition and input into a topic is permitted. A participation process where this is evidenced is thought to reduce the likelihood of 'co-option' of citizen voices into policy decisions. Examples here would be work to bring in different forms of knowledge into decision-making spaces, for example in the form of the 'lived expertise' of citizens (Hill O'Connor et al., 2023; Scottish Government, 2022c).

Through identifying these pathways to impact, Goodin and Dryzek (2006) intended to challenge the persistent 'stories of failure' which tend to dominate literature on public participation processes, stating it was 'necessary to counter those skeptical of the impact of such innovations, and to illuminate the subtle as well as the obvious ways they can make a difference' (Goodin & Dryzek, 2006, p.225). It is important to note, in their analysis, they are not blind to the many, and varied, challenges which can stand in the way of realising these impacts (explored in more detail in later sections). Indeed, they acknowledge that the pathways to impact described above can be fuzzy and indirect, but also that public participation processes have, at times, had 'major impacts on macro-politics' (Goodin & Dryzek, 2006, p.238).

Building on this work, other theorists have developed typologies for understanding the impact of public participation processes. These also seek to define impact as multifaceted and multi-level, incorporating both direct and indirect consequences or changes which emerge from public participation. Often, theorists emphasise the need to look beyond specific policy or legislative change as the primary, or only, indicator of impact (Bua, 2019; Demski & Capstick, 2022; Langkjaer & Smith, 2023; Russell, 2017). Indeed, for Russell (2017) primarily viewing impact through the prism of 'high authority' processes (which are not commonplace, Bua, 2019; Goodin & Dryzek, 2006), runs the risk of missing out on other, important, impacts which may occur. In this context, it is important to recognise the different impacts which take place across social, political and public spheres (Russell, 2017). This typology speaks to the pathways identified by Goodin and Dryzek (2006), but groups them into categories of sites 'where a range of impacts might be found' (Russell, 2017, p.3):

Indeed, through their exploration of this typology of impacts in the context of three practice examples of participation in Australia, Russell (2017) evidences that achieving impact across all of these categories is possible, albeit complicated. This study argues that public participation processes play an important 'influencing' role in political decision-making, with the capacity to 'not only influence decisions in the empowered sphere [institutional decision-making], but also [to] influence debate in the public sphere and amongst other groups' who feed into policy decision-making (Russell, 2017, p.4). Along with Goodin and Dryzek (2006), this typology makes the case for thinking about the potential spectrum of 'impacts on the system as a whole' (Russell, 2017, p.4), which can, and do, arise from public participation processes.

Further work to understand impact has also been instigated through the uptick in deliberative processes, most notably citizens' assemblies, in policymaking to tackle climate change (Smith, 2024). Whilst the aim of these processes is to ultimately influence decision- making, often at a national / country level related to planetary health, recent work has reflected previous arguments that impact must be thought of as varied and multi-level (Demski & Capstick, 2022; Langkjaer & Smith, 2023). There is recognition in this literature that previous evaluation of climate assemblies has tended to focus on process (see for example, Andrews et al., 2022; Elstub et al., 2021), rather than on the longer-term fate of the participatory outputs and recommendations within the political and social systems to which they are connected (Demski & Capstick, 2022). In order to remedy this, and to evolve thinking around impact generally, the Knowledge Network on Climate Assemblies (KNOCA) has developed an evaluation framework, which builds on the previous work of Goodin & Dryzek (2006) and Russell (2017). It aims to identify the broad-ranging and multiple impacts which may arise from citizens' assemblies. Like Russell's typology, the KNOCA framework groups impacts across different spheres, in this case: Policy, Social and Systemic (Demski & Capstick, 2022). Intended for use by practitioners and designers of processes, it details the different types of impact which may occur across these spheres and identifies three categories of impact: Instrumental, Conceptual, and Capacity-building, as detailed in Table 4. This framework provides detail on both the key actors involved (both from within and outwith policymaking processes) and the types of changes which might occur, notably to:

• How things work and what happens
• How people think
• What people do (Demski & Capstick, 2022)

As with the previous typologies, this framework recognises the 'limits in any attempt to disentangle the threads of cause and effect' (Demski & Capstick, 2022, p.2) in policymaking, particularly within the realms of climate action. Instead, this framework aims to provide a tool with which participatory practitioners can identify areas of impact as well as think about how to effectively understand where a process has had influence at the point of design. The rationale is to ensure that the aims and objectives of an assembly are properly articulated within the design process and any necessary pathways to impact, and follow-up (Langkjaer & Smith, 2023), considered prior to the convening of the process (Demski & Capstick, 2022). Whilst impact on policy is still prominent within this typology, it also details the multiple ways in which a participation process might influence: i.) policymaking processes via changes to knowledge and understanding of key actors;

• ii.) public discourse via changes to media coverage or public behaviour / attitudes; iii.) democratic systems via the development and use of deliberative processes and increased trust in institutions.

In short, there are some clear themes across these three typologies when it comes to conceptualising impact:

• Impact is complicated and multi-level, occurring across different spheres, including public, political / policy and civil society.
• It is important to think beyond direct policy change and consider impact on policy and decision-making processes and the knowledge, skills and capacity of key actors within, and outwith, the policymaking system.
• Impact can be both direct and indirect and can include influencing public debate / media coverage of particular policy issues as well as broader influence on democratic systems and participatory capacity within policymaking institutions.