Evaluation of participatory budgeting activity in Scotland 2016-2018

1. Part One - Policy Context and Evaluation

Introduction

The sections in Part One provide the policy context leading up to, and during, the evaluation, and the approach to the evaluation. Section 1.1 outlines the international and Scottish policy contexts of the development and implementation of PB to contextualise the research conducted during this evaluation. Section 1.2 offers a summarised chronology of the Scottish Government's action to promote PB and Community Choices, including the institutional and financial arrangements made in supporting a national development programme. Section 1.3 details the practical approach to the evaluation design in a changing environment with multiple community, organisational, and political interests in play.

Key Points

• There is enthusiasm for increased and improved community engagement.
• There is political commitment to experiment with PB as a concept to be transferred and adapted within the Scottish context.
• While the fluid and dynamic policy context in Scotland has provided enabling conditions for the adoption of PB as a positive approach to improving community engagement, it has also contributed to the variances in definitions, meaning, and focus of how PB has been implemented conceptually and in practice.
• Policy changes have moved at a different pace from developments in practice and have not consistently been informed by practice.
• The approach to the evaluation design aimed to reflect and keep pace with evolving practice with distinctive contexts at the local level and across Scotland.
• Participatory budgeting is considered to be an innovative policy and policy instrument for improving community engagement and supporting community empowerment.
• A dynamic policy context is driving process change at the local level.
• There is political commitment from central government.
• The Scottish Government have invested in funding to support development and exchange of practice.

1.1. Landing PB in Scotland: The Policy Context

1.1.1. Introduction

Empowering local communities to engage directly in how and where public funds can be used to shape public services and their delivery is a principal aim of PB (Gomez et al., 2016), necessitating increased connectivity, trust and participatory democracy for communities (Gonclaves, 2014). As highlighted throughout this evaluation project, a core characteristic of PB is its potential to transform the relationship between citizens and the state through the increased involvement of local communities in financial decision-making processes at a local level. PB has taken many forms as it has developed and adapted to different financial, political, economic, and social contexts. The adoption and implementation of PB have been approached differently depending on whether the initiative is led by citizens, local or national government, or civil society organisations.

1.1.2. Forms of PB

Scotland's recent experience of adopting, adapting, and implementing PB in Scotland has been the focus of ongoing analysis and dissemination (Escobar et al. 2018; Brun-Martos and Lapsley, 2017; Harkins and Escobar, 2016, 2015). The 2017 Interim Report from the evaluation study, confirmed the developing nature of PB implementation in Scotland. Additionally, it drew out the range of conceptual and operational challenges and differences among those involved in different processes and approaches to PB and characterised these elements then as "transactional, transference, and transformational" (O'Hagan et al. 2017). It argued that while a long-term focus of PB is the transformational inclusion of communities in budgeting decisions for mainstream services, distinct challenges persisted in the theorising and design of PB, implementation and achievement of policy objectives in a public finance context of reducing resources and the ongoing reform of public services.

PB has been characterised, either separately or in parallel, as a policy instrument (governance) or as a policy device (community engagement tool) (see Harkins and Escobar, 2015, p.13). As a community engagement tool PB-styled activity has been framed and operationalised as a means of bringing local residents and community members together to discuss and deliberate on local priorities, decide on local priorities and for the allocation of resources to meet those agreed needs. In this context it is not necessarily a means of transferring elements of power and control of mainstream budget allocation to communities, nor does it contain the transformative intent of changing the power relationships between public authorities and local communities.

PB is a developing innovation globally, with limited adoption to date across the UK (Brun-Martos and Lapsley, 2017) with notable exceptions of the activities of civil society organisations and encouragement from social enterprises such as PB Partners and their predecessor the Participatory Budgeting Unit. From the introduction of pilots in Scotland, PB has been described as progressing through 'generations', from small-scale pilots based around local areas and small grants, into the development and operationalising of the 'Community Choices Fund' by the Scottish Government from 2016, marking the second generation. The 2017 commitment to extend participation into mainstream council budgets takes local authorities and the Scottish Government into what Escobar et al. (2018) have referred to as the third generation of PB in Scotland. This background provides useful insights for the wider evaluation which had four key areas of focus: the impact of PB on local communities, services, democracy and how it is tackling inequality.

Scotland has had its own distinctive experience of PB, from isolated activity in different parts of the country in the early 2000s comprising a range of initiatives led by community-based organisations, such as Leith Decides, and intermediaries like the Coalfields Regeneration Trust. Since 2014, the Scottish Government has funded a national programme to support the introduction of PB through local authorities and the creation of structures for networking, information exchange, PB champions, and implementation of digital voting methods. This level of governmental support is unusual, and a distinctive feature of the Scottish experience. The policy and financial context is also specific to Scotland; as with all examples of policy transfer and policy learning, local context is highly significant. In the academic literature, there has been considerable discussion about the form and nature of PB with a range of descriptions and characterisations in play, some of which are discussed in this report and used to frame the findings from the three-year evaluation.

For many of the contributors to the evaluation evidence presented here, PB is a community engagement tool, intended to bring local communities together for local and small-scale decision making. In terms of deliberative and participatory democracy these are important and valuable activities. There has been significant enthusiasm for the concept of PB and a considerable amount of activity by local government officers and elected members to deliver PB events and processes. However, the various approaches led by local authorities conceived and designed in the early 'generations' of PB were not necessarily intended to realise more substantial structural and procedural changes that longer term, transformative participation in local resource allocation decisions will require. Upscaling in the ways that will be required to turn PB in Scotland into a process of community participation in mainstream budget decisions as proposed in the 2017 Framework Agreement, presents new challenges for local authorities, their public sector partners, and communities.

Furthermore, the introduction of PB has to be considered in the context of austerity and budget cuts, giving rise to apprehensions that PB has been utilised as a way to deliver elements of reconfigured public services. Whilst budget allocations were reducing, local authorities were expected to remain focused on their key objectives in reducing inequalities and supporting communities by building capacity and resilience. The tensions in Scotland's public finances persist as allocations both to and from the Scottish Government remain constrained.

The policy context in Scotland is complex and dynamic. The role of local government and Scotland's governance is the focus of the 'Local Governance Review' (LGR) launched by the Scottish Government in 2018 and which will be reporting in 2019. This review was underway at the same time as this evaluation was running. The aim of the LGR is to identify potential reforms in the way that Scotland is governed to give greater control to communities The findings and outcomes from the LGR will be relevant to the implementation of implications for action to support PB in the wider context of public sector reform and changes in local governance arrangements.

There is strong political rhetoric and legislative underpinning for empowered community engagement and PB sits within that policy framing. How that intention has been operationalised and with what effects is the focus of the evaluation across four areas of impact - local communities, services and democracy, and impact on tackling inequalities. Identifying how PB is defined and operationalised by individuals in communities and local councils, and the implications of different perceptions and expectations for PB has been an essential part of the evaluation. Mindful of the wider context of resource constraint and the transference of responsibility, the evaluation has also explored the transformative potential of PB. This approach has shaped the substantive analysis of the evaluation of PB activity from 2015-2018 set out in this report.

1.2. Scottish Government action to promote PB and Community Choices

1.2.1. Introduction

The Scottish Government began its explicit political and financial support for PB in 2014. With an initial focus on local government, PB was presented as a way of engaging local communities in the allocation of public funds for public services, defining PB as a concept which is "recognised internationally as a way for people to have a direct say in how local money is spent."^[1] This section summarises the chronology of Scottish Government support for PB.

Key Points

• The Scottish Government funded a national programme for development of PB knowledge, practice and exchange.
• PB is variably defined, conceptualised and operationalised by local authorities.
• There are multiple frames of community engagement and community empowerment that are shaping the introduction of PB in Scotland.

1.2.2. Policy context

The lead-in to the formal introduction of PB in Scotland dates back to 2007 following consultation and dialogue with communities and local government through the Scottish Government's "Strengthening Scotland's Communities" programme. This resulted in "Community Empowerment: Celebrating Success: Inspiring Change" [a joint action plan with COSLA in 2009]^[2] which expressed a desire to create empowered communities and highlighted PB as a way through which this could be achieved. This action plan also contained a commitment to implement pilot PB schemes to tackle "Anti-Social behaviour … across three Community Planning Partnership areas as part of the community empowerment agenda"^[3]. In 2010, the Govanhill Equally Well PB Pilot was evaluated by Glasgow Centre for Population Health (GCPH). Among the recommendations from GCPH in 2012 was the call for 1% of public investment budgets to be allocated to PB. This was based on the proposal from the UK PB Unit, which operated between 2002 and 2012, as an appropriate level of mainstream funding allocated to public decision making without adversely affecting service delivery^[4].

PB in Scotland has primarily been framed around 3 key areas of legislation and policy direction:

• recommendations for greater cohesion between people and place from the Christie Commission (2011);
• recommendations of COSLA's Commission on Strengthening Local Democracy in 2014;
• and the Community Empowerment (Scotland) Act 2015.

In framing public service reform in Scotland the Christie Commission was cognisant that, for public services to be effective, they should be designed and delivered in conjunction with communities. Equally, reforms to public services should empower individuals and communities via such involvement rather than being delivered 'top-down' by councils. PB was signalled as an initiative that can act as an enabling policy instrument in supporting local community involvement in the financial allocation, design and delivery of public services. The 2014 Commission report^[5] sought to strengthen local democratic control and decision making by enhancing the ability of local government to raise revenue locally. It stated that participatory budgeting was "seen as becoming 'the standard by which [participation in decision making] is delivered in Scotland"^[6].

The Community Empowerment (Scotland) Act 2015 embedded new rights for citizens and communities to participate in policy and decision making, including through the introduction of Participation Requests and Asset Transfer Mechanisms, but not an explicit right or requirement for PB. The emphasis and intention of the Act nonetheless was to increase and enhance community participation in decision making.

A further legislative element with significant potential to support PB is the Public Sector Equality Duty (PSED) contained within the Equality Act 2010. PSED is a positive duty on public authorities to promote equality and tackle inequalities through ensuring policies - including budgets and budget processes - are subject to robust equality-impact assessment and that the processes are consulted upon with relevant stakeholders; to eliminate and mitigate unequal impacts and differential experiences; and to foster good relations in the community. These summarised elements of the PSED, arguably present a proxy definition of PB as it shares the transformative intent (Hepple, 2014) of PSED. PSED has not been as prominent a driver for PB as the other levers, and the evaluation has considered the extent to which it has been maximised as a supporting platform for PB activity at local or national level.

1.2.3. PB implementation in Scotland

Building on the legislative and policy foundations outlined in the previous section, the Scottish Government made a commitment in 2014 to support PB "as a tool for community engagement" and set up a PB Working Group. According to the Scottish Government, the PB Working Group "works in partnership with the Scottish Government to inform the development of PB so that it is scalable, empowering and transformative. The group includes representatives from national organisations working with communities, plus academics, civil society, PB experts, local authorities and central government. Its remit is to oversee the development of PB in Scotland, support its links to other community empowerment initiatives, and advise on the infrastructure required to help its implementation and impact."

As set out in Table 1 below, the first actions to support PB included a funded training programme for self-selecting local authorities in 2014. 26 local authorities participated in the PB training, and 20 signed up for PB consultancy support funded by the Scottish Government and provided by PB Partners. The Scottish Government Programme for Government in 2015 stated a commitment to 1% for PB as part of the Community Choices agenda, and reinforced the commitment in the 2016/17 Programme for Government. In January 2016, a further allocation of funding was made available to local councils to support match-funding for PB activity. Fourteen councils applied and delivered a range of activities in the 3 months to March 2016.

Table 1: Summary Chronology of PB in Scotland

2014	Scottish Government commitment to support and promote PB as a tool for community engagment; PB Working Group established.
2014	PB experts provided training for local authorities across Scotland to introduce PB to Local Authorities: 113 delegates from 26 local authority areas attended the events.
2015	20 councils receive paid PB consultancy support.
2015	Glasgow Caledonian University commissioned to conduct 'Evaluation of PB Activity in Scotland.'
2016	Scottish Government fund 20 councils on a match-funding basis to run PB activity. 14 applied and shared £530,267 to help them build on and maintain their PB activity which resulted in 50 PB events in the first 3 months of 2016.
2016	Scotland's first Open Government National Action Plan included a commitment on PB to empower communities through direct action ensuring they have influence over setting budget priorities.
2016/17	Scottish Government Budget announces £2 million Community Choices Fund to support PB. 33 organisations secured £1.7m and £300k was used for the national support programme which included: consultancy support; digital engagement tools; and training.
2016/17	122 community choices events across the country; 39,000 people voting and 1,352 local projects were successful in getting a share of £2,621,441 (£1.7m Community Choices Fund plus match funding from local authorities).
2017	Interim Evaluation report produced by Glasgow Caledonian University evaluation team and published online by the Scottish Government following presentation at national PB Conferenece in November.
2017/18	Scottish Government commit further £2m to Community Choices Fund. 33 organisations shared £1.5m Community Choices Fund for events in 2018. £500k for the national support programme included: consultancy support; digital engagement tools; support and advice; an evaluation programme, capacity-building to develop practitioners and a COSLA PB Development Manager post.
2017	Scottish Government and COSLA Community Choices 1% Framework Agreement that 1% of council budgets across Scotland will be subject to community choices.
2017	Scottish Government commission Glasgow Disability Alliance[7] to explore what the barriers to participation are for disabled people in decision making and PB and propose solutions for tackling these barriers.
2018	PB was highlighted at the Scottish Attainment Challenge Pupil Equity Fund (PEF) Events as an innovative and effective approach to engage pupils and parents in funding decisions. Also included in the PEF Operational Guidelines 2018.

(Source: based on Scottish Government presentation to 'Theory of Change' Workshop, 16 April 2018, Edinburgh.)

The Community Choices Fund was announced in 2016, renaming PB as 'Community Choices' and allocating £2million to support PB activities across all public authorities, not only councils, and expanding eligibility to community organisations. In 2016, local authorities were delivering a range of events-based engagement processes for community participation in deciding on the local small grant schemes already in place, previously been conducted internally within local authorities. These small grant schemes were used as the starting point from which to introduce more participatory elements to these processes which were to be supplemented with additional funding allocations for local projects enhanced with Scottish Government funding. Community Choices continued in 2017/18 with a further £2million allocated, including £500,000 to support the national programme of consultancy, digital engagement tools, support and advice and capacity building for practitioners. This fund also included a national PB Development Manager post based within the Convention of Scottish Local Authorities (COSLA). PB Partners, an independent consultancy, provided the bulk of the local authority training in the lead-in to and following the introduction of the Community Choices Fund, and continue to provide advice to the PB Working Group.

By this time, the national policy position was moving ahead of the local practice. At local authority and community level, the practice continued to focus on the distribution of small grants, with some thematic activities around mental health, young people, older people, and limited innovations around housing revenue grants, traffic and road spending, and pilots in health and social care, and public transport commissioning. After extensive discussion and consideration on the desirability and implications of shifting community engagement into decision making on mainstream council budgets, the Scottish Government and COSLA issued a Framework Agreement in 2017. Characterised as a "pivotal point for mainstreaming PB in Scotland" (What Works Scotland, 2018), this Agreement commits local authorities to ensuring that 1% of council budgets across Scotland will be subject to community choices.

The specifics of this continuing commitment are for individual councils to decide how to approach what is being called 'mainstreaming PB'. The Framework Agreement attempts to bring together multiple purposes and objectives for PB as it:

"sets PB as the enabler for active participation of citizens in local decision making. It establishes a shared expectation that elected members, senior officers, civil society and local communities will use PB to go beyond the current arrangements for consultation and engagement… The Framework sets out that…the longer term strategic aim of public sector reform can be achieved by applying spend to the greatest areas of need, where social cohesion can be developed or maintained." (COSLA, 2017, p.1)

The apparent intention was to reflect the best practice standards for PB as issued by the PB Unit in 2010 whereby a "partnership approach to PB is taken with mainstream funding identified across a partnership for mainstream services with an aim to shaping how services are delivered in the area." The Framework also "recognises that actively involving local people can make them less passive consumers of public services and more supportive of new models of delivery."

From the statements above, PB is concerned with the active participation of citizens in local decision making; engaging all stakeholders beyond consultation; allocating increasingly scarce resources and reformed public services on the basis of place and need; and shifting local people from 'passive' recipients of services into decision makers on modes of delivery. It is clear from the evidence gathered in this evaluation that as PB has been progressed in Scotland the concept has been loaded with intent and expectation, resulting in a range of interpretations and varying policy objectives attached to it. These multiple expectations of PB comprise a significant conceptual and operational load on public authorities, with an ongoing focus on local authorities.

Furthermore, these expectations for PB exist in a context of diminishing public finances and reductions in allocations to local government. While the strong emphasis in public policy rhetoric in Scotland has been on community engagement and empowerment, cuts to public finance have formed the dominant context. The gradual implementation of these policies of engagement and empowerment has been happening in an environment of ongoing public service reform and diminishing public resources. Local government in Scotland has experienced a period of sustained austerity and as such the landscape, funding and delivery of public services had shifted considerably. The redistribution of risk from central UK Government to devolved governments and subsequently local communities has been apparent, as downward pressures exist from 'austerity' measures and other policies (Asenova et al., 2013; 2014).

A final element of the analysis of the policy context in Scotland concerns the clarity of the conceptual definitions of 'participatory budgeting' and what the term 'participatory budgeting' is understood to mean. The variances in understanding and the application of participatory budgeting as a community engagement tool or method, or as a focus for political change in the relationship between government and citizens is discussed in detail in the subsequent findings sections. The definitions of participatory budgeting used by the Scottish Government during the evaluation period are relevant to how the evaluation study was designed and in the formulation of the findings.

The Scottish Government has identified PB as:

"A tool for community engagement [that] complements the objectives of the Community Empowerment (Scotland) Act 2015, which provides a legal framework to promote and encourage community empowerment and participation. Creating new rights for community engagement and placing new duties on public authorities." (Scottish Government, 2016)

The Scottish Government website offers the following characterisation:

"Participatory budgeting (PB) is recognised internationally as a way for people to have a direct say in how local money is spent. We support PB as a tool for community engagement and for developing participatory democracy in Scotland. PB can:

• support the principle of Public Service Reform that says people should have equal opportunity to participate in decisions shaping their local community and society.
• complement aspirations in the Community Empowerment (Scotland) Act 2015 to give communities more powers to achieve their own ambitions.
• help deliver the Public Sector Equality Duty by advancing equality of opportunity and fostering good relations between different groups."

The reference to supporting the delivery of PSED has been added during the course of the evaluation, and reflects the findings from the Interim Report at year 2 (O'Hagan et. al, 2017) that highlighted the need for stronger direction to integrate equalities analysis and practice in PB.

1.3. Approach to the Evaluation

1.3.1. Introduction

This section outlines the approach to the evaluation including the evaluation criteria, research methodology and understanding the wider context. Between 2015-2018 the research team undertook interviews with local authority elected members, officials and finance directors from urban and rural Scottish councils who were involved in the early implementation of PB through Scottish Government funding. In addition, the team attended PB events across Scotland to observe PB processes and held focus groups, interviews and conducted a survey with community representatives to explore the lived experience of PB. The following section presents a detailed overview of the research conducted during this evaluation.

Key Points

• There are four areas of impact that the evaluation focuses on: impact on communities, services, local democracy, and tackling inequalities.
• A mixed case study approach was taken, the sample comprises urban, rural, and mixed; single and multiple local authorities are included within the case studies.
• Multiple methods of qualitative evaluation were utilised across the evaluation, including observation, interviews, learning set, documentary analysis, and surveys.
• The evaluation extended to three years at the end of the second year to consider the potential impact of introducing the 1% target (Community Choices 1% Framework Agreement).

1.3.2. Evaluation Criteria

The overarching focus of the project was to evaluate PB activity in Scotland. in relation to four central areas, as set out in the Scottish Government specifiation for the evaluation:

• What are the impacts of the PB approaches and activities on communities?
• What are the impacts of the PB approaches and activities on services (where the PB process relates to a significant service budget)?
• What are the impacts of the PB approaches and activities on local democracy?
• Explore the relationship between PB activity and its agenda to tackle inequalities?
• Explore the implementation of the 1% mainstream approach to PB.

The fifth element of the evaluation was added in 2017 following the introduction of the Scottish Government / COSLA Community Choices 1% Framework Agreement.

In constructing the areas of inquiry for the evaluation, the key research questions were broken into the following elements:

1. Impacts of the PB approaches and activities on communities?

• Development of individual skills, experience and confidence.
• Development of organisational capacity.
• Improvements in social capital, social cohesion, etc..
• Improved perceptions of influence and attitudes towards community action.
• Improved perceptions of local services, and potentially improved experiences of services.

2. Impacts of the PB approaches and activities on services (where the PB process relates to a significant service budget)?

• Changes to services arising directly from PB processes (including questions of whether resources have been reallocated to disadvantaged areas).
• Changes to responsiveness of services.
• Changes to partnership working.

3. Impacts of the PB approaches and activities on democracy?

• Levels of civic participation.
• Connections between Councillors and community organisations in their ward.
• Councillors awareness and ability to identify participatory forms of democracy at ward and council level.

4. Relationship between PB activity and its agenda to tackle inequalities?

• PB could be a vehicle that enables all sections of the community (irrespective of gender, disability, ethnic origin, colour, citizenship or other social status or identity, specifically those characteristics protected under the Equality Act 2010) to engage in local decision-making.
• That it reduces, rather than increases, inequalities within and between communities and effectively expands inclusive participation.

5. Implications of the implementation of the 1% mainstream approach to PB?

• In what ways are local authorities rolling out the 1% focusing on two elements - a close-up, action research focus on perspectives from practitioners and community activists, and on the organisational preparedness for implementing the 1% commitment?

1.3.3. Methods

Adopting multiple qualitative methods for engagement and data generation, the evaluation comprised elements of action research to reflect the participatory nature of the focus of the evaluation (PB); a case study design for sampling and data capture across a range of variables (see Table 2); and an extended series of semi-structured interviews with relevant stakeholders. Our original proposal was for a series of case studies to be constructed to provide sample data on the range of activities that were already underway when the evaluation commenced, and which would allow flexibility to respond to changing processes and practices as the evaluation continued. The selection criteria for the sample case studies were proposed by the evaluation team and agreed in consultation with the Scottish Government at the inception of the evaluation.

The evaluation team proposed criteria set out in Table 2 below. Mindful of the scale of the project in relation to the notional workload - as distinct from the nominal allocation in the original specification - this included consideration of costs and time efficiencies for the project, including consideration of co-terminous local authorities, travel time for researchers, and other practical elements of the research design, including access to contacts and pre-existing research relationships.

Table 2: Case Study Selection Criteria

	Western Isles	Glasgow	Edinburgh	Aberdeenshire	Pan Ayrshire (North, South, East)	Fife
Experience of PB		X	X			X
New to PB	X			X	X
Commitment to PB	X	X	X	X	X	X
Structures in place			X		X	X
Budget identified	X	X				X
Urban		X	X
Mixed				X	X	X
Rural	X
Diverse community		X	X			X
Cost efficiencies for the project budget		X	X	X	X
Ease of access	X	X	X	X

The evaluation design aimed to engage with a variety of stakeholders within the case study areas as well as with representatives of a broad range of actors to represent the wider context and to offer a uniquely detailed and nuanced insight into PB in Scotland. Going beyond the original evaluation specification the researchers took a staged approach to capture the changing landscape of conceptualising and implementing PB activity. A list of the events, meetings, seminars, conferences, and other engagement opportunities attended or created by the evaluation team is contained at Appendix One.

Data Gathering Methods

A range of data gathering methods were used to capture qualitative data on the experiences and perceptions of the different 'actors' engaged in PB in Scotland. Experiences and perceptions gathered ranged from local people attending or participating in PB events, local council officers involved in designing and delivering events and formulating PB processes, and local councillors involved in local and strategic planning of PB events and longer-term approaches to council finance, and finally in relation to the 1% target Directors of Finance were interviewed.

In Stage one, semi-structured, more formal interviews were conducted with council officers and elected members using 'topic guides' or interview schedules of common questions. More informal conversations recorded as fieldnotes were conducted with participants and observers at PB events. In Stage 2, further topic guides were prepared for follow-up interviews with council officers and elected members, and for focus groups and interviews with community members. An online questionnaire and a learning set approach were also developed at this stage. In Stage 3, a further set of questions were produced as interview schedules for the exploration of attitudes to the 1% target with council officers and representatives. Separate topic guides were formulated for interviews with third party organisations in the different stages.

Institutional interviews in all stages lasted between one and one and half hours representing a significant time commitment from council and other organisational participants. Mindful of these time commitments and the constraints on local community members, the evaluation team selected contact forms to identify candidates for follow-up telephone or focus group participants. Members of the evaluation team also attended PB working group meetings in two case study areas as observers. These meetings provided further background to the planning and policy contexts of PB activity in these case study areas and helped identify PB events to attend, and potential interview candidates from the local authorities and other partner organisations.

Table 3 below summarises the evaluation activity across the three stages. These stages overlapped, praticularly in 2016-2017 when interviews, attendance at events, and desk-based research were ongoing simultaneously. A more detailed description of each stage is provided below and in Appendix 1.

Table 3: Evaluation Activities 2015-2018

Stage One- Overview of PB practice across local authorities
2016	20 interviews with council officers and elected members Attendance and observation at 11 local PB events
Stage Two- In-depth Case Studies
2017	Attendance at 17 PB events across the 6 case study areas to gather primary data Desk analysis of published results from 30 events across 5 local authority areas
	5 Interviews with community members
	2 Focus Groups
	Learning Set established
	18 Follow-up interviews with council officers and elected members
	31 responses to a participant survey
Stage Three- Lessons for Mainstreaming
2018	Continuation of Learning Set
	16 Interviews with council officers and finance departments
	4 organisational interviews

Stage one

The first stage of research comprised 20 interviews between 2016-2017 with local authority elected members and officials from 20 councils who were involved in the early implementation of PB through Scottish Government funding, as previously published in the Interim Report. As set out in Table 2 (above), six case studies were then selected on a range of criteria including; experience of PB, urban/rural context, localised allocations for PB funding in place, experienced or novice PB practice. The findings from these interviews, documentary analysis, fieldnotes and observations, participant questionnaires informed the Interim Report on the evaluation project, published in 2017.

Observation of 11 PB events across Glasgow, Edinburgh, Pan-Ayrshire (North, South and East) and Fife which were recorded in the evaluation fieldnotes along with informal conversations and observations of the PB processes. These processes and events included speed dating, stalls, 'Dragon's Den' style presentations, all-day voting, online voting, and short events with timed voting periods.

Stage two

The second stage of the research was carried out to gain further insight into the implementation and practice of PB within the case studies, with a particular focus on the experiences of community members. This stage involved a range of activities by the evaluation team including:

• Attendance at 17 PB events across the six case study areas to gather primary data on the design, participation, bidding and award processes.
• Analysis of the results of 30 events from Glasgow, Fife, and the three Pan-Ayrshire councils for financial year 2016/2017, using a combination of data generated from attendance at events and secondary data from local authority websites. This data included summaries of bids proposed and awards made at the events attended by the evaluation team.
• Online questionnaire in 2017 to capture the experiences of those bidding and attending PB events in their local community. Contact forms were distributed among participants at PB events in Glasgow, Fife, Edinburgh, and the Ayrshires. Questionnaires were distributed using contact details and consent forms collected at events and via local authority contacts who had consent from individuals to contact them directly. The total number of questionnaires circulated is unknown due to the distribution via local authority representatives. The exercise generated 31 responses from across the case study areas.
• Using action research methods, a learning set of 5 people was formulated comprising members of community organisations, council officers and representatives from the voluntary sector all involved in various aspects of PB across three of the case study areas. The learning set participants came from different case study areas and met 5 times between August 2017 and June 2018. The learning set used action research methods, with members reflecting on practice in their local area in which they were engaged or which they were observing. As PB activity was already underway in most of the case study areas when the evaluation started, the methods for action research were revised and adapted. The thought had been to recruit community researchers at the start of the process, however because PB activity was underway the format was changed to a learning set approach. Participants reflected on their experiences and perceptions in a series of informal meetings with semi-structured questions used to focus on dialogue and learning about PB activities in order to consider areas for development and change.
• Follow up interim-stage interviews with 18 officers and elected members from three of the case study areas were conducted to capture the learning and change since the stage one interviews.
• 2 focus groups and 5 interviews were held with community members involved in local PB events. These participants were recruited via the contact sheets circulated at the PB events in Glasgow, Fife, and the Ayrshire(s) attended by the evaluation team in the winter/spring of 2017.

Stage three

The third stage of the evaluation focused on the move towards PB mainstreaming and the implications of the 1% target set by the Scottish Government. In this stage of the evaluation, the learning set continued to meet, reflecting on the ways that PB has been framed, changes that are required, and what it means to 'mainstream'. A supplementary workshop was organised in one of the case study areas not represented in the learning set, designed to generate further insight into the learning and reflections emerging from the action research group. A further 16 interviews were conducted. These included final interviews with council officers previously interviewed from the case study areas, one local authority not involved in stage one, and one interview with a local authority also interviewed in stage one (but not a case study local authority). These interviews also included 3 Directors of Finance and one Budget and Resources Committee Convenor departments to explore the perspectives on the 1% target and the processes that are being established to ensure that it is met.

This activity demonstrates sustained engagement with case study authorities with a range of experiences, in addition to revisiting non-case study authorities, and adding in 'new' councils through overlapping stages of the evaluation. Over almost 3.5 years, this evaluation provides a longitudinal perspective and cummulative appraisal of reflections on developing practice in PB from local authorities, communities and other stakeholders.

Across the different stages there was also a strand in the evaluation that sought to investigate the wider context for PB. Interviews were carried out with representatives from the following organisations: PB Partners, COSLA, Scottish Community Development Centre (SCDC), Coalfields Regeneration Trust, and informal discussions with Glasgow Disability Alliance. In addition, informal conversations and fieldnotes with local actors at PB Scotland Network conferences and events, an Equality and Human Rights Commission conference (February 2017), and COSLA elected members' events in 2017 and 2018, which the evaluation team attended as observers to informally explore and capture the ways in which communities and councils have been supported in carrying out PB activities. Additionally, the team attended a number of events organised by bodies interested in the development and practice of PB. For example: PB Charter, Glasgow Disability Alliance (GDA), Equality and Human Rights Commission, Common Weal, What Works Scotland, PB Network Scotland, and SCDC. These events offered insight into the broader conversations happening around the development and implementation of PB, beyond the local authorities and the specific case studies selected for this evaluation.

This work was supplemented with two events organised by the evaluation team that brought together representatives from organisations involved in PB, beyond the 6 case study areas, to 'sense check' the emerging findings and receive feedback on the interim report. A full list of events attended and organised is contained as Appendix 1 to the interim report.

These activities and research across the three stages and the strand to understand the wider context have generated a rich set of data to inform this evaluation report. The quotes presented are drawn from the different data gathering methods noted earlier. Contributors have been anonymised to the extent that it is possible to do so by removing place names, local authority areas, organisation names and roles, and using generic titles instead.