People's Panel for Wellbeing 2022 and Beyond: process and learning evaluation

2. Methods

A combination of documentary analysis, in-depth interviews and a focus group were used to answer the research questions. These methods are outlined in subsequent sections, followed by a description of the analytical approach taken. This research was granted ethical approval by University of Glasgow College of Social Sciences Ethics Board.

a. Documentary analysis

The Scottish Government research team provided a series of key documents that offered insight into some of the practical processes and resources required to set up the People’s Panel (for example, consent forms, information sheets, feedback surveys, needs assessments), the ways that the findings from the panel were reported to other parts of the government (for example, reports) and the format and content of the meetings (for example, presentations, facilitator guides, planning documents). A full list of the documents included for analysis are in Appendix one. A detailed reading and analysis of the documents addressed RA1 and some initial insights into the RA2. The analysis also offered an overview of the process that supported the development of the topic guides used in the in-depth interviews.

b. In-depth interviews

To supplement the documentary analysis and to offer up-to-date evidence of emerging ideas and impact, interviews were conducted with members of the People’s Panel and with members of the policy community from across the Scottish Government.

The Scottish Government research team sent an email to each of the panel members with information about the research and sought permission to send on their contact details to the researcher. At the time of the research there were 23 panel members, all of whom were emailed by the Scottish Government research team with information about this research, seeking permission to send on their contact details to the researcher. Out of 23 panel members, 11 responded positively and these names were sent to the researcher. From this list, interviewees from the People’s Panel were sampled to reflect which breakout group they were a member of, within the People’s Panel meetings. These breakout groups were organised by the research team based on the members views and behaviours around COVID-19 (for example, the extent to which COVID-19 guidance was followed and vaccine status), level of accessibility needs and a group for newer members who joined the panel from meeting 4 onwards (November 2022). There were five small groups in total, one interviewee was randomly selected from each of the groups plus an additional interviewee from group 4 as that was a larger group.

The policy community interviewees were sampled for variation across levels of seniority, types of roles, ensuring that members of different teams were included. Working with the research team the sample also included those who were involved in different parts of the Panel process. For example, delivery of information, requesting a particular policy theme or question, using the People’s Panels discussion to inform their work. A total of 12 interviews were carried out, 6 with panel members and 6 with members of the policy community within the Scottish Government.

The topic guides for the interviews were developed using the steps laid out by Mason (2002), beginning with the consideration of the ‘big research questions (2002: 69) as per the research aims above. These were broken down into smaller, more specific sub-categories that were used to inform the set of questions in the topic guide. The topics covered included motivation and rationale for involvement in the process, description and understanding of the process, perspectives on what went well and what could be improved. A final open question was included to give all interviewees the chance to add anything that they felt may have been missed in the interview.

c. Focus group

Following the analysis of the interviews and key documents a focus group with the research team was organised to discuss some of the key findings and reflect on the People’s Panel process. As the team responsible for organising and facilitating this work in the future, it was an opportunity to reflect on the successes of the process, what could have been done differently and the potential future directions for the work. The final People’s Panel event had not taken place at the time of the focus group, therefore there was not a chance to discuss the entire process. However, in offering comments on the initial draft of this report the team were able to offer additional views having had chance to consider the entirety of the process. Comments that were made in interactions that followed the focus group are indicated clearly in the following sections.

d. Analysis

The interviews and focus group were recorded and transcribed verbatim. The qualitative analysis software Nvivo was used to analyse all the data. The approach was a combination of data-driven and theory-driven coding. This allowed for analysis of the data based on previous theory and for the analysis to surface new ideas and themes that came directly from the data. The coding framework drew on previous work on modes of participation and policy stakeholders’ ideas about the role of public participation and qualitative data in policy making (Hill O’Connor et al, 2023) and used existing good practice principles to understand the extent to which the People’s Panel process aligned with them. Combined with data driven coding, this was a flexible approach designed to capture new themes not covered by previous theory. For example, there was a key theme from the data that highlighted the different approaches required to support groups where there are very diverse perspectives.