Health Board Elections and Alternative Pilots: Final Report of the Statutory Evaluation

2 Methods

The Scottish Government set us eight objectives for this research.

This section explains how we have addressed them.

Statutory Aim 1: Assess the level of public participation in Health Board elections

Candidate experiences

2.1 We wrote to all candidates for the elections, in both Dumfries and Galloway and Fife, asking to arrange a short interview prior to the election. We interviewed all candidates who consented. We discussed their background and motives for seeking election, their experience of the election process, and their expectations about their role if they were to be elected.

Voter turnout

2.2 We selected a random sample of 6000 names from the electoral register, 3000 from Fife and 3000 from Dumfries and Galloway. These samples represented roughly 1% and 2% of registered electors, respectively. We examined those electors' behaviour in three ways.

2.3 We sent a survey form to these electors' registered addresses within a few days of the election, and followed up with two reminders if required. Our survey attracted a 31% response rate, which is a fair response for a survey of this type. The survey asked electors whether they had voted in the Health Board Elections, and the General Election for comparison, and why (or why not). It also asked how much information they had about the two elections, gauged their level of interest in the two elections, and collected data on age, sex, ethnicity, education, length of residence in the Board area, disability, carer status, dependent children, general health and contact with the NHS. All forms were marked with an identifying number which allowed us to link electors' responses to their entry on the official marked register.

2.4 We then examined the entries for those 6000 electors on the marked registers for both the Health Board Election and the General Election. The identifying numbers on the forms allowed us to link returned survey forms with postcodes, and by comparing these postcodes with the Scottish Index of Multiple Deprivation we were able to test whether the responses were coming disproportionately from members of the sample of 6000 who were living in affluent or deprived areas. This gave us an indication of how far respondents to our survey were representative of the population. Examining the marked registers also enabled us to identify respondents who had not recalled whether they voted or not accurately.

2.5 Finally, we conducted in-depth interviews with 20 electors.

Young electors

2.6 These were the first Scottish elections in which 16- and 17-year-olds were eligible to vote. Recognising that young electors' experiences of the process might be distinctive, we paid particular attention to the views of 16- and 17-year-olds. No respondents to our survey who fell into this category responded to our requests to arrange in-person interviews. We therefore contacted some members of youth organisations and arranged focus groups with students in appropriate schools and colleges. These focus groups probed students' knowledge about the elections and views on how far they met the needs of young electors.

2.7 A total of eight focus groups involving 59 pupils were conducted: three in two secondary schools in Fife and five in two secondary schools in Dumfries and Galloway. We chose the schools that we approached with the objective of maximising the difference between their catchment areas. In Fife this was an affluent catchment area, and a catchment area with significant levels of deprivation. In Dumfries & Galloway, groups were conducted in a town centre school and a rural school. Students were in 5th and 6th year, and between 16 and 18 years old. Focus groups took place in class time, during the timetable slots allocated to Personal and Social Education or Modern Studies. Focus groups took place between November 2010 and February 2011. This was longer after the election than ideal, but delayed by the process of gaining ethical approval and agreement from schools.

Statutory Aim 2: Assess whether having elected members on Health Boards led to increased engagement with patients and other members of the public or improved local accountability.

Board interviews and observations

2.8 Interviews with Board members were a key component of our research strategy. Existing members were clearly in a strong position to observe any changes in how the Board operated, including increased engagement and improved accountability. We explicitly raised the issue of the relationship between the Board and members of the public in our interviews. Newly elected members were asked about their behaviour since being elected to find out whether they related to the local population differently from other non-executives in the past or their new colleagues. They were also asked specifically about what they perceived their relationship with the public to be, and to whom they saw themselves as accountable.

2.9 We interviewed members of the pilot Health Boards before the elections to establish a clear picture of how each individual Board was operating at that point. We interviewed both Executive and continuing Non-Executive Directors after the elections had taken place to find out what they thought had changed, and we interviewed all elected members at least once. We eventually interviewed most of the members who were in place both before and after the elections, and all the elected members^[4]. We also contacted key non-Board staff and stakeholders.

2.10 We regularly observed public Health Board meetings, as well as private events such as committee business and development sessions by negotiation with the Board Chairs. These observations informed our interviews.

2.11 In addition, we conducted selected interviews in Tayside Health Board to keep abreast of the kind of development in public and patient engagement taking place across the NHS.

Media monitoring

2.12 We reviewed press coverage of the pilot Boards between March 2010 and December 2011 to find out whether the pilots were having an impact on coverage. We used the NHS Scotland Media Monitoring database (www.media.scot.nhs.uk) to pick up coverage from the BBC, ITV, the national newspapers (the Sun, Daily Record, Scotsman, Herald, Daily Mail, etc.) and some of the local newspapers (the Courier, the Evening News, and the Press and Journal). This database also picks up some press releases from the Scottish Government and large charities. In both Dumfries and Galloway and Fife the local print media are highly fragmented, with many local newspapers covering individual communities. The media monitoring service does not cover these. We accessed print copies of these newspapers and made copies of articles that referred to the elections, discussed or quoted Board members, or raised issues that were relevant to our research.

Statutory Aim 3: Estimate the cost of holding the pilot Health Board elections and the estimated cost of holding future Health Board elections in all Health Board areas

2.13 We obtained figures for the costs of the pilots from the pilot Health Boards, the Dumfries and Galloway and Fife Returning Officers' staff, and the Scottish Government Public Appointments Unit. We consulted with experienced Returning Officers' staff, OCPAS assessors and civil servants for information on the details of election and public appointment processes and their cost implications. We have then estimated the cost of any national rollout based on the pilot figures. The details of our calculation are presented in the section on 'Costing'.

Statutory Aim 4: Review existing research evidence on elected health boards from elsewhere and draw out implications for the development of pilots in Scotland

2.14 We conducted a review of both academic and policy literature on elections to bodies analogous to territorial Health Boards in other jurisdictions. That review is being published alongside this report and contains its own methodology section.

Statutory Aim 5: Examine the process of implementation of, and arrangements for, the pilot health board elections

2.15 We examined the voting materials and attended one of the counting sessions. We have examined media coverage of the Health Board elections, and interviewed Returning Officers' staff to gather their views on the implementation of the elections. We also invited candidates in the elections, key staff of the Health Boards, stakeholders and a number of voters to discuss the election process in their interviews

Statutory Aim 6: Impact of alternative pilots

2.16 Our examination of the alternative pilots was broadly similar to our approach to the elections, apart from the analysis of the election process and results. We observed Health Board meetings, committees and development sessions both before and after the new members were appointed. We monitored media coverage of the alternative pilot Boards, and surveyed applicants for the new non-executive posts ^[5]. We raised the issue of the alternative pilots with stakeholders. Perhaps most importantly, we arranged in-person interviews with existing members of these Boards to find out about any changes they had noticed over the course of the pilot, and interviewed the new members to find out about their personal experiences.

Statutory Aims 7 and 8: Strengths and weaknesses, costs and benefits of both the pilots and alternative pilots and implications for future models of rollout of elections and models for public engagement and involvement in NHS Scotland.

2.17 We have prepared a summary of what our respondents have seen as the key strength and weaknesses, costs and benefits of the pilots in this report. As requested, we have also included respondents' reflections on the execution of pilots so that, if Parliament does choose to roll out any of the pilots nationwide, improvements can be made based on past experience. We conclude by offering suggestions for improving the pilots. These are either ideas that were suggested to us during the course of our research and that we find plausible or they are ideas that arise from own experience and knowledge of the literature.