Organ and tissue donation - opt-out system: five year monitoring and evaluation plan

Data Sources, Indicators, and Implementation Activities

Background

This section gives an overview of different data sources (indicators) that the monitoring and evaluation program intends to draw upon over the five year period, to track potential impacts on the Act's intended outcomes. It is organised according to the three strands: Public Attitudes, Workforce, and Monitoring Data.

This section also provides an overview of some of the activities conducted to support implementation of the Act. These activity overviews provide context to understand how potential impacts on particular outcomes might occur. They also serve as potential monitoring tools for particular outcomes. For example, Outcome 2 (increasing awareness of donation and legislation change among under-represented groups) is currently not evidenced by data sources, but it is being addressed through implementation activities centred around stakeholder engagement and media marketing about the Act.

Baseline Data

Baseline data was gathered (up to February 2020) to underpin this Monitoring and Evaluation Plan, so that the five year monitoring and tracking data can be compared to a 'pre' legislation implementation context. An overview of the baseline data can be found in Annex 2 and full details are in the Baseline Report.

Impacts of the COVID-19 Pandemic on Data

The Act was originally planned for implementation in Autumn 2020, however due to the COVID-19, implementation was postponed to March 2021. The pandemic appears to have had impacts on organ and tissue donation and transplantation activity in the UK. The NHSBT Annual Organ Donation and Transplantation Activity Report 2020/2021 reported significant drops in donation and transplantation rates in 2020, which were not consistent with activity trends prior to March 2020. Therefore, the baseline data against which this monitoring and evaluation plan will be measured covers the period prior to the onset of the COVID-19 pandemic, namely up until 29 February 2020.

As implementation of the Act was postponed to March 2021, this does create a missing year of data (March 2020 - March 2021), however given the significant contextual changes created by the pandemic, exclusion of this data will provide a more accurate baseline view. It should be noted that the implementation of the legislation (March 2021) occurred during continued COVID-19 pandemic restrictions, and that the pandemic may continue to impact donation activities during the monitoring and evaluation period.

Potential Monitoring and Evaluation Data Sources

A. Public Attitudes

Data: Public attitudes survey

• At baseline, The Scottish Government commissioned the Scottish Health Council to survey members of the public about organ and tissue donation, using the Our Voice Citizen's Panel^[5]. This provided an understanding of public awareness, understanding, and views on donation in general and about the legislation, as well as behaviours in relation to organ and tissue donation.
• The Scottish Government will undertake similar surveys at approximately two and five years post implementation.

Data: Marketing surveys

• The baseline data drew on five omnibus surveys that were undertaken to monitor impacts of a media marketing campaign (see below). The surveys looked at public attitudes towards the change to an opt-out system, as well as impacts of the media campaign.
• Future marketing campaigns will also be evaluated, although the specific timing of these surveys is yet to be determined.

Implementation Activity: Media marketing campaigns

• These activities included broader awareness raising work during the baseline period and the 'Big Change' marketing campaign, from January 2021. The aim of these activities was to inform the public of the donation law change and encourage people to register a donation decision on the NHS Organ Donor Register and tell their family members about their decision.
• Scottish Ministers are under an ongoing duty to raise awareness about the opt out system and donation in general, so there will be ongoing campaigns which will continue to be evaluated. These are likely to be run annually, although the specific timing and content of these campaigns is yet to be determined.

Implementation Activity: Stakeholder engagement

• The aim of this workstream is to raise awareness with harder-to-reach groups, and groups with specific interests in relation to the opt-out system, to incorporate their views into the implementation of the Act. The bulk of these activities have taken place from July 2019 to implementation.
• Future stakeholder engagement will be scoped as part of business as usual activities. Any activities will be recorded and monitored over the five year monitoring and evaluation period in order to contextualise outcomes within the Public Attitudes strand.
• This workstream offers the best potential monitoring information for Outcome 2 (increasing awareness of donation and legislation change among under-represented groups).

B. Workforce

Data: Qualitative research with NHS Scotland staff

• At baseline, a Scottish Government researcher conducted qualitative research (via interviews and focus groups) with NHS Scotland staff^[6] involved in donation. This gathered views on donation generally, the move to a deemed authorisation system, and what could help ensure it is a success.
• A scaled down version of this qualitative research is recommended during the five year monitoring and evaluation period to track changes in workforce attitudes and views.. Two potential options include:
  • Telephone or video interviews with a sample of NHS Scotland staff involved in donation.
  • An online survey sent to all NHS Scotland staff involved in donation (matching the staff groups listed).

Data: Confidence rating surveys pre- and post- training with NHS Scotland staff

• At baseline, online surveys were sent to staff to evaluate the impact of training undertaken in preparation for the implementation of the Act. These surveys were sent pre- and post- training, and measured changes in reported confidence around key areas of practice, as well as key learning sought by and gained through the training.
• In order to track levels of confidence over the five year monitoring and evaluation period, a survey replicating the baseline staff questionnaire will be needed. The key areas of practice to be tracked include:
  • Staff understanding of the new legislation
  • Explaining changes to an opt-out authorisation system to fellow colleagues
  • Following the correct procedures to ensure new legislative requirements are followed
  • Understanding of the changes being made around Pre Death Procedures in the Human Tissue (Authorisation) (Scotland) Act 2019
  • Explaining Pre Death Procedures to peers
  • Explaining the changes in legislation to patients and patient families

Implementation Activity: Training aimed at NHS Scotland staff involved in donation processes*

• The aim of this workstream is to ensure compliance and confidence with new legislative requirements among all operational staff involved in donation processes in Scotland.
• *It is unclear in what form these training activities will continue, beyond an online tool. An overview of training activities will be recorded and monitored over the five year monitoring and evaluation period in order to contextualise outcomes within the Workforce strand.

Implementation Activity: Donor family support

• Routine, informal support is offered to donor and potential donor families by SNODs as part of practice prior to the implementation of the Act. However, these activities are not tracked by SNODs, NHSBT or SNBTS therefore there is no published data source for this information.
• One approach for gathering information on this is to gather information from the Lead SNOD about the range of support offered to donor families in the year prior, or a short survey to be sent to all SNODs gathering this information anonymously. It is important to note that this would serve as monitoring information, not evidence against which to measure the success of outcome 16.

C. Monitoring Data

Data: Organ donation data

• Routine data is collected by NHSBT on referral, family approach, authorisation, donation, and transplantation rates.
• The NHSBT Scotland Organ Donation Services Team release statistical reports providing a summary of data relating to potential and actual organ donors, as recorded by NHSBT via the Potential Donor Audit (PDA), the accompanying Referral Record and the UK Transplant Registry:
  • Scotland Organ Donation Services Team: Detailed Report Actual and Potential Deceased Organ Donation 1 April 2019 - 31 March 2020
  • Organ Donation and Transplantation Activity Report 2019/2020
  • Organ Donation and Transplantation data for Black, Asian and Minority Ethnic (BAME) communities: Report for 2019/2020

Data: Tissue Donation Data

• Some routine data is collected by SNBTS on referral, family approach, authorisation, donation, and transplantation rates.
• Currently, this data is not routinely published, but SNBTS will seek to collect and share this data with Scottish Government for monitoring and evaluation purposes.

Data: Organ Donor Register data on opt-in and opt-out registrations and withdrawals

• Routine data collected by NHSBT and published within the Organ Donation and Transplantation Activity Report 2019/2020.
• Detailed Scottish specific data will be made available by NHSBT for Scottish Government monitoring and evaluation purposes.

Table A lays out which data sources address which outcomes. This shows that there is at least one source of potential data or information from implementation activities to monitor progress towards most outcomes. However, there is also uncertainty over the future availability of some sources of information, particularly for outcomes two and 16.

Table A: Measuring and Contextualising Outcomes by Information Source

Strand	Information Source	Outcomes
		1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16
Public Attitudes	Public Attitudes Survey	√		√			√	√	√					√
	Marketing Surveys*	√		√			√	√	√
	Academic and Public Sector Research**
Workforce	Qualitative research with NHS Scotland staff				√	√					√			√
	Surveys with NHS Scotland Staff				√	√					√			√
Monitoring Data	NHSBT: Organ Donation Data					√			√	√		√	√		√	√
	SNBTS: Tissue Donation Data					√				A		√	√		√	√
	Organ Donation Register data								√
Implementation (and potentially ongoing) activities	Activities in the section below offer monitoring information or contextual information about potential impact on the outcome, not data against which to measure impact.
	Media marketing campaign*	√		√		√	√	√	√	√	√
	Engagement with partners and stakeholders*	√	√ B	√		√	√	√	√	√	√
	Training aimed at NHS Scotland staff involved in donation processes*				√	√					√	√	√
	Routine, informal support offered to donor families by SNODs*																√ C

Notes:

*The specific details of information sources marked with an asterisk (*) are still be determined, as long-term business planning within Scottish Government and the NHS is still underway.

** Relevant academic research will be used where appropriate.

A: SNBTS is currently reviewing how it will collect this information and the availability of this data source will be monitored.

B: There is no currently available evidence to address Outcome 2 (increasing awareness among under-represented groups) – the only available information comes from this stakeholder engagement workstream

C: Outcome 16 (support for relatives) is not impacted on by the Act's package of implementation activities.

√ Boxes are marked with a tick to indicate that a particular information source serves as an indicator for a particular outcome of the Act.