Health and social care - data strategy: consultation analysis

Part 3 – Empowering Industry, Innovators and Researchers

The final part of the consultation explored opportunities to use high quality data for innovation, industry and research, which can support the delivery of health and social care services. Questions covered access to data for research and innovation, the infrastructure needed to ensure timely and secure access to data, and Artificial Intelligence (AI).

Access to data for research and innovation

12A. When considering the ethics of accessing health and social care data for commercial, development and research purposes, how do you think health and social care data should be used by industry and innovators to improve health and social care outcomes?

Q12A received 109 open responses where respondents set out ways in which data could be used to improve health and social care outcomes. Suggestions included using health and social care data to: aid research into new and innovative healthcare solutions; evaluate the impact of health and social care interventions; and monitor population health trends to inform future policy and planning. These broad themes align with the reasons respondents said they would be willing to share their data at Q4B.

Many responses to Q12A related to other sections of the consultation paper. For example, several respondents emphasised the need for strict data protection protocols, and others argued that health and social care data should not be used by industry solely for commercial purposes e.g. advertising, developing and selling products and services, or insurance, or for any activities which promote harmful behaviours e.g. smoking. Some noted they were not wholly opposed to commercial bodies profiting from the use of population data if there were demonstratable benefits for the public or patients. Such comments have been included within the analysis of other more relevant questions.

Advancements and innovation in healthcare

The most prevalent theme was support for using health and social care data to drive medical innovation and advancements through research. Respondents felt that access to large, aggregated datasets could aid the development of new medical technologies, interventions and treatments, which could lead to improvements in the public's health and quality of life. Some noted specific examples of advancements in disease diagnostics and treatment which could be realised through the utilisation of health data. A few noted that health and social care data could be used by industry to improve the efficiency and speed of clinical trials, by aiding the identification and recruitment of candidates.

"Data collected by the [health and social care] system is fundamental for progressing medical research. It is essential that innovators have access to [health and social care] data to drive insights into fundamental biology and natural history of disease; identify of risk factors associated with disease; uncover potential opportunities and targets for intervention; and ensure that services and interventions are as efficient and effective as possible." – Cancer Research UK

"From a [research and development] perspective, exemplar projects such as Artificial Intelligence Assisted Capsule Endoscopy building on SCOTCAP will potentially be a game changer for how to optimise use of [Artificial Intelligence] for Colon Capsule Endoscopy Images as a part of future bowel cancer diagnostics." – The Digital Health & Care Innovation Centre

Evaluating the impact of health and social care interventions

Many respondents suggested that health and social care data should be used to evaluate the long-term impact of health interventions. Findings of such analysis could lead to the optimisation of services and treatments and be used to improve patient care in the future. A few described access to data as being critical for assessing the performance and cost effectiveness of medicines, interventions and technologies in clinical practice. The Office for Statistics Regulation also called for the strategy to include using data to hold governments and service providers to account.

"The wealth and scale of clinical and administrative data that is collected from patients provides 'real world evidence' of how medicines, devices and interventions that patients receive actually have the desired outcomes amongst the populations they are intended for… This is a major benefit stemming from having population level clinical and administrative health and care data available in a timely and safe way for research, with the ability for efficient data linkage to track patient pathways through the heath and care system." – Research Data Scotland

Shaping policy and planning

Using health and social care data to monitor population trends such as disease prevalence, social care demands, and use of services was suggested by several respondents. It was noted that analysis of such trends could be used to forecast future need and demand and inform government policy, investment and service planning.

"The RCOT believe that health and social care data is vital for future planning. It's particularly important to gain a better understanding of the needs of our workforce and communities and the workforce planning and training which will be required to meet those needs." – Royal College of Occupational Therapists

Importance of collaboration

Some respondents stressed that optimising the application of health and social care data will require effective collaboration between Government, the NHS, social care and industry. A few cited examples of existing models which have successfully allowed health and social care data to be used to support innovation and industry collaboration, including OpenSAFELY, Early Access to Medicines Schemes (EAMS), ChemoCare and DataLoch.

Other themes

Some supported granting industry access to health and social care data as they felt it could lead to the creation of jobs in the medical research sector. A few suggested that health and social care data could be utilised to address inequality but did not set out any specific actions or applications of data which could be used to achieve this.

12B. How can industry and innovators maintain the trust and confidence of the people of Scotland when using their health and social care data for research purposes?

Q12B received 106 responses. The most common suggestion for how industry and innovators can gain the trust and confidence of the public when using their health and social care data was through being open and transparent about how their data will be processed. These comments have been included in the analysis of Q12D, which focussed specifically on transparency. Other suggestions for how trust can be built and maintained between industry and the public are set out below.

Anonymity

Many respondents suggested that trust could be built by ensuring that researchers only have access to aggregated and non-identifiable data. They reasoned that this is the best way for researchers to guarantee the confidentiality and privacy of patients.

"Data should be anonymised wherever possible to protect the confidentiality of citizens." – East Renfrewshire HSCP

Compliance with data protection legislation and ethical approval protocols

Another prominent suggestion for how industry can improve trust and confidence was through strict compliance with data protection legislation and ethical codes of conduct. Examples of regulations, schemes and ethical protocols cited by respondents included: GDPR, the Public Benefit and Privacy Panel for Health and Social Care (HSC-PBPP), Disclosure Scotland and independent ethical approval panels or academic ethics boards.

"The ability to access data by applying to an independent, expert ethics body would be a good way to maintain trust for the public that their data is being used properly, and in a way that will serve the public good." – Individual

"While it is accepted that the government relies on the private sector to innovate, a new legal framework needs to be developed by considering a cross-section of laws which includes data protection law, the law of confidentiality, IP, competition and company laws." – School of Law, University of Leeds

Emphasise benefits of using data

Some felt that trust could be built through innovators and industry making greater efforts to communicate the benefits of using the public's health and social care data. For example, industry bodies could demonstrate how their research aligns with public health priorities, or its contribution to medical breakthroughs and advancements in technology and treatments. Some called for greater collaboration between industry, NHS, government and academia in communicating the benefits of industry access to population health and social care data.

"Demonstration of the benefits of accessing data and how innovation has driven service improvement will improve public trust in how their data can be used for a good purpose." – The Innovative Healthcare Delivery Programme

Choice and consent

Informed consent was stressed as an important factor in building trust between the public and industry. Respondents suggested that industry access to public health and social care data should be based on individuals' choice of whether to opt in to a particular study or not, allowing the public to feel more in control of how their data is used.

Less commonly mentioned themes

A few other suggestions for building trust between industry and the public included:

• Establishing a centralised, national approach to information governance.
• Improving public knowledge of data subject rights through education and public awareness campaigns.
• More consultation with the public to understand concerns relating to the use of health and social care data in research, and how these can be addressed.
• Audits of industry bodies processing population level health data.
• Industry bodies increasing investment in cyber security.
• Introducing greater financial penalties for misuse of data or data breaches.

Some respondents emphasised the critical importance of building trust between the public and industry in order to make medical advances through research.

"Public trust is vital if we are to fully harness the unique potential of our health and care data." – Scottish Industry Life Sciences Group's subgroup on Digital & Data

A few individuals expressed a view that there was nothing that could be done to increase their trust in industry bodies.

12C. What do you believe would be unacceptable usage of Scotland's health and social care data by industry, innovators, and researchers?

Using data for commercial gain

Among the 104 responses to Q12C the most common objection to the use of health and social care data by industry was where the data would be used solely for commercial purposes or financial gain. Specific examples included using data for: market research and targeted advertising; developing and selling products and services; and predictive technologies such as insurance. Some noted that they were not wholly opposed to commercial bodies profiting from the use of population data if there were demonstratable benefits for the public or patients.

"Anything that is purely commercial (e.g. wanting to understand their market share of a medicine, rather than measuring the benefits and harms of different medicines in a class). The driving principle is that there should/could be benefit for the Scottish population from the analysis. Not just benefit to the commercial operation funding it." – Institute of Genomics and Cancer, University of Edinburgh, Western General Hospital, Edinburgh

Personally identifiable data

Many respondents felt strongly that industry should not be given access to people's health and social care data if it has not been aggregated or anonymised, or if it can be used to identify individual patients or service users. A few added that it was essential to ensure data is not used for any purpose that could lead to individuals being negatively impacted, for example, sharing medical history with government departments or insurance providers.

"Sharing or transmission of any personally identifiable information must never happen." – Individual

Activities not in the public interest

Several respondents felt it would be unacceptable to grant access to population health and social care data for any research that is not in the public interest or does not have the support of NHS or the government. Some felt data should only be shared with industry bodies and researchers who can demonstrate clear benefits to the public and have appropriate safeguards in place. Some emphasised that data should not be used for any research that promotes harmful behaviours such as smoking or alcohol consumption.

Other unacceptable uses

A few other unacceptable applications of data included:

• Any research that has not been through a robust ethical approval process.
• Any research that does not fully adhere to all legal, regulatory, privacy and security obligations.
• For any purpose outwith what the data subject has explicitly consented to.
• Purposely manipulating or misinterpreting data which leads to misleading findings.

12D. How should industry, innovators and researchers be transparent about their purposes in accessing, and the benefits of using, health and social care data?

Make clear how data will be used

The most common suggestion across the 101 responses to Q12D and the relevant comments from Q12A for how industry, innovators and researchers can be transparent about their use of health and social care data was through making clear to data subjects the aims of their work and the intended use of data. Respondents suggested that researchers do this through direct communications with data subjects or by publishing the relevant information online. A few urged researchers to be transparent about conflicts of interest and commercial interests when accessing the public's health and social care data.

"Relevant researchers, innovators etc should be required to set out a clear scope which describes the purpose, breadth and intended outcomes of their use of health and social data." – Renfrewshire HSCP

Publish information about data security and ethics

Several respondents suggested that researchers should publish information about their data security protocols and the ethical approval processes they have navigated. Respondents called on researchers to clearly articulate how data will be kept safe and secure, and evidence how they are meeting the key principles of GDPR.

Publish results or findings

Some respondents felt that researchers accessing health and social care data should be required to publish their findings. A few felt this should be required regardless of whether the intended outcomes had been met or the study has been a success. A small number suggested case studies would be a good way to communicate findings to the public.

Accessible information

Respondents discussed the importance of researchers providing accessible information about their use of health and social care data. For example, some stressed that research findings should be free to access and not hidden behind paywalls and should have content that is easily understood by the public, i.e. not overly technical or jargonistic. A few added that such information should be available in a range of accessible and tailored formats, including hard copies, large print, Braille and BSL.

"It is important that industry, innovators and researchers are able to clearly articulate – using accessible language – their purposes in accessing health and social care data and the subsequent benefits to patients and the public at large." – The Cloud Consulting and Technology Association

Other themes

Some other themes were present among responses to Q12D:

• Many stressed the importance of transparency, sharing concerns about low public awareness of how health and social care data is used.
• There were calls to introduce channels for the public to ask research bodies questions about their use of health and social care data.
• A few suggested resources and frameworks which researchers could use for guidance on transparency including: the Association of the British Pharmaceutical Industry's principles for analysis and use of health data, the 'Five Safes' framework and Microsoft Responsible AI principles.

Infrastructure

13A. We want to create an infrastructure that supports access to data for research and innovation in a safe, secure, and transparent way, how should the Scottish Government seek to store and share health and social care data for research in order that it can best facilitate easier access that is still safe and secure?

Infrastructure considerations

Reflections on the value and role of safe havens or trusted research environments was the most prevalent theme across the 99 open responses to Q13A. As well as straightforward endorsements of the use of safe havens to store and share data, some described how safe havens should be resourced, managed and structured. Suggestions included greater investment, for safe havens to work together or with other partners, access to the latest technology, to learn from the private sector or be led by Research Data Scotland. The benefits of using safe havens were also stressed, including improved data security, scope to provide approved researchers with access to linked, unidentified data, and building public trust in the use of their data for research.

"Build on and invest more in what we already have and works well. The usage and expansion of the current national and regional Safe Havens infrastructure to provide a more comprehensive approach to the collation and effective curation of health and care data in Scotland seems a sensible way forward. This may require a mandate from Government for public sector bodies to capture and effectively share local and regional information." – The Digital Health & Care Innovation Centre

Ensuring infrastructure complies with international standards was raised by several respondents who noted this would enable collaboration. Specifically, one advocated an approach which ensures data sets are scalable across countries, noting that commercial partners will gravitate to data sets which are pan-country scalable. Another mentioned a need to share data across the UK to generate large data sets on rare diseases. One suggested the Scottish Government considers becoming a participant in the European Health Data Space, as this could present opportunities to access additional health data.

Interoperability was mentioned by some respondents, mostly stressing the need for this to be a key feature of the infrastructure. A few noted that use of international standards would ensure interoperability. A need for stakeholders to play a role in the infrastructure design was highlighted by some respondents; some also advocated patient involvement while others more broadly stressed the need for a partnership approach across sectors.

Some respondents reflected on data storage, typically noting that data would be stored on a cloud, and that this could create timely and effective access to data, security, control over access, computational power, and opportunities for researcher collaboration. A small number commented on open platforms, calling for open data which is accessible and, for example, can be used by small organisations and independent researchers. A few stressed the need to stay aware of evolving technology.

A less commonly mentioned theme was the importance of independently auditing Scotland's infrastructure, for example to test security and vulnerability and the timely access to data and approval processes.

Small numbers commented on risks and challenges of developing data infrastructure. One stressed the data infrastructure could not cover all relevant information, noting the importance of genetic, lifestyle and environmental factors. A few described delays and frustrations with current data systems and their hope that a new infrastructure could remove existing barriers.

Data access and usage considerations

The second most prevalent theme across comments was discussion of access to data. Different views were evident in comments on ease of access to data – some argued that processes should be clear and simple to engage with, others stressed that security should override ease of access. Respondents emphasised the need for a secure system which minimises the risk of breaches or data misuse, suggesting registered users or regulated access, access and usage tracking, the protection of identities within the data set and robust firewalls, encryption and sophisticated approaches to security. Some suggested that data sets should be available at national, health board or hospital level.

While calls for the use of aggregated or anonymous data was another prevalent theme, a small number called for pragmatism, suggesting data sets could be linked while maintaining confidentiality. Some comments on anonymity included discussion of need to use technology and governance standards to enhance security and trust.

Different aspects of transparency were identified in some responses. This included: calls for transparent language, processes and data sharing agreements; transparency on what data is held, who is accessing it and why; for results to be published; and clarity about the role of any commercial partners. Small numbers stated that data sharing agreements should be transparent, but not so laborious that they impede engagement with data.

The role of the Scottish Government and others

The role of the Scottish Government in establishing the infrastructure was the third most common theme. Respondents called on the Scottish Government to: provide clarity on key issues such as responsibilities, governance and security; allocate resources to build the framework or expand existing resources; and establish or allocate responsibility to a national body to manage and govern the infrastructure.

On the latter point, very small numbers suggested a recognised national information governance office and a single set of standards that all Health Boards comply with for research, or one accountable organisation to store health and social care data which has a national, statutory, and legal responsibility. Some talked generally about the role of a dedicated service, noting this would allow access and provision of data to be managed and scrutinised, and referenced the resourcing, training and standards that such a service would require. A small number, including Public Health Scotland, mentioned Research Data Scotland and described the value of its work; another called for further investment to expand the reach of its work and engagement with networks.

"There should be a designated body which has responsibility for managing all aspects of the access to health and social care data such as ensuring the necessary procedures and safeguards are in place e.g. relevant policy and legislation, criteria for gaining access, management of onboarding and offboarding and regulation of participants. Any infrastructure should not be replicating the storage of large amounts of data but ensuring that the infrastructure in place intelligently manages access in an efficient manner." – Aberdeen City HSCP

Other less commonly mentioned themes

• On governance, GDPR and standards, a few respondents stressed the need for data storage to comply with GDRP, for Data Protection Impact Assessments (DPIAs) to be publicly accessible, and to consider how data governance frameworks may evolve across the UK, EU and wider world.
• Comments on a single person identifier highlighted that this would allow different systems to connect and make data linkage robust and comprehensive, highlighting the potential to get greater value from the CHI number through which researchers can follow patients from birth to death.
• Benefits stemming from greater control through a single point of access, such as accountability and oversight, standard access processes, and high levels of security.

13B. What do you believe are the key data needs and gaps that are faced by industry, innovators, and researchers when it comes to Scotland's health and social care data?

Of the 85 responses to Q13B, most comments related to challenges that industry, innovators and researchers have in accessing data in the health and social care sector. These responses explored issues such as the disparate datasets held by different organisations, data linkage, complex procedures to request data access, and concerns around data quality. There were also some specific data gaps identified in responses.

Challenges

The most common theme was the disparate datasets held by different organisations in the health and social care sector. Many respondents described a lack of integration and interoperability among different organisations' systems, causing difficulties in accessing, sharing and collating data between organisations.

"A key data need and current gap is the ability for real time data sharing and the reality of fragmented data. We do not know much is presently being missed – in terms of intelligence and opportunity – by not having right information in the right place at the right time." – Scottish Care

"ABPI members highlight a lack of interoperability between datasets (stifling potentially valuable analysis and insight), inconsistent access arrangements and a generally fractured data environment as the key barriers holding back data driven innovation in the UK." – ABPI

Some respondents emphasised the importance of data linkage and that more needs to be done to improve data linkage in health and social care. Comments focused on the value of data linkage in tracking data held about individuals in different datasets; this enables analysis of a patient's journey through health and social care, the outcomes achieved and the factors that determine health outcomes.

"Linkage of data for individuals across a period of time is critical for many industry, innovation and research projects. However, this is currently very challenging to do with truly anonymised datasets." – Community Pharmacy Scotland

Procedures for requesting access to data were another challenge highlighted by some respondents. Comments focused on a need for easier and quicker processes to allow researchers, industry and innovators to request access to data, as well as difficulties caused by variations in procedures across different Health Boards.

"Simpler and timelier processes are required for requesting, accessing and gaining permission and approvals for access to data." – Cancer Medicines Outcomes Programme (CMOP)

Some respondents cast doubt on the quality, accuracy and completeness of health of social care data. This has been examined in detail across Part 2 of this report.

"Poor quality of data, poor integration and communication between systems, poor allocation of the necessary technology. This creates an unnecessary burden on researchers and health and social care professionals who require additional time to access and use data that could be used more productively." – ENRICH Scotland

Other challenges, each identified by a few respondents, included: gaining timely access to data; public understanding of health and social care data; issues around governance, security and anonymity; organisational culture, leadership and resistance to change; gaps in digital literacy skills including data analysis; lack of awareness of what data is available; and resources to collect and analyse data. Two mentioned challenges associated with bureaucracy, and one said there was a lack of data available in digital format.

Data gaps

Respondents identified a wide range of data gaps, but there was little consensus. The most identified gap related to protected characteristics and other information about inequalities such as socio-economic data. Some felt there should be more of a focus on collecting and analysing data about these characteristics.

"Currently data on ethnicity, gender, sexuality, socio-economic status, caring responsibilities and disability are not routinely collected as part of health and social care data. To fully understand the extent of health inequalities in Scotland the collection of data on inequalities and marginalised groups needs to be significantly stronger." – Voluntary Health Scotland

A few noted a need to improve data in the social care sector specifically. For example, COSLA noted gaps in social care workforce data, data around unmet needs in communities, and about choices and support being provided under Self-Directed Support. SCLD (Scottish Commission for People with Learning Disabilities) cited the 2020 Office for Statistics Regulation's review of adult social care statistics in Scotland, which identified gaps in data collection, resources weighted towards health data, inconsistencies in data definitions, poor data quality, delays in reporting and inaccessible data.

Gaps in data about primary care and patient outcomes were mentioned by a small number of respondents. GMC Scotland (General Medical Council) identified a gap in their understanding of the risks and challenges of clinical practice resulting from limited data availability. Other themes where respondents felt more data should be collected included: secondary care services; data on long-term conditions including cancer and kidney disease; patient experiences; the scale of the health and social care workforce; and drug safety and clinical trials.

Further suggestions, each made by one respondent, included: waiting times; disabilities; rare conditions; medical imaging data to support diagnosis; health and social care services provided by charities or religious organisations; research related to the work of Allied Health Professions (AHPs); and product usage.

Innovative technologies

14A. Used appropriately and well, technologies such as Artificial Intelligence can help to improve decision making, empower health workers and delivery higher quality health and social care services to citizens, improving how you receive health and social care services. What are your views on the benefits of using AI to improve the delivery of health and social care services?

Three types of comments were evident in the 103 responses to Q14A. Many respondents expressed an overall view for or against the use of Artificial Intelligence (AI) or elaborated on the perceived benefits. Further detail on these comments is provided below. Many other respondents highlighted concerns about the use of AI and commented on safeguards around its use; these responses are included in the analysis of Q14B.

Overall views on AI

Mixed views were expressed on using AI, though on balance the majority of respondents were in favour. Many individuals and organisations expressed a generally positive view, encouraging the use of AI. Several caveated their agreement with an assumption that AI would be used safely and alongside some of the safeguards outlined at Q14B.

"There's no question that we've reached a point that AI is not only useful but that denying its use for the population of Scotland would be unfair." – Scottish Clinical Imaging Network

"The potential for AI's role in improving health and social care services is vast, and we are fully supportive of this to continue." – Advance Care Research Centre (ACRC)

Several respondents expressed some support but stressed that AI is only appropriate in certain circumstances and that the introduction and use of AI should be carefully considered. They typically argued that clear parameters should be set out for its use which consider what is of most value to services and services users, and an assessment of any potential risks or unintended consequences. A few respondents argued that AI should only be seen as a tool to assist humans in decision making and should not be relied on.

"It is critical that the narrative and framing around AI use in healthcare is set firmly from the outset – that it is a very promising supportive tool, but no replacement for complex clinical decision-making." – Community Pharmacy Scotland

Some argued that the effectiveness of AI still needs to be proven, and a few expressed clear opposition to AI, arguing that there is unjustified hype around what it can achieve.

Benefits of using AI

The most mentioned benefit, particularly by health bodies and HSCPs, was the use of AI in diagnostics. Respondents highlighted how AI could result in earlier diagnosis and better treatment. Some specifically noted the use of AI in image analysis in cancer detection and dermatology, for example. Others described how AI could detect exceptions or anomalies in health data, which could lead to conditions being identified and preventative measures being put in place. A few noted that AI could be used as a triage tool, or stressed that it should be used to complement, rather than replace, decisions made by clinical professionals. Related to this, a small number noted the potential to use AI to predict future risk. While most comments related to diagnosis focussed on healthcare, Scottish Care cited their 2020 TechRights report which details some examples where AI is already supporting citizen independence and preventative care approaches in Scotland.

"AI can be particularly useful in recognising anomalies in images that are hard or impossible for the human eye and this can flag to a human radiologist for example cases that should be followed up during screening services. Given the massive learning capacity of deep learning algorithms, it qualifies them to handle such variance and detect characteristics well beyond those considered by humans. Moreover, the use of AI in digital pathology setting to make predictions regarding treatment response can enable the selection of more effective treatments for patients" – Cancer Research UK

Another prevalent benefit was that AI could enable more efficient use of resources within health and social care, allowing staff to provide enhanced clinical care rather than routine tasks. This could result from automation of some processes and using AI in diagnostics as above, or from the ability to better predict the need for resources as below. This was again commonly mentioned by health bodies and HSCPs.

"Speech and text recognition are already employed for tasks like patient communication and capture of clinical notes, and their usage will increase. It also seems increasingly clear that AI systems will not replace human clinicians on a large scale, but rather will augment their efforts to care for patients. Over time, human clinicians may move toward tasks and job designs that draw on uniquely human skills like empathy, persuasion and big-picture integration." – NHS 24

Some respondents described the potential for AI to detect relationships and patterns in data, particularly large data sets, and to derive additional insight from that analysis. Respondents noted that it would be impossible for people or traditional IT to analyse and draw conclusions from the same volume of data, or do so quickly.

Using AI for planning, forecasting and decision making was another theme mentioned by some respondents. The analysis of large data sets or population data could be used to predict future demand, plan and target service provision, and manage public health.

14B. What safeguards do you think need to be applied when using AI?

Governance, regulation and ethical approval

The most common safeguard mentioned across Q14A/B was the need to ensure AI is used in line with all relevant governance, regulation and codes of conduct to mitigate risk and avoid harm. Others noted the need to have appropriate legal and ethical frameworks in place and regularly reviewed, while a few specifically suggested that some AI could or should be classed as a medical device and be subject to regulatory approval. Some reiterated the need to comply with appropriate data security and privacy regulations, and for the transparent use of data and presentation of any outcomes or results from using AI.

"The Scottish Government's AI strategy nicely sets out the safeguards about using AI well, with data ethics and transparency at its heart." – Research Data Scotland

"For the full benefits of AI technology to be realised, they need to be supported by trustworthy and cybersecure data, combined with responsible use. Appropriate data governance and people consent are key to fully seize this opportunity in a sustainable way." – Microsoft

Human oversight

Another prevalent theme was the need for human oversight of AI. This would involve a 'human in the loop' to review of algorithms and testing for bias, peer review of decisions resulting from AI and the potential to intervene should AI go wrong. A few respondents called for clinical staff and expert input into the design of AI models and algorithms, and in the review of results. This could be from epidemiologists, radiologists, statisticians etc. A few called for training but did not elaborate on what is required.

"Use of AI should be informed by a suitable trained health care professional to intervene if something may harm a patient. However, AI requires clinical input to interpret correctly and findings / outputs. AI can only empower health workers if the health workers have a say in what is planned or implemented, how the technology will be used and how the outputs may affect care." – Cancer Medicines Outcomes Programme (CMOP)

Bias

Several respondents highlighted the importance of recognising and addressing any bias built into AI which could skew the results. Respondents described how bias could result from unconscious bias held by programmers, or from incomplete data being used. Some respondents, including the University of Edinburgh and Health Data Research UK, specifically highlighted the potential for bias against marginalised groups, in particular ethnic minorities, who may be less represented in data. The Health and Social Care Alliance Scotland recommended carrying out Equality and Human Rights Impact Assessments to fully consider the impacts of AI on different population groups.

"As highlighted by a breadth of literature, traditional artificial intelligence development and training methods can often entrench bias and prejudice within AI systems, negatively impacting Black and minority ethnic groups and, in particular, BME women. If AI is to be incorporated into health and social care decision making processes, every precaution must be taken to ensure that biases within the training, deployment and evaluation process are identified and eliminated." – Coalition for Racial Equality and Rights (CRER)

Testing and evaluation

Another recurring safeguard, raised by several respondents, was to regularly trial, monitor and evaluate the effectiveness of AI to ensure they continue to be fit for purpose.

Other considerations

While not specifically safeguards, many respondents outlined themes which they felt ought to be considered for AI to function effectively. Most commonly, several noted that AI will only be effective if it uses high quality, complete, accurate, representative and linked data.

"In order to benefit the most from machine learning and other more complex statistical models (aka AI), health data needs to be big, of good quality and linked effectively. Linking health data across all services will allow models to identify risk factors and build up a picture of how data points interact at a much deeper level. The more data it is given the richer this picture will be." – Office for Statistics Regulation

Engaging the public was mentioned by some respondents. They suggested that patients should be informed when AI has been used to make decisions about them, and that greater public awareness of AI and its benefits is needed to improve trust and confidence.

A few respondents – including Public Health Scotland, The Innovative Healthcare Delivery Programme, University of Edinburgh and Scottish Cancer Patient Reported Outcome Measures (PROMs) Advisory Group – called for investment in data infrastructure, so that Scotland has the technological capabilities to use AI effectively. This includes a better network of safe havens or trusted research environments for data sharing.

"There is a need to utilise significant data sets to ensure that AI applications are accurate and inclusive. The proposed approach to modernising Scotland's health and social care infrastructure and data curation will be crucial, if we are to unlock the benefits of AI and attract further research and innovation activity in this space." – Scottish Industry Life Sciences Group's subgroup on Digital & Data

Two respondents raised the importance of being aware of unintended consequences of AI; that AI does not drive inequalities because data is not available across the population, and that funding for AI is not diverted from other work or improvements.