Data standards in Scotland's public sector - framework for action: consultation analysis

2 Data standards in Scotland's public sector

Digital technologies are a mechanism by which Scotland can pursue its National Outcomes. The importance of effective digital infrastructure and technologies will be at the heart of a new Digital Strategy for Scotland which is due to be launched in March 2021. However, Scotland's digital ambitions are dependent on data; and access to data that is ready for reuse is dependent on data standards.

Scotland's public sector ambitions are currently being pursued in a less than ideal data environment. The public sector in Scotland is structurally fragmented and approaches to data collection, management, and use are diverse. This creates a patchwork of hard to connect, or even find, data assets, which in turn makes data reuse challenging.

Unlocking the potential

Unlocking the potential of Scotland's public sector data is vital if Scotland is to succeed in its economic, social, and environmental ambitions. Widespread adoption of data standards which support the creation and management of high-quality data, and enable public sector data to be reused, is essential to create the environment for success. The cost of not doing so is hinted at in a comment from a survey respondent "Current data practices fail to fully utilise the investment by the taxpayer in the data".

When data standards are applied to data, the standards operate as virtual bridges that can connect disparate data sources together and can connect potential users to the data. These virtual bridges enable the integration of data from different sources and improve access to data for reuse. A survey respondent noted "organisations can't work effectively together if they can't easily share appropriate data". The value of data standards is that they create better and more accessible data which can be reused. If public sector data assets are of sufficient quality and available for appropriate reuse, the following benefits are expected:

• better understanding of a problem or situation which can inform the design of better, quicker, and more effective responses;
• a more informed public sector, and more evidence-based decision-making at a local and national level;
• enhanced digital transformation of public sector services, and consequently a better experience of public services for Scotland's residents;
• greater transparency, accuracy, and value in the delivery of public sector services; and
• more R&D and innovation in the design and delivery of public sector services.

Furthermore, if data standards can simplify the process of access, consent, and interoperability, and create machine-searchable data and metadata, many hours, days, weeks, months of time could be saved across each of Scotland's public sector organisations. Time which is currently deployed in finding, fixing, and manipulating data from diverse sources – a process which may be duplicated several times for similar purposes.

The pursuit of such benefits, and others, is driving the activity described in the various case studies in Annex Report B.

• The National Digital Platform (case study B) is creating a single data repository for core clinical data in Scotland. The ambition is to improve care, reduce cost and mistakes, and support more R&D and innovation in the delivery of health and care services.
• The RESPECT project (case study C) is seeking to significantly improve the end-of-life care provided to Scotland's residents through better information (data) flow.
• Police Scotland (case study D) is using new data management capabilities to drive organisational improvement and support Police Scotland's objectives, creating 'easy access to a single view of trusted, linked data'.
• The Community Data Project (case study E) aims to improve the knowledge and intelligence available in and about a community. The aim is to support community-led service design and engage the community in decision-making, as well as improving the actions and decision-making of the local authority.

The success of these initiatives is dependent on data standards, and many of them are dependent on data standards being consistently deployed across more than one public sector organisation.

Exploring the challenges

Scotland's public sector organisations are aiming to move toward more citizen centric service delivery and combined with increasing deployment of digital tools, the demand for data and data sharing is expected to increase. Therefore, interoperability across multiple public sector data sources will become increasingly important. Furthermore, Covid-19 has increased demand from organisations and individuals for quick access to reliable and meaningful public sector data.

Data standards do exist in Scotland's public sector however the consultation activities have highlighted the lack of a coherent approach across the sector, and a lack of widespread understanding of the potential value that could be generated from the more effective reuse of public sector data assets. Furthermore, survey respondents and workshop attendees have highlighted that the application of existing data standards can be patchy, and the level of data maturity across Scotland's public sector organisations is mixed.

The consultation activities have been undertaken with data literate individuals who already engage to some degree with data standards. Despite consultees being from a diverse range of sectors and roles there was the sense of a common understanding of what should be done. A survey respondent put it as follows:

"It has become clear that there is a pressing requirement for clear, universally adopted standards to enable the re-use of public sector data. In particular, with increasing pressure on resources, there's a need by the public sector to make better use of their own data, as well as that of other agencies - and the adoption of standards is a vital component of this."

If data standards can be used to the unlock the vast potential of Scotland's data, it will provide a significant boost to social, economic, and environmental ambitions. However, consultees have highlighted a number of challenges and dilemmas that affect the adoption and application of data standards and hinder progress. It seems inevitable that these challenges will have to be addressed or mitigated if substantive progress is to be made. These challenges are described in the remainder of Section 2 and include:

• a lack of strategic leadership on public sector data;
• a fragmented landscape;
• limited learning or sharing of best practice;
• skills gaps;
• data quality (and data maturity);
• legacy systems;
• barriers to data sharing;
• disinterest; and
• investment required.

A lack of strategic leadership on public sector data

There has been no national strategy around public sector data, and this means there has not been a coherent approach or emphasis on the development and use of Scotland's public sector data assets. This strategic gap is recognised by the Scottish Government and is part of the rationale for commissioning this work on data standards. Furthermore, Scotland's forthcoming Digital Strategy (March 2021) is expected to include commitments on data and data standards, including a Data Transformation Framework (DTF), and a new AI strategy for Scotland, also due to be launched in March 2021, is expected to highlight the need for accessible and high-quality data.

Fragmented landscape

Perhaps the most significant consequence of the previous lack of strategic leadership on data, is the fragmented nature of the landscape for data and data standards in Scotland, including within fields that could be considered a single community of interest. A survey respondent noted "Data is still kept in silos where little if any metadata exists to signpost the data". Another noted "In my area there is a lot of duplication of data and a lot of effort being put in by individual organisations to create their own databases of information needlessly".

There are efforts to address this challenge. Examples include initiatives by the Digital Office for Scottish Local Government and the Improvement Service, both of which work with local authorities to develop common approaches to data, and there are national initiatives in health such as the National Digital Platform and the Telecare Minimum Data Set. The consultation for the new Digital Strategy placed significant emphasis on creating more common platforms and practice.

Limited learning or sharing of best practice

The consultation activities provided little evidence of sharing of best practice or learning between Scotland's public sector organisations, which is perhaps a direct consequence of the fragmented landscape described above. Much of the development appears to be occurring in relative isolation. The lack of shared learning and development slows progress in the sector as a whole and enables diversity of approach to become the norm.

Skills gaps

The subject of skills gaps frequently arose in the consultation activities. Commonly identified issues around skills and expertise are described below.

• If data standards are to support the optimal reuse of data, individuals who make data standards decisions require understanding and knowledge of how the collected data could be used by others. This requires individuals to think beyond the primary purpose for which they might be collecting and storing the data.
• The existence of clear data standards, supported by data experts, in relatively mature systems is not enough to guarantee success as the standards may not be consistently applied. This inevitably reduces data quality and limits the potential value of the data. The value of data standards, and data, is optimised if at point of data capture individuals have the skills to ensure appropriate and consistent application of data standards.
• For the value of data and data standards to be realised, individuals must have the capacity to find appropriate data, and have the skills to handle and interpret it appropriately. There are gaps that need to be addressed in skills for finding, accessing, and interpreting data.
• A further area of concern is that organisations with limited understanding of data security may unintentionally misuse data or increase the risk of data breaches. Skills to ensure the appropriate use of data will be increasingly required.

Data Quality

The quality of data collected, and any data standards deployed, may be sufficient for the primary purpose for which the dataset was created for, but it may not be good enough for wider application and reuse. Consultees view data quality as a necessary partner to data standards. There is work underway by the UK Government Data Quality Hub to create a data quality framework that will provide guidance on how to assess, manage and improve the quality of data to ensure the best outcome possible. There is also work underway in Scottish Government on implementation of a data maturity assessment model, which is part of a wider piece of work on a data transformation framework.

Legacy systems

Another challenge highlighted in the consultation activities is that legacy systems can 'trap' public sector data and hinder new, open, and more flexible approaches to data and data standards. The trap may be created by financial constraints that make replacing or updating a system difficult, contractual issues, and/or issues around data ownership. These challenges may be particularly challenging if the data is held in a third-party system as demonstrated in the Telecare Minimum Data Set case study.

However, it is also clear from the consultation activities that legacy systems or fragmented data sources need not act as an absolute barrier to better data or data sharing. Where legacy or difficult to change systems create a barrier, pragmatic solutions may be available. A long-running example of addressing the challenge of fragmented data sources is One Scotland Gazetteer (case study F). The One Scotland Gazetteer collates data from Scotland's 32 local authorities and uses data standards to create a single consistent and accurate land and property dataset. The Police Scotland case study (case study D) also highlighted that when dealing with legacy systems, pragmatism may be required and a cost benefit assessment can help to decide between investing in securing the 'ideal' data standard, which may require complex and therefore expensive changes to an existing system or accepting a modified standard so that it can be more easily adopted into a legacy system.

Consultees highlighted that the challenge of updating systems and data standards should be a key consideration when undertaking procurement or development exercises. Embedding data standards and flexibility at the point of procurement is considered vital to ensuring the system remains fit for purpose and will support data reuse over the long-term. The Digital First Service Standard is an example of how to approach design and use considerations before the procurement or development process begins.

Barriers to data sharing

Regulation is important to protect personal data and there needs to be a legal mechanism to enable the appropriate sharing of data. However, consultees report that barriers to sharing data include the lack of understanding of the legal mechanism, and data controllers, who may recognise the value of sharing data, but err on the side of caution in data sharing. In response to this challenge, a survey respondent stated "There are risks around the publication of public sector data for reuse that need to be considered when applying standards. However, with the appropriate level of scrutiny and consultation with the data owners and data protection officers within organisations, I feel these risks can be largely mitigated." Another stated "Decent and caveated metadata should guard against misuse of data".

Disinterest

The role and nature of data standards are not well understood. There is a sense among those consulted that 'we've been speaking about this for ages', and there is frustration that more progress has not been made. Consultees report there has been historic disinterest in data and data standards outside of the data community. Consultees believe data standards is seen as a niche technical issue, and its importance and potential value to frontline service provision is not understood. The Improvement Service shared the poll to the right which indicates that respondents believe the biggest issues faced around data are a lack of understanding at senior levels about its importance, and a disconnect between policy making and data experts.

However, there is a sense that this may be shifting as a broader range of stakeholders are slowly but increasingly recognising the potential value of data. In the survey conducted for this project, 83% of respondents believe that responding to the economic and social challenges of Covid-19 will increase the demand for data standards in Scotland's public sector. When asked to explain their answer, respondents supported their view with statements around a growing understanding of the value of data and therefore data standards in Scotland's recovery, the importance of reliable data in answering difficult questions, and an increasing need to share reliable data between organisations.

Investment required

Rapidly rising demand for public sector data reuse appears inevitable. The workshops, case studies and survey have highlighted multiple pathways for the development of data and data standards in Scotland's public sector. In some spheres of Scotland's public sector, the business case for investment has been made. However, the scale of investment for many of Scotland's public sector organisations may be substantial and the potential return on investment poorly understood.

Poor investment decisions around data and data standards can be costly, both in terms of lost opportunity, and in financial terms if future 'fixes' are complex. Furthermore, investment in data and data standards is unlikely to be a one-off investment and instead will require ongoing investment to ensure the optimal value is secured from the investment. Investment can be expected to include digital infrastructure, workforce skills development, and ongoing review and development of data standards to optimise interoperability and accessibility. However, to understand the value of investing in data standards, the potential value of the organisation's data must first be understood. This means the disinterest and lack of understanding described in the previous paragraphs are a challenge that must be overcome.

The Police Scotland case study is an example of how substantive progress requires substantive commitment and several interconnected initiatives in order to achieve 'easy access to a single view of trusted, linked data'. Police Scotland has initiatives on data and data standards governance, data ethics, co-production of data standards, a central data warehouse, and skills development, and this is all supported by a newly created Police Scotland Data Office. The significant investment has been justified by Police Scotland in order to support organisational change, improve and enhance day-to-day policing activities, enable new predictive services, and support better partnership working.

Diversity of approach

The diverse and fragmented characteristics of Scotland's public sector means that different approaches to data and data standards are being pursued. In some cases, a centralised system is being pursued to collate data that is currently held in multiple data sources onto a single platform or system to better support reuse, for example the National Digital Platform. In other cases, a federated system can be used whereby the data owners maintain responsibility for their data source and data reuse. A federated system collates and manipulates data from multiple data sources into a form that enables reuse of the data, for example the One Scotland Gazetteer.

The presence of alternative approaches may not be problematic in itself. However, data standards for Scotland's public sector will have to be sufficiently flexible to respond to the circumstances within a particular community of interest or use case, and ultimately enable the optimal reuse of Scotland's public sector data, regardless of the system from which the data originates.