Open data in Scotland: a blueprint for unlocking innovation, collaboration and impact

3. Open data in Scotland

3.1 Policy environment

Scotland has a number of strategy initiatives directly or indirectly related to open data:

• Freedom of Information (Scotland) Act, 2002: An Act of Scottish Parliament which entitles any person to be given information held by a Scottish public authority upon request, subject to certain conditions and exemptions.
• INSPIRE (Scotland) Regulations, 2009: An implementation of EU Directive 2007/2/EC which requires Scottish public authorities to make spatial datasets (for example, map data) available.
• The UK and Scottish Statistical System: Official statistics in Scotland provide an accurate and up-to-date picture of the economy and society. There is a Code of Practice for Statistics that sets the standards that producers of official statistics should commit to. Compliance with the Code gives users confidence that published government statistics have public value, are high quality, and are produced by people and organisations that are trustworthy. Publishing data openly is an important component of the Code of Practice for Statistics.
• Open Data Strategy, 2015: Scotland's Open Data Strategy established the principle of ‘open by default’ and aimed to promote open data in order to improve the delivery of public services, catalyse social economic benefits through innovation and reuse, and improve transparency and accountability of the public sector. The strategy is supported by guidance and a toolbox.
• Scotland's Open Government Partnership (OGP) Action Plan, 2021-2025: Scotland is a local member of OGP. The OGP is a global partnership between civil society and government committed to transparent, participatory, inclusive and accountable governance. Data and digital is one of five commitments made as part of Scotland’s OGP action plan. It includes plans to open up datasets relevant to other commitments, and to develop the open data infrastructure.
• A Changing Nation: how Scotland will thrive in a digital world, 2021: Scotland’s Digital Strategy underlines Scotland's commitment to thriving in a digital world. The strategy identifies a number of priority areas where digital initiatives can contribute: improving the delivery of public services, economic recovery and growth, addressing climate change, and improving digital skills and connectivity to ensure participation. The strategy identifies open data as a key implementation tool and commits to making more open data available and discoverable.
• Scotland’s AI Strategy: Trustworthy, Ethical and Inclusive, 2021: Scotland’s AI strategy sets out the goal to make Scotland a world leader in trustworthy, ethical and inclusive AI. The ongoing supply of high-quality and responsibly-produced data is identified as one of the foundations for success, and open data as a key tool to make this happen.
• Greater access, better insight, improved outcomes: a strategy for data-driven care in the digital age, 2023: Scotland’s first digital strategy for health and social care is aimed at improving how data is used in the sector to enable the public to access and use data to improve their health and improving how health and social care outcomes and services are delivered. The strategy identifies the need for useful, relevant, and accessible data to be made available openly, within a strong framework that ensures data is shared appropriately.
• Participation Framework, 2023: Scotland’s Participation Framework provides guidance and tools to improve public involvement in the design and delivery of policy and services. The framework was delivered as part of Scotland’s OGP commitments. The framework identifies the publication of data as a tool to enable informed participation. The tools and techniques in the framework are also relevant for planning open data publications that meet user needs, build trust and ultimately generate impact.

In addition, there are a number of practical initiatives directly or indirectly concerned with improving open data publication, use and impact:

• The Digital Directorate hosts the ‘Better Data Community (Making Data Better, Smarter and More Open)’. This community of practice is intended for anyone in the public sector who is interested in improving the quality of the data collected in the public sector, how it can be more effectively managed and reused. It allows practitioners to share best practice for the publication of open data.
• The Digital Data Maturity program has created a Data Transformation Framework that will increase the digital capability of public sector organisations. The program has run several cohorts and is now trialling a self-service approach.
• Outside the public sector, Code the City, a charity that campaigns for better open data, hosts a Slack community, an annual conference, and its own data portal.

3.2 Publication landscape

Open data-related roles and responsibilities provide important context for assessing the state of open data publication and use. In Scotland, there is a key distinction between:

• Policy responsibility, which sits with the Open Data Team^[58] in the Data Division of the Digital Directorate of the Scottish Government; and,
• Publication responsibility, which is delegated to potentially hundreds of public sector bodies, including local authorities, government departments and organisations, the Scottish Parliament, and NHS Scotland.

This highly dispersed responsibility for open data publication in Scotland is not unusual but it does present a dual challenge to:

• Translate the enthusiasm and direction of travel from the Scottish Government into publication by public sector organisations; and,
• Guide current publication efforts into a set of coordinated publication practices that will result in a coherent, accessible and impactful data publication landscape across Scotland.

These challenges are analogous to those faced by large multi stakeholder data initiatives that rely on the distributed publication of data that would otherwise be locked up in organisational silos to generate a useful corpus of shared data. Scotland may benefit from adopting a similar high-level approach to coordinating distributed publication at scale to that used by these initiatives, which combines advocacy and strategic work, ongoing outreach and partnership-building, and the provision of practical resources and support. This would imply some intentional blurring of the line between policy responsibility and publication responsibility.

Scotland’s results on open data to date have been mixed. Scotland has some excellent open data, including geospatial data, and key resources like Scotland’s Census and Official Statistics that are both trusted, accessible and well-used. In addition, huge progress has been made on the publication of data in complex domains like fiscal transparency. But that said, the coverage of open data is uneven (and particularly low among local authorities) and there is a sense that progress and enthusiasm has stalled.^[59]

3.3 Findings from workshops

The desk research was complemented with a series of public workshops to discover attitudes towards open data among technical and non-technical practitioners and stakeholders in Scotland. The workshops, run by Fractals Coop, were designed to provide an accessible and open forum for a wide range of participants to talk about their experience of, and hopes for, open data. Participants in the workshops were drawn from a range of sectors including central and local government, other public sector, civil society, academia and the private sector, with a variety of job roles and technical or policy expertise represented. Unsurprisingly all attendees had some experience with open data, and were therefore not fully representative of the much larger pool of those who might benefit from open data in the future.

Source: Scottish Government, Summarised findings of workshops run by Fractals Coop, in conjunction with Open Data Services

Several important themes emerged from these workshops:

• Policy: Participants emphasised the need for clearer direction on open data and on adopting open data policies that are clear, implementable and enforceable. Participants noted the drawbacks of the unenforced ‘open by default’ approach and suggested combining incentives for good practices, support for implementation, and accountability for those not meeting their obligations. Participants also noted that open data could be attached to broader policy initiatives and strategies, including on AI and digital, and that alignment with international practices, such as the EU Open Data Directive, would be helpful.
• Mindset and Culture: Participants identified cultural and mindset barriers in the public sector as a significant barrier to open data publication. Publishing open data is perceived as risky with few tangible benefits for producers and, as a result, there was a noted reluctance to share data openly and a need for a cultural shift towards more collaborative and open practices. Concerns about data publication errors, including incorrectly publishing personal or commercial data, and the need for clear guidance and training were also emphasised. It was noted that data can be an off-putting and overly technical term, which narrows the conversation on openness.
• Needs of Data Producers: Participants noted that data producers face numerous practical challenges when publishing open data, such as skills and resourcing gaps, a lack of practical support, and gaps in enabling infrastructure such as a centralised repository for open data. Participants noted the helpfulness of existing guidance and the need for updates and expansion to such material, to include standardised tools and practices for data sharing and interoperability and to cover key topics such as data anonymisation techniques.
• Needs of Data Users: Participants identified several key areas where data users encounter difficulties with open data. These include: discovering and accessing relevant information, finding data that is high quality and up-to-date, understanding the context and potential applications of the data, and having the right skills to access and use data. There was also a call for more user-friendly systems and tools that can facilitate easier interaction with and analysis of open data, especially for non-technical groups.
• Impact and Value: Participants were generally enthusiastic about the potential for open data and stressed the need to shift wider perceptions by demonstrating the economic and social benefits. Participants suggested that showcasing successful use cases and real-life applications could inspire greater adoption and innovation and noted the need for mechanisms to measure and communicate the impact of open data initiatives to justify ongoing investment.

These insights highlight many of the important links in the open data value chain model (Commit-Build-Use-Learn) discussed in section 2.4 and the issues that stop open data in Scotland from reaching its full potential.

3.4 Opportunities for the future

Scotland has an opportunity to take advantage of a policy environment which is amenable to open data publication, and enthusiasm and capacity in the community and private sector about the potential of open data, as a strong foundation for driving forward the next phase of impactful open data implementation.

The key will be to identify areas of strength and capacity within the Scottish Government, and define carefully the role that the Scottish Government will play in the wider landscape. It is important to set expectations about responsibilities and outcomes, and to prioritise an approach that will allow the Scottish Government to have the most impact, making best use of limited resources available.

This is a direction setting and coordination role, with responsibility for the actual implementation distributed among other stakeholders. The Scottish Government can, for example, advocate internally to ensure the culture and expertise is conducive to meeting high level policy goals; identify specific areas where open data would have a significant impact; convene communities of practice and help to coordinate and intentionally direct their activities; identify and commission appropriate support for data publishers; monitor, measure and celebrate the benefits created by open data projects. This has the potential to create a positive feedback loop, where demonstrating impact increases enthusiasm and buy-in internally and externally, setting the stage for further improvements and innovations in the long-term.

The following section highlights some more specific findings from the workshops, linking these to detailed recommendations on improving the impact of open government data in Scotland.