Open Government action plan 2021 to 2025 - commitment 3: data and digital commitment

Progress to September 2024

Commitment 3: data and digital commitment

Objective

Open up data relevant to other open government themes, such as key climate change datasets used by government for modelling and reporting, data on public transport and public sector expenditure.

Milestone

Milestone: Continue to support the Fiscal Transparency Programme, climate change policy and other areas relevant to the open government themes to encourage open data reuse and share best practice on data standards, FAIR and open data principles.

Milestone (2024 to 2025): Improve our understanding of the technological solutions for deliberation, to support the Scottish Government Participation Framework and to inform potential future applications to expand participation around public good uses of public data and AI.

Progress and next steps

Members of the Data Division participated in the Fiscal Openness and Transparency Commitment Group Meeting in June 2024.  We provided constructive technical input around open data and data standards in discussions with the portal’s contractors and programme manager. During discussions, we learnt about good practice and approaches to publishing fiscal data from other administrations in the OGP Nordic+ event earlier in the year. This is helping us to understand the types of approaches we might want to take for improving the way we publish data in Scotland. The contractors for the fiscal portal granted us access to the portal to help us develop an understanding of the data publishing processes, pipelines work, data presentation, and how the data can be updated in a sustainable manner. A further consideration of our work in this area is the critical role of data quality for enabling improved access to information.  

We have started to explore the steps needed to increase the amount of grants data published openly. Working with internal colleagues and interested stakeholders, we are beginning to identify the use cases and support needs. We will continue to develop an understanding of the challenges and look for specific areas to test approaches.

We are continuing to contribute to the Health and Social Care Data Board. The opportunity for health boards to take part in the data maturity programme was well-received when promoted at a recent board meeting.

Market research on the existing civic technologies for participation was published on 31 July. This work builds on previous deliberations of the Scottish Collective Intelligence Community (SCIC).  It was commissioned to support a newly launched CivTech challenge seeking to explore how technology could help us enhance and scale the use of the Scottish Government's Participation Framework around data and AI.

The market research was led by University of Edinburgh researchers who are also members of the SCIC. It found poor interoperability, a lack of end-to-end support and underdeveloped ethical considerations beyond data handling, but opportunities for reflective design for whole policy-cycle participation and support for offline participation.

Specifically, the CivTech challenge - co-sponsored by the Data Division and the Open Government team – asks ‘how can technology help us deliver high-quality, scalable, public participation in decision-making, inspiring trust around ethical data use and sharing, AI, and wider public good activities.’ Interviews will take place over autumn with the accelerator stage expected in January.

Next steps

• continue internal advocacy to influence data improvement and open data work
• the upcoming meetings of the SCIC will focus on drawing connections between the SCIC and Open Government, and supporting the CivTech challenge process

Status

On schedule.

Objective

Run a CivTech challenge to evaluate if technology can make public sector data easy to find, assess outcomes and set out the way forward.

Milestones

Milestone (2024 to 2025): Develop a considered approach to dataset sourcing and management for Find.Data.Gov.Scot to improve the discoverability of identified datasets, and by November 2025, add 10 more data sources to Find.Data.Gov.Scot.

Milestone (2024 to 2025): Deliver an improved user feedback channel and run workshops with data producers and consumers to identify missing areas, address data quality issues and promote use. Investigate whether Find.Data.Gov.Scot can be linked to the UK Data Marketplace Government Data Catalogue.

Progress and next steps

Following a successful meeting of dataset owners in April, user feedback has informed further refinements to the front page of find.data.gov.scot. The updated homepage provides clearer Scottish Government branding, easy access to the service roadmap, popular data owners and popular data topics.

This post-delivery phase aims to promote the data discovery tool. We also want to improve its use by data producers and users, based on their feedback. New datasets are being added and improved navigation allows users to easily recommend and submit datasets. Users can access the beta AI metadata generator, API, and data sources by signing up to the site.

To help promote uptake a new blog will be published, and we will look to provide more prominent placement on Scottish Government websites, as well as considering where it can be integrated into existing services.

Next Steps

• publish Scottish Government Digital blog
• continue with promotion and user engagement, including hosting a webinar for the Data Lab’s community platform

Status

On schedule.

Objective

Set up the Data Transformation Framework stating what ‘good data’ looks like and the process by which organisations can improve – this focuses on opportunity for organisations to improve data maturity, data literacy and adoption of standards, through collaboration and engagement with local government and other public sector bodies, to be useful for civil society.

Milestones

Milestone (2024 to 2025): Launch the data maturity network for cohort alumni on Knowledge Hub, to provide post-project support and networking, including published resources for data standards and data governance (complete and test by summer 2024).

Milestone (2024 to 2025): Start to develop resources for data ethics, innovation and architecture and provide strategic guidance on the four foundations to data improvement that align to these pathways.

Milestone (2024 to 2025): Develop and launch self-supporting data maturity guidance.

Milestone (2024 to 2025): Complete cohorts 3 and 4 of the Data Maturity Programme by June 2024, launch cohort 5 in September 2024 (if funded).

Progress and next steps

Ten public sector organisations have been recruited for the fifth cohort of the Data Maturity Programme, starting at the end of August. This part of the Data Transformation Framework (DTF) helps organisations consider their ambition for data, assess the current level of data maturity and make plans for data improvement. The cohort will run until the end of February.

Sixteen organisations took part in cohorts 3 and 4, which ran concurrently over the period from October 2023 to June 2024 and have now completed the programme.

Plans are well developed to launch the Data Maturity Network for alumni in September to offer ongoing support post project. The network will take a collaborative approach to understand the needs of organisations that have taken part in the programme to develop and review new DTF resources, provide peer support and further develop their understanding of data improvement with expert sessions on topics such as AI and open data. The network responds to the increasingly recognised need to help organisations sustain systemic change in how data is understood, managed and used.

Testing has been completed for the DTF resources in two main areas: guidance for data governance (including training, implementation and creating a framework); and metadata (asset discovery, tooling, making metadata discoverable, storing data). The next stage is to revise and update existing written guidance on the four foundations of data improvement (data leadership, strategy, governance and discovery). Originally developed by the Australian Government, these will be updated and aligned to the experience and needs of Scottish organisations.

The first iteration of an e-learning package designed to support managers with staff taking part in the data maturity programme has been developed. Further refinement will take place in the next period alongside detailed scoping of a more comprehensive self-supporting package.

Next steps

• launch cohort five of the Data Maturity Programme
• launch the Data Maturity Network
• revise and update foundational four guidance
• refine and scope e-learning packages

Status

On schedule.

Objective

Review the front end of our official statistics open data publishing platform, www.statistics.gov.scot.

Milestones

Milestone (2024 to 2025): Initiate and run a discovery, which aims to gain a better understanding of the current service.

Milestone (2024 to 2025): Complete discovery and produce a report which advises on options and recommendations for next steps.

Milestone (2024 to 2025): Based on evidence and recommendations from the discovery, either move to next steps in improving the service in an alpha phase; or make small / limited changes to the service.

Progress and next steps

We are nearing the end of the discovery phase of a project to improve Scotland's Official Statistics open data publishing platform, statistics.gov.scot. We have been looking at user needs; the technology and processes underpinning open data publishing; and the wider policy landscape.  We regularly publish open data on statistics.gov.scot.

The agile discovery aims to understand the service’s current and potential user base, their needs, problems, and barriers, how the service currently addresses these, and how it could be improved to address these better in the future.

We have been learning more about our users and understanding what they want and need. We believe our users fall into two main groups:

• data publishers - those that produce data information and statistics that they wish to share and disseminate
• data users - those who want to access this information to use, interact with and create value, ranging from general citizens to expert technical users

From May to July 2024, we ran a number of interactive workshops and usability sessions (four for internal publishers and users; and four for external users) on what works and what doesn’t work on the site. The outcomes of these sessions are:

• an overview of the current users of the site and the tasks they would like to achieve
• an understanding of the gaps in the current provision of the service
• suggestions for improvements from a range of users

To ensure we included a range of external users from outside the Scottish Government, we approached groups such as the Better Data Community of Practice, the Open Data Scotland community, the Data Lab community, and ScotlandIS.

We then ran co-design sessions with publishers and users to explore the identified opportunities and problems which emerged during the research.

We also ran a series of targeted 1-1 interviews with 11 key stakeholders to obtain a more strategic expert view of the service from a wider policy and technical perspective. We interviewed representatives from internal government as well as from other public bodies and community groups.

In total, we directly engaged with 46 people within the user research phase of the project. The key outputs from this user research are: a user needs catalogue, user problem statements, a summary of technical challenges; and a final user research report.  

We have conducted other relevant activities as part of the discovery phase. Some of these include:

• producing a stock take of what data is held on the site along with a summary of user analytics and producing process maps outlining the steps involved for publishers
• reviewing the features of a number of other public sector open data portals within Scotland and from other administrations
• considering the findings of Open Data in Scotland: A Blueprint for Unlocking Innovation, Collaboration and Impact in the context of improving the statistics.gov.scot service

Next steps

• finalise the report summarising the findings of the user research workstream of the statistics.gov.scot improvement project discovery phase
• produce a paper which advises on options and recommendations for next steps for the statistics.gov.scot improvement project, based on evidence collated in the discovery phase  

Status

On schedule.

Objective

Increase the amount of Scottish public sector open data being published, through collaborations such as the Data Standards and Open Data Community of Practice.

Milestones

Milestone: Publish the independent report and recommendations outlining a more strategic approach to supporting open data in Scotland.

Milestone: During the remainder of 2024 initiate a project plan, and develop a roadmap, with the outcome of creating a clear vision for purpose-driven impactful open government data in Scotland and priorities for its implementation.

Milestone: Set up an open data external reference group to provide accountability to the open government data commitment and to foster and long-term collaborative approach to open data.

Milestone: Build on the series of open data events held with the Better Data Community of Practice in March 2024 with a series on data standards in May 2024 and encourage knowledge sharing about common challenges in making data better, smarter and more open.

Progress and next steps

We published an independent report Open Data in Scotland: A Blueprint for Unlocking Innovation, Collaboration and Impact on 29 August 2024. This has been written by Open Data Scotland on behalf of the Scottish Government and is supported by a blog from the Open Data team. The report advocates for a more purpose-based approach where open data can be used to maximise areas of value and impact. These are:

• transparency, accountability and empowerment
• innovation and economic growth
• management and monitoring
• collaboration and coordination

The report takes into account the views of a range of stakeholders. These include open data practitioners in Scotland and community groups. We would like to thank them in helping to shape the findings of this report. 

It contains a number of recommendations which outline how the Scottish Government can position itself as a leader in open data, and in turn help inform evidence-based decision making:

• firstly, to establish a vision for open data in Scotland, which positions open data as a key tool for advancing policy priorities, rather than a tick box exercise. Crucially, this approach requires long-term commitment and collaboration among stakeholders: from data publishers, to the public and private sector, and civil society
• secondly, to develop a pathway for implementation that will lead to impact, by making informed decisions about the technologies and standards used to publish and access data, providing appropriate support to publishers and celebrating success

Following a successful series of open data themed events held over March and April, the Scottish Government’s Better Data Community (Making Data Better, Smarter and More Open) held eight events on data standards from May to July. The events created opportunities to discuss how we can define the scope of standards, sell the benefits of standards and implement data standards. They included speakers from the Data Standards Authority, the Scottish Data Standards Boards and TPXimpact on adopting the Open Referral UK data standard.

Over the next few months, the community is running events on data in policy development which will focus on how we can learn from others to improve how we shape, develop and implement policies to achieve effective and sustainable outcomes. Our speakers include Gordon Guthrie (Research Fellow at the Scottish Government), Tameside Council who will be discussing the Scalable Approach to Vulnerability via Interoperability (SAVVI) project and Edinburgh Climate Change Institute (ECCI).

Next steps

• agree and co-produce a vision for open data with a range of external stakeholders
• start to scope out an open data influencing and reference group to provide accountability to the open government data commitment and to foster a long-term collaborative approach to open data
• complete the series of data and policy themed events and prepare for the Better Data Community conference

Status

On schedule.

Objective

Develop a public register of AI algorithms.

Milestones

Milestone (2024 to 2025): Begin a phased approach to the public sector wide roll out of the Scottish AI Register.

Milestone (2024 to 2025): Initial stage to produce SG specific guidance and support materials for the creation of AI.

Milestone (2024 to 2025): Identify existing use cases and new SG AI projects to be added.

Milestone (2024 to 2025): Work with SG Data Science colleagues to ensure the adequate recording of all corporate AI use.

Milestone (2024 to 2025): Work with procurement colleagues to build guidance into purchasing processes.

Milestone (2024 to 2025): Continue promotion of the product through the Scottish AI taskforce to help future roll out.

Progress and next steps

On 22nd August 2024 an article was posted on the Scottish Government intranet alerting staff to the mandating of the AI Register, beginning the process of collating entries.

The AI Library function, detailing AI policies, models and risks and their mitigations has now been made available to all staff. Supporting them in considering how best to responsibly use AI.

We are working in conjunction with the Scottish Government’s automation team to identify and record corporate AI use.

Next steps

• internal promotion via Scottish Government intranet and learning platform and in person team meeting/away day presentations, will continue to identify new use cases to be recorded
• early work to consider how the Scottish Government might develop its own AI policy and supporting documentation, in conjunction with data science colleagues has been undertaken and looks very positive. This will continue and expand to link up risk management and shared documentation options
• work on getting the 60+ use cases already identified within core Scottish Government recorded
• initial consideration of how the mandate can be rolled out to organisations outwith the core SG/Agencies
• work with Cyber, Procurement and Information/Data Protection teams to embed consideration of AI into our corporate policies

Status

On schedule.