Open Government action plan 2021 to 2025 - commitment 3: data and digital commitment

Progress to November 2022

Milestone: Open up data relevant to other open government themes, such as key climate change datasets used by government for modelling and reporting, data on public transport and public sector expenditure

Update and next steps

We wish to open up data relevant to other open government themes by linking with colleagues who work in the three open government commitment areas where data acts as an enabler (financial transparency, climate change, health).  We firstly want to understand what specific questions are being asked for these commitments and seek to explain how data can be used to help answer these questions.

In autumn 2022, we have held meetings with other commitment leads to explore the role of data as an enabler for opening up government.   We are keen to collaborate in the first instance with those working in the fiscal openness and transparency commitment.  Going forward, we ideally would like to establish datasets which meet specific needs in this area.  This will be accompanied by specific data standards and metadata requirements to enable sustainable publication.

We will link with the climate change commitment in due course. We have held initial conversations with the health and social care commitment and have helped colleagues drafting the Health and Social Care Data Strategy prepare a chapter on FAIR Data Principles (Findable, Accessible, Interoperable and Reusable).

Status

On schedule.

Milestone: Run a CivTech challenge to evaluate if technology can make public sector data easy to find, assess outcomes and set out the way forward

Update and next steps

The Scottish Government has been working with the Data & Intelligence Network and  NatureScot on a CivTech Data Discoverability Challenge - CivTech 6 – Challenge 12 – Public sector data — The CivTech to evaluate if technology can make public sector data (held across multiple portals) easier to find, access and use.

The solution, named ‘Dtechtive’, developed by Dtime,  enables easy and efficient data search using simple language and contextual information. It identifies datasets and provides indicators of popularity, quality and age. The solution will be available through an open API and the Beta release of the open data search is proposed for November 2022.

Over the last month we have been engaging with the advisory group, a group consisting of public, private and 3rd sector organisations and for the meeting in October the advisory group concentrated on the features for the data provider.

Status

On schedule.

Milestone: Set up the Data Transformation Framework stating what ‘good data’ looks like and the process by which organisations can improve – this focuses on opportunity for organisations to improve data maturity, data literacy and adoption of standards, through collaboration and engagement with local government and other public sector bodies, to be useful for civil society.

Update and next steps

The Data Transformation Framework (DTF) provides the scaffolding and support to empower and sustain improvement projects and is supported by the Data Maturity Programme (DMP), which is the driver to activate change in public sector organisations.

The first cohort of the Data Maturity and Pathways Programme (DMPP) has completed, delivered with seven public sector organisations, five local authorities (Aberdeenshire, East Lothian, Glasgow, West Dunbartonshire, West Lothian) Renfrewshire Health and Social Care Partnership and Transport Scotland. Whilst this has been very much a learning project it has shown that there is a strong need for strategic leadership, direction and guidance to enable public sector organisations to achieve desired data improvements and to become data driven in their thinking and decision making. The main challenges from this first cohort were:

• limited internal capacity – it’s hard to create the space and time required to undertake the research and planning required to pursue data transformation
• securing buy-in for data transformation from others – data transformation and its potential benefits are not well understood
• identifying what to prioritise – the opportunities for data transformation can feel limitless and overwhelming

The next cohort can expect to face similar challenges in undertaking this project and will be encouraged to think how their approach to the tasks can mitigate or avoid them. The design of the programme has also been adapted to allow more time for buy-in prior to undertaking the data maturity assessment.

Results from the assessments themselves show that for each individual organisation, there was considerable  range of opinions about data maturity. However, overall, the strongest themes for data maturity were data, culture and use and the areas most identified for improvements were tools, analysis and skills. The overall conclusion for the Scottish Government and cohort members is that public sector organisations have lots of data, they spend a lot of staff time working with it, they use it for many different purposes, however they are not optimising its value as a strategic resource.

The second Data Maturity cohort is now underway, working with 10 organisations.  The project has been redesigned as a result of the findings from the first cohort.  The project now has 4 main tasks to be completed by the cohort: project planning, ambition setting, research (including data maturity assessment) and data transformation planning.  The project will run until March 2023 and we are aiming to package up and share a number of the tools used by the cohort, to be used by the wider public sector. 

The Personas project, designed to identify initial users of the Data Transformation Framework,  is almost complete and a report is expected shortly. Five personas have been identified which will be used to inform and validate development work for the Data Transformation Framework.

Both these and the earlier data personas will be made available on the Framework for wider use. 

Status

On schedule.

Milestone: Review the front end of our official statistics open data publishing platform, statistics.gov.scot.

Update and next steps

Scotland’s Official Statistics open data publishing system, statistics.gov.scot was launched in 2016.  Since then, the open data team within Scottish Government has continued to improve the site and to increase the number of relevant datasets which it holds. As the site has been live for a number of years, we wanted to review it. We worked with a company called User Vision to carry out user research on the site. The main objectives were to determine what tasks users came to complete on the site, and whether they were successful in doing so.  We ran a script on the site and only ran limited promotion of the survey to reduce potential bias amongst respondents.

57% of respondents had visited the site in a personal capacity, rather than for professional reasons. The findings were supplemented with one-to-one in depth interviews with a selection of individuals who had completed the survey, and with other stakeholders. This research indicated that people have not always been able to complete the tasks they intended to do on the site.

We worked with stakeholders and contractors to develop a new more simplified home page for statistics.gov.scot. This was launched in July 2022. This improvement is the first of a series of longer term activities to develop the site. We intend to work in an iterative, “agile” manner to achieve further improvements. We recognise that some of these changes are likely to be substantial.

Further information on the new home page and next plans can be found with the Scottish Government digital blog, Next steps for Scotland's Official Statistics Open Data platform.

We are always keen to better understand use cases for our statistics open data platform. For example, in October 2022, a member of the public produced visualisations of council tax data from statistics.gov.scot  and published a blog about their experiences with open source code to help others use and re-use data these data to help them build their own produce.

Over the past 6 months, the Scottish government open data team have delivered a series of demonstrations to external stakeholders on how to best use Scotland’s Official Statistics open data platform, statistics.gov.scot. For example, we have spoken to representatives from councils and from charities such as the Trees for Life rewilding charity  As part of these sessions, we have sought feedback on how people are using the data from the platform.  We have also run a series of internal analytical seminars to help users publish open data on statistics.gov.scot. We have also run one on the importance of metadata in finding and re-using open data, and one on the  R Opendatascot package to get data from statistics.gov.scot. This package gives developers the commands they need to probe the dataset, and request exactly what they want for their  analytical pipeline or shiny app. This empowers people to easily build products from openly available data from statistics.gov.scot.

Status

On schedule.

Milestone: Increase the amount of Scottish public sector open data being published, through collaborations such as the Data Standards and Open Data Community of Practice.

Update and next steps

The Scottish Government is keen to collaborate to increase the quantity and quality of relevant open data to help inform decision making for citizens and to grow the economy and improve people’s outcomes.  For open data publication to be sustainable, we need to ensure that useful and relevant datasets are being published and the data are sufficiently mature.

We are building a Community of Practice amongst those in the public sector, in the first instance, with an interest in data (including data standards / open data / data innovation). The Community of Practice (CoP) on Data Standards and Open Data is a network of representatives from public sector organisations aiming to share knowledge, experiences and lessons-learnt on data standards and open data.  The broad aim of this community is to further our common goals of increasing the adoption of data standards and the publication of open data within the public sector, which are vital if Scotland is to succeed in its economic, social, and environmental ambitions.

We are in the process of refreshing and expanding the SharePoint site for the CoP. We met with CoP champions in November 2022 (around 20 representatives with an interest in data standards and open data across the Scottish public sector).  Going forward, we will like to use the group to test out our thinking on refreshing open data guidance.

As part of the  inaugural Scottish AI Summit in March 2022, the Scottish Government open data team, in conjunction with the Data Lab, organised and facilitated a panel session on  “Open Data: Enabling Transparency, Empowering Communities and Creating Economic Opportunity”. We sought constructive feedback on the composition of the panel and the types of questions we might want to include in the discussion from a key member of Code the City, an Aberdeen based civic-hacking initiative who are focused on using tech and (open) data for civic good.  This builds on comments made by Scotland’s First Minister at the Data Summit in November 2021, in which she stated the importance of high quality open and transparent data which both she and individuals used to make decisions on the pandemic. Our speakers each gave a short pitch on why they think open data is important and how they have used open data to make decisions.  They considered what success looks like for open data and what the challenges are, as well as the links between AI and publishing open data.  The outputs of this panel discussion are helping us shape our thinking on next steps around developing refreshing updated technical guidance on open data.

We have also been linking with stakeholders in the third sector and the private sector, through alliances such as ScotlandIS and the Open Data Institute (ODI).   We have also been speaking to a wide range of stakeholders at flagship Scottish events such as the DataFest Data Summit  to better understand how consumers are using open data to benefit their communities.  We are hoping to use these conversations to help grow our civil society base for the data commitment.

Status

On schedule.

Milestone: Develop a public register of AI algorithms.

Update and next steps

Scotland’s AI strategy aims to make Scotland a leader in the development and adoption of trustworthy, ethical and inclusive AI.

Earning public trust through a meaningful and effective dialogue around the use and benefits of AI is foundational to achieving this aim. The Scottish Government is leading by example, developing an open and accessible register of the AI algorithms used in the public sector.

The register will not be a simple static platform to inform the public, but a fully documented set of processes and methods for the development and deployment of AI systems. This will help enable conversations both internally and externally throughout the lifecycle of the system.

The register is currently being developed as part of a Pre-Commercial Agreement with Finnish start-up Saidot.

We will launch the register by the end of 2022/23 in keeping with the timeline agreed in Scotland’s AI Strategy, with a target date of January 2023.

Milestone with end date December 2022: Make initial steps with CivTech challenge on finding data. We plan to engage with civil society around findability of data as part of this challenge

Status

Finished.

Milestone with end date December 2022: Pilot Data Maturity Pathways project which will guide six public sector organisations through an end to end journey

Status

Finished.

Milestone with end date December 2022: User research to inform user journeys on statistics.gov.scot - and agree next steps on improvements for statistics.gov.scot.

Status

Finished.

Milestone with end date December 2022: Agree and establish communications channels and understand user needs. This includes setting up the Data Standards and Open Data Community of Practice.

Status

Finished.

Milestone with end date December 2022: Make initial steps on CivTech challenge on what data people are looking for through the finding public sector data challenge.

Status

Identify thematic areas – such climate.

Status

Finished.

Milestone with end date December 2022: Review and develop future milestones.

Status

In progress. Starting to think these through for 2023.