Open Government Action Plan commitment 3: improving how we share information

This activity was completed in December 2019.  There is now a smooth flow of data regularly updated and published onto www.statistics.gov.scot as csv files.

The National Performance Framework measures and keeps track of how Scotland is performing in relation to the Scottish Government's purpose and national outcomes.

The government's purpose is to focus on creating a more successful country with opportunities for all of Scotland to flourish through increased wellbeing, and sustainable and inclusive economic growth.

We have developed an mechanism for publishing datasets and their relevant equalities data breakdowns on the National Performance Framework onto our Official Statistics Open Data Publishing Platform.     

We have now published the National Performance Framework as 3* csv file on statistics.gov.scot. This is an efficient publication approach which we have agreed with the National Performance Framework team.  It is being updated on a regular basis, to coincide with updates to the National Performance Framework indicates.  It is intended that this will link with the Equalities Evidence Finder, which is published by the Scottish Government.

The open data team in the Scottish Government have assessed the suitability of all official statistics for publishing on statistics.gov.scot, by providing an approximate estimate of the time involved in producing a typical dataset onto statistics.gov.scot.  

Whilst the team were planning to implement a more formal means of collating this information by asking statistics producers to provide this information, it was felt that it was not appropriate to conduct this due to the COVID-19 pandemic. Furthermore, we are aware that some statistics publications have been deprioritised due to COVID-19 pressures.

There is also an overhead involved in training statistics producers up to publish onto www.statistics.gov.scot and intervention from the open data team to check and publish every dataset onto the open data publish datasets.  We are also aware of technical limitations in publishing large 5* open datasets.   Given all of these factors, combined with the time involved to prepare datasets as open data, we feel that statistics.gov.scot is not suitable for all official statistics in the Scottish Government’s publication schedule is as it is not sustainable.

However, this process is helping the open data team identify high value open datasets, and this will help us determine what we think is sustainable and useful for any future statistics open data publishing platform.

We have published some new datasets at small area on population, to assist with contextual information around COVID-19, and have updated a number of small area datasets on statistics.gov.scot .  We have reconfigured the small area population estimates datasets on statistics.gov.scot to overcome technical issues with the current platform.

We have published an easier to use data zone to higher geography lookup file.  This was in response to help users who were building apps and products on COVID-19, and has been well-received.

We have also provided statistical support to technical colleagues in the Scottish Government to build the COVID-19 postcode checker. 

Whilst not directly published on statistics.gov.scot, our colleagues in Public Health Scotland have published a dashboard based on open data on the number of COVID-19 cases at intermediate geography level.  This is updated every day, and is linked to from the Scottish Government website.

Google analytics have shown us that we have received a record high number of page views of the "Find places" tool for people finding out what area they live in.  This is largely because users are searching for the name of the neighbourhood that they live in (intermediate zone) by searching by area or postcode to find out the latest rates of COVID-19 in their local area. 

Since April 2020, we have published daily COVID-19 management information on www.statistics.gov.scot, and taken from the Scottish Government daily updates.   These are in addition to daily and weekly open data publications published by. Public Health Scotland

Whilst it has been successful to an extent and we have received record numbers of downloads for this dataset and for www.statistics.gov.scot as a whole in 2020 we are aware that because the Scottish Government COVID-19 data are management information,  there are often unforeseen and unavoidable changes to the structure and definitions of data at very short notice.  This means that the COVID-19 open dataset on www.statistics.gov.scot  is increasingly complicated as a result.   These issues are largely a reflection of the continually developing response to the pandemic.   We are therefore reviewing the frequency at which we publish these data, and have now agreed to publish these data on a weekly basis as of 23 December 2020.

We have worked constructively with procurement colleagues to publish 5* linked open data on the Scottish Government’s aggregate spend for the Top 50 Suppliers.  This was published for the first time on 4 December 2020 and covers the years 2019/20.

The COVID-19 crisis has resulted in an upsurge in dashboards being built which have driven by published open data from organisations such as the Scottish Government, Public Health Scotland and National Records of Scotland.  

For example, these open datasets have enabled Scottish Tech Army volunteers to build a dashboard reusing a number of different published sources of open data.  The website plays musical notes to create sonic graphs of Covid-19 cases that allow the visually impaired to keep track of infection rates and fatalities, using the latest official data for health boards and local councils.  

We have built strong links with the open data team in Public Health Scotland over the course of the COVID-19 pandemic so we are clearer as to where different sources of data are published. As part of these discussions we have shared our experiences of open data publishing – this is helping us to consider improvements our open data processes going forward.

On the back of the Scottish Open data Unconference in September 2020, we are starting to put together a series of case studies which showcase the economic, social, and environmental value of open data, and at a variety of different scales.

Whilst we in the open data team in the Scottish Government have had not had capacity to build publicly available local area profiles in this period due to the COVID-19 pandemic, our colleagues in Public Health Scotland have produced a helpful dashboard on the number of COVID-19 cases at intermediate geography level:

https://public.tableau.com/profile/phs.covid.19#!/vizhome/COVID-19DailyDashboard_15960160643010/Overview

 Under the sponsorship of the Scottish COVID-19 Data and Intelligence Network, and in conjunction with the DataLab, we undertook a “Data Catalogue Assessment” to better understand the level of adoption, maturity and accessibility of data catalogues for open datasets published on statistics.gov.scot.   There have been equivalent scoping exercises, with a focus on search engine optimisation, for other government open data portals such as the SpatialData.gov.scot and RemoteSensingData.gov.scot Metadata Portals.

We have also been working in partnership with other public sector organisations such as the Scottish Cities Alliance to share best practice on metadata and data standards. We have also worked with the UK Data Standards Authority who have recommend a set of metadata standards for sharing and publishing data, to improve data sharing across government. We have also linked with Police Scotland to advise them about our experiences of managing an open data publishing system, and we are keen to continue these links.

We are in the process of undertaking a thorough review of all open datasets as published on www.statistics.gov.scot to ensure that the datasets are up to date and relevant.

Members of the Scottish Government open data team helped to fund and participate in the online Scottish Open Data Unconference, organised by Code the City, a civic hacking initiative interested in hack weekends, open data, workshops, and idea generation tools.

The team chaired a number of discussions including a session on seeing how it could be easier to find open datasets produced by different public bodies (this issue has been brought into sharper focus by COVID-19), and on use cases around open data – which we are keen to take forward.