Open Government Action Plan commitment 3: improving how we share information

The National Performance Framework measures and keeps track of how Scotland is performing in relation to the Scottish Government's purpose and national outcomes.

The government's purpose is to focus on creating a more successful country with opportunities for all of Scotland to flourish through increased wellbeing, and sustainable and inclusive economic growth.

We have developed an mechanism for publishing datasets and their relevant equalities data breakdowns on the National Performance Framework onto our Official Statistics Open Data Publishing Platform.     

We have agreed an efficient publication approach with colleagues in the National Performance Framework team and this will link with the Equalities Evidence Finder, which is published by the Scottish Government. 

The Scottish Government publishes a  12 month calendar  of its forthcoming statistical releases in spreadsheet format; this includes both Official and National Statistics publications; this is in line with the requirements of the  Code of Practice for Statistics.

🙂 On track 

As at 4 December 2019, there are 36 data zone level datasets published on statistics.gov.scot. Since the launch of Scotland’s Open Government Action Plan 2018-2020, we have updated 32 data zone level datasets, including 5 new datasets.

The 5 new datasets we have published are on:

• Deaths
• Personal Tax Credits: Number of Families
• Scottish Index of Multiple Deprivation - Housing Indicators
• Scottish Index of Multiple Deprivation - Education Indicators
• Scottish Index of Multiple Deprivation - Health Indicators

We also publish data from the Scottish Survey Core Questions at Scottish Parliamentary Constituency, Westminster Parliamentary Constituency and Electoral Ward level. 

• Perceptions of Local Crime
• Disability
• General Health
• Mental Wellbeing
• Smoking
• Unpaid Care Provision
• Household Tenure

We intend to publish more small area datasets on

• electric prepayment meters
• households not connected to gas network
• electricity consumption
• gas consumption
• flood risk areas

We intend to publish a number of datasets which are used to produce the Scottish Index of Multiple Deprivation (SIMD) as linked open data at data zone level when it is published in early 2020 and we are collaborating with analysts who work in this area to ensure that these datasets are published. 

We have met with Scottish Government procurement colleagues who are keen to publish contractual expenditure data onto statistics.gov.scot.  We can currently publish these datasets as flat files, from which our colleagues are keen to produce interactive apps which illustrate expenditure.  We are working with our open data publishing contractors to develop improvements in the publishing system so that this type of data can be more easily published as 5* linked open data.    This will ultimately determine the success as to how well we achieve this milestone.  We are also in discussions with Transport Scotland around publishing vehicle charging data on statistics.gov.scot.  

We have already published data on energy performance certificates as flat files on statistics.gov.scot, and these data are kept regularly up to date.

Earlier this year, we delivered training for analysts to produce visualisations and apps and profiles directly from our open data portal, statistics.gov.scot.   We asked analysts to identify a business need for building products from statistics.gov.scot.  We ran a follow-up event in which analysts learnt from each other when developing products from statistics.gov.scot.  The intention is to build capacity and skills across data producers in the Scottish Government and elsewhere. There are 6 products in progress.   A number of apps have now been built directly using data from statistics.gov.scot – for example the Scottish Household Survey app , the GDP app, and the Scottish Centre for Regional Inclusive Growth dashboard.   We have built an R package to make it easier for users to develop apps and dashboards directly from statistics.gov.scot.

As mentioned above, we have delivered training for analysts to produce visualisations and apps and profiles directly from our open data portal, statistics.gov.scot.    

The Scottish Government sits on the Profiles Collaboration Group, which is represented by a number of public sector organisations who produce local area profiles and we are sending a survey to assess discoverability of profiles.  We encourage people building profiles to use common standards and definitions, to allow easier reuse of data and discoverability. 

Users of official statistics data often need data from many sources, and find it both frustrating and time consuming to have to search for data across multiple portals/websites, and then process into a consistent format ready for analysis.  

We are working with other organisations such as NHS Scotland and DWP to automatically harvest their data across from their open data portals using APIs onto statistics.gov.scot.  This means that data only need to be uploaded once, and the data can be reused.   This allows us to meet user needs by letting them get their hands on data about Scotland, covering diverse topic areas and in a consistent format on statistics.gov.scot. At the same time, it minimises the burden on teams of data loaders in other organisations by only requiring them to publish to one data portal, for which they have internal support.  

We have completed a successful pilot study using 12 datasets on care homes and a dataset on alcohol related hospital discharges, which have been initially produced by NHS Scotland which we have republished onto statistics.gov.scot

 We are conducting a systematic review of metadata in all datasets and we will working with data providers to ensure that the metadata are of sufficient quality, and highlighting examples of good practice to users.     We ask users to include the following details as part of their metadata.    

• Description of dataset
• Confidentiality Policy
• Quality Management
• Accuracy and Reliability
• Coherence and Comparability
• Accessibility and Clarity
• Relevance
• Timeliness and Punctuality
• Revisions

They should be consistent with common data standards and protocols wherever possible. This allows for better interpretation and reusability of data.

Since the start of 2019, we have worked with statistics producers to review and improved the metadata for their datasets which they are publishing.   This has amounted to around 200 datasets in total. 

On 26 April 2019, the Scottish Government chaired a roundtable meeting on Open Data and Data Literacy. This featured over 30 representatives from central and local government, the private sector, academia, the third sector and community groups and we discussed the challenges people are facing with data literacy and in accessing open data.  This will help shape how we make our data more discoverable and useable going forward.   We have published minutes from the meeting along with a short blog.  

Following on from this meeting, we held a meeting with community groups and local authority representatives to discuss the opportunities of using, accessing and understanding data for empowering communities at a local level for decision making. On the back of this, we have had more focused meetings with specific organisations such as the   Corra Foundation  , who support grants to communities with a particular interest in place-based working.  We explained the benefits of embedding open data into analytical briefings.