Open Government Action Plan commitment 3: improving how we share information

The National Performance Framework measures and keeps track of how Scotland is performing in relation to the Scottish Government's purpose and national outcomes.

The government's purpose is to focus on creating a more successful country with opportunities for all of Scotland to flourish through increased wellbeing, and sustainable and inclusive economic growth.

We have developed an mechanism for publishing datasets and their relevant equalities data breakdowns on the National Performance Framework onto our Official Statistics Open Data Publishing Platform.     

We have now published the National Performance Framework as 3* csv file on statistics.gov.scot. This is an efficient publication approach which we have agreed with the National Performance Framework team.  It is being updated on a regular basis, to coincide with updates to the National Performance Framework indicates.  It is intended that this will link with the Equalities Evidence Finder, which is published by the Scottish Government.

The Scottish Government publishes a  12 month calendar  of its forthcoming statistical releases in spreadsheet format; this includes both Official and National Statistics publications; this is in line with the requirements of the  Code of Practice for Statistics.

We are working with colleagues to identify the best way of collecting this information.  This could help shape the next steps of open data publishing. 

🙂 On track 

As at 18 February 2020, there are 39 data zone level datasets published on www.statistics.gov.scot. Since the launch of Scotland’s Open Government Action Plan 2018-2020, we have updated 35 data zone level datasets, including 6 new datasets.

The 6 new datasets we have published at data zone level are:

• Scottish Index of Multiple Deprivation 2020
• Personal Tax Credits: Number of Children
• Personal Tax Credits: Number of Families
• Scottish Index of Multiple Deprivation - Education Indicators
• Scottish Index of Multiple Deprivation - Housing Indicators
• Scottish Index of Multiple Deprivation - Health Indicators

We have also published historic SIMD data (such as 2016 SIMD) as a separate linked open dataset on www.statistics.gov.scot. 

We also publish data from the Scottish Survey Core Questions at Scottish Parliamentary Constituency, Westminster Parliamentary Constituency and Electoral Ward level. 

• Perceptions of Local Crime
• Disability
• General Health
• Mental Wellbeing
• Smoking
• Unpaid Care Provision
• Household Tenure

We intend to publish a small area dataset on hospital related admissions.  

We have met with Scottish Government procurement colleagues who are keen to publish contractual expenditure data onto statistics.gov.scot.  We can currently publish these datasets as flat files, from which our colleagues are keen to produce interactive apps which illustrate expenditure.  We are working with our open data publishing contractors to develop improvements in the publishing system so that this type of data can be more easily published as 5* linked open data.    This will ultimately determine the success as to how well we achieve this milestone.  We are also in discussions with Transport Scotland around publishing vehicle charging data on statistics.gov.scot. 

Earlier this year, we delivered training for analysts to produce visualisations and apps and profiles directly from our open data portal, statistics.gov.scot.   We asked analysts to identify a business need for building products from statistics.gov.scot.  We ran a follow-up event in which analysts learnt from each other when developing products from statistics.gov.scot.  The intention is to build capacity and skills across data producers in the Scottish Government and elsewhere. There are 6 products in progress.   We are aware of a number of apps which have now been built directly using data from statistics.gov.scot – for example the Scottish Household Survey app , the GDP app, and the Scottish Centre for Regional Inclusive Growth dashboard.   We have built an R package to make it easier for users to develop apps and dashboards directly from statistics.gov.scot, called opendatascot.

As mentioned above, we have delivered training for analysts to produce visualisations and apps and profiles directly from our open data portal, statistics.gov.scot.    

The Scottish Government sits on the Profiles Collaboration Group, which is represented by a number of public sector organisations who produce local area profiles and we are sending a survey to assess discoverability of profiles.  We encourage people building profiles to use common standards and definitions, to allow easier reuse of data and discoverability. 

Users of official statistics data often need data from many sources, and find it both frustrating and time consuming to have to search for data across multiple portals/websites, and then process into a consistent format ready for analysis.  

We are working with other organisations such as NHS Scotland and DWP to automatically harvest their data across from their open data portals using APIs onto statistics.gov.scot.  This means that data only need to be uploaded once, and the data can be reused.   This allows us to meet user needs by letting them get their hands on data about Scotland, covering diverse topic areas and in a consistent format on statistics.gov.scot. At the same time, it minimises the burden on teams of data loaders in other organisations by only requiring them to publish to one data portal, for which they have internal support.  

We have completed a successful pilot study using 12 datasets on care homes and a dataset on alcohol related hospital discharges, which have been initially produced by NHS Scotland which we have republished onto statistics.gov.scot

 We are conducting a systematic review of metadata in all datasets and we will working with data providers to ensure that the metadata are of sufficient quality, and highlighting examples of good practice to users.     We ask users to include the following details as part of their metadata.    

• Description of dataset
• Confidentiality Policy
• Quality Management
• Accuracy and Reliability
• Coherence and Comparability
• Accessibility and Clarity
• Relevance
• Timeliness and Punctuality
• Revisions

They should be consistent with common data standards and protocols wherever possible. This allows for better interpretation and reusability of data.

Since the start of 2019, we have worked with statistics producers to review and improved the metadata for their datasets which they are publishing.   This has amounted to around 200 datasets in total. 

We are keeping on top of international developments around metadata, such as the DCAT2 standards, and working with data standards colleagues to see how search engine optimisation and discoverability can be improved between different open data platform.  In this way, we hope that people and machines can find and reuse data more easily.

On 26 April 2019, the Scottish Government chaired a roundtable meeting on Open Data and Data Literacy. This featured over 30 representatives from central and local government, the private sector, academia, the third sector and community groups and we discussed the challenges people are facing with data literacy and in accessing open data.  This will help shape how we make our data more discoverable and useable going forward.   We have published minutes from the meeting along with a short blog.  

Following on from this meeting, we held a meeting with community groups and local authority representatives to discuss the opportunities of using, accessing and understanding data for empowering communities at a local level for decision making. On the back of this, we have had more focused meetings with specific organisations such as the   Corra Foundation  , who support grants to communities with a particular interest in place-based working.  We explained the benefits of embedding open data into analytical briefings.