Open Data Consultancy Final Report

Summary

This report presents the outcomes of the Open Data Consultancy study that Swirrl IT Limited was commissioned to carry out for the Scottish Government from September to November 2013.

'Open Data' is data that is accessible to anyone (usually via the internet), in a machine readable form, free of restriction on use. Adoption of this approach to information sharing is growing in the public sector, with anticipated benefits for transparency, efficiency of government and economic growth.

The study included a pilot of the Linked Open Data approach to data publishing, incorporating the Scottish Index of Multiple Deprivation, selected education data and supporting geographical information. Linked Data is a specific web-based technology for data access and data integration, sometimes known as '5-star' data. Linked Open Data is the application of linked data in an open context.

The pilot demonstrated that linked data publishing by the Scottish Government is feasible at modest cost. It provides flexible and powerful mechanisms for producing dynamic visualisations and downloads of data, as well as machine-readable access. The pilot demonstrated how data from different sources can be usefully interconnected.

A process of gathering user feedback has now begun to assess the reaction of data users to the techniques demonstrated in the pilot.

The study analysed what steps would be needed to extend the pilot to the whole of Scottish Neighbourhood Statistics; and concluded that this is feasible.

Linked open data helps make data more discoverable and makes it easier to combine data from different sources. These are challenges that are also encountered in the non-public information environment within the Scottish Government. The study assessed whether the technologies used for open data could also be a good solution for improved data management within the Analytical Services of the Scottish Government. In this context it is important to have control of security and versioning and to protect the investment in existing tooling and skills. Any new system must allow integration with various data analysis software packages.

Linked open data seems promising for internal data management, but is less proven in this context than for open data publishing. Therefore a small scale experiment would provide useful insight to assist a future decision on the suitability of this approach.

Finally, the report assesses what would be required to allow the Scottish Government to use the linked open data approach across all its data publishing activities. It presents a high level view of what such a change in practices would involve and what benefits it might bring.

The objective is more effective exchange of information, both internally and externally: an ambitious goal that will take time to achieve. It will depend on interoperability of many different data-holding systems. To enable that interoperability requires establishing a series of standards: existing open standards from the World Wide Web Consortium combined with the government's own standards and conventions, and supported by reference data sets. Finding the right balance of standardisation and flexibility for innovation should enable a successful ecosystem of data sharing.

This will need to be backed up with new knowledge and skills amongst government staff and a culture and goals where the success of a data curator is measured by how others use their data. The value of open data is created when someone puts it to good use: choices of technology and design of processes should be made with that in mind.