Care Home Data Review - Full Report

Data quality, Insight and Relevance

Data quality and accessibility were themes that repeatedly came through the discussions we held with stakeholders as part of the review. Linked to this were concerns expressed about the coherence of, and insight from, both the data collected and subsequent analysis.

Finding & accessing data

The 2020 OSR report Adult social care statistics in Scotland noted that:

“there are multiple social care statistics producers in Scotland. This can have negative consequences for users of these statistics, as they are required to go to different sources to get all the information they need about a topic, or if different sources provide similar but not necessarily consistent information about the same issues.”^[34]

Data users told us that they were not always aware what data and analysis was available, what was covered by the data or where it could be accessed. In addition, not all the data and analysis that could be publicly available is made available for public accountability. Furthermore, having data held in multiple places creates timeliness issues, whether it is having to collate (and link) data or synthesise analyses.

In order to improve the ease of finding and accessing care home data, Scottish Government has published information about all the data collected by national bodies in a central data mapper. This contains links to the relevant source documentation, along with basic information (or metadata) about each data collection, such as its purpose, frequency and provides information on the benefits of the data collection. Furthermore, Public Health Scotland has developed a Metadata Catalogue which is a central repository of publicly available health and wellbeing indicators for Scotland. The metadata catalogue should make it easier for people to find and use existing data intelligence relating to specific topics including information published on care homes and care home residents. The catalogue includes data from various sources including Public Health Scotland, the Office for National Statistics and the Scottish Government. The catalogue can be accessed via the ScotPHO website.

For local government data collections, the Local Government Online Data Returns Register (ODRR) aims to operationalise the mapping of all data returns undertaken in Phase 1 of the Local Government Data Platform project, to provide a searchable interface for users. The ODRR will be a key component of the future LGDP and will provide detailed information and metadata on each data return, including listing all data items within each return, in order to identify areas of duplication or potential linkages between datasets.

The register aims to support better co-ordination of data collections, by providing information such as collection windows, links to templates, guidance etc. Work is underway with Scottish Government, PHS and the Care Inspectorate to onboard all data returns related to Social Care, building on and expanding the work done on the care home data mapper.

Context and interpretation

In addition to care home data being available from a number of different sources, results are published in a number of different of formats, i.e. statistical reports, tables, spreadsheets and interactive dashboards. Whilst the variety of options for accessing data is generally considered a positive, feedback to the review noted that the format in which data were presented sometimes lacked sufficient narrative to “support [users] to understand what the data is telling us”^[35] and / or context and guidance on the appropriate use of the data.

Issues around context and interpretation are particularly important when, as noted above, there are a number of different data sources providing similar information. Stakeholders reported that there was an absence of coherent analysis about the care home sector.

“[data analysts] need to provide data insight, not data deluge.”

Professor Sir Ian Diamond, UK National Statistician

It is apparent that despite there being a number of data collections and relative abundance of care home data, some data users still struggle to be able to answer some of the key questions they have relating to care homes. A critical component to providing the insight required is for data producers to work with data users to understand and agree the key questions which we would like to answer. This will allow us to both assess whether we have the data to be able to answer them and, if not, how we may best go about obtaining the required data, if possible. It is also important that, where possible, analysis should attempt to bring together or reference the data available across a range of sources to produce a narrative of what the combined data are telling us on key topics, including contextual data from local data providers.

Standardisation and rationalisation of data collections should help in attaining a more consistent/coherent picture of care home data. However, where differences remain, data producers will need to work together to better identify overlaps, provide clear signposting to relevant data sources on a topic, provide information on how they piece together and provide details on when and how to use different sources.

Case study: Data for insights

The Scottish Government is currently undertaking a programme of work on ‘Data for Insights’ as part of the Health and Social Care Data Strategy to adopt a whole-system approach to creating insight from data that allow us to improve services.

To achieve the goal of greater discoverability, accessibility, interoperability, and reusability of data, planners need to have better access to appropriate and near real-time data, to better analyse, understand and improve processes and services at different levels and better inform strategic planning priorities.

In response to the recommendations from the ‘Creating Insights’ chapter of the Data Strategy, a Near Time Data Service was launched. This initiative is a collaborative effort involving the Scottish Government, Public Health Scotland and National Services Scotland.

The initial focus of the collaborative has included the successful launch of two new management information dashboards in time for Winter 2023/24. The two dashboards are:

• Whole Systems Winter Dashboard (WSW)
• Social Care & Delayed Discharge Dashboard (SCDD)

The dashboards have initially been rolled out as Minimum Viable Products (MVP’s) with access granted to over 900 users, in time to support reporting on winter pressures. Work continues on how to improve data for insights including around how to visualise and share care home data.

Knowledge exchange and support

During the stakeholder engagement, people reported that there was limited support, including insufficient analytical staffing resource, lack of opportunities to learn from good practice, lack of training in using care home data and linked data, insufficient support in interpretating analysis.

“Support to understand what the data is telling us.”

There are a number of proposed actions identified by the review which should help address the limited support. Under the Strategic Oversight and Co-ordination chapter, the establishment of new Data Provider and Data User Networks should help provide opportunities to learn from good practice. Furthermore, the Local Intelligence Support Team (LIST) in Public Health Scotland (see box below), along with the Digital Office’s ‘Data Products and Services Catalogue Working Group’, and ‘Health and Care Data Returns Working Group’, provide opportunities for sharing good practice, enhancing data collection processes to be more efficient whilst meeting the needs of the user, and providing insightful data intelligence. Further work needs to be done to consider how best to support care homes in gaining access to appropriate learning and support, for example the Digital and Data Capability Framework on the TURAS site.

Furthermore, data producers all have a role in sharing their expertise in working with care home data (and linked care home data). Some of this knowledge exchange could be achieved via the new Data User Network.

Local Resource: Local Intelligence Support Team (LIST)

The Local Intelligence Support Team (LIST) at Public Health Scotland (PHS) provide data and analytical support to all 31 Health and Social Care Partnerships (HSCPs) and as many GP clusters concurrently as resources allow. LIST undertake a broad range of work with HSCPs to source, analyse and/or develop data and intelligence to support decision-making. A vital part of the LIST operating model (as demand frequently exceeds capacity) is that the work the team undertakes is agreed with key contacts in each HSCP area, reflecting where they feel LIST input can best add value, in complement to local capacity and knowledge.

LIST support is tailored to requirements and can involve any or all of the following, depending on need:

• As much as possible, sourcing/analysing pre-existing, nationally collected data;
• If national data not (fully) suitable, sourcing/analysing pre-existing, locally held data (where granted access by local partners, in accordance with Information Governance procedures in that HSCP area);
• If pre-existing data not (fully) suitable, supporting the development and usage of bespoke local data.

Quality and completeness of existing data

The Code of Practice for Statistics states that the quality of data can be measured by their accuracy and reliability, coherence and comparability, and timeliness.

As part of the Care Home Data Review’s stakeholder engagement^[36], data users highlighted issues with the quality, timeliness and completeness of care home data.

Inconsistent data definitions / coverage

Potential inconsistencies in terminologies, definitions and coverage of data collections both across local areas and individual data collections limit the ability to produce consistent and comparable data. As previously noted, if data definitions are not consistently understood and applied then the data collected may not be useful for the purpose they are collected – and in some cases may be misleading. Some of these inconsistencies should be improved by the recommendations outlined in other chapters (i.e. Standardisation).

Users also noted that different collections sometimes cover different care home populations, however this is not always clear in the presentation of the data. Users requested that producers of statistical publications provide greater clarity around the population covered by the different data collections, for example, what care home residents are included or excluded.

Timeliness of access / publication

As noted in the Data Gaps section, the usefulness of data and analysis can be reduced by the length of time it takes to publish the analysis. The need for timely statistics was raised in the OSR report:

“Policy makers, care providers, residents and their families therefore need timely and insightful statistics to help make decisions.”

Understanding the different needs and uses of data will also allow us to evaluate the trade-off between the timeliness of data versus the quality of the data required. For some purposes, fully quality assured data is required to allow a full and rich analysis of the data, whereas in other cases the important aspect is having data “good enough” in time to inform decisions.

Data providers noted that, depending on the purpose, data can be out of date as soon as submitted. They noted that what is required is data that would allow forward planning based on future demand, which will require more context around the complexity of care.

Incomplete coverage / low response rates

Feedback to the review stated it would be useful to have information disaggregated to a more local / granular level to help inform local understanding and decision making. Users noted that data is not always available at the required level or where it is available, it is sometimes not deemed accurate enough to use. The Code of Practice for Statistics states that “data should be coherent across different levels of aggregation, consistent over time, and comparable between geographical areas, whenever possible.”

Data providers suggested that incomplete coverage or low response rates may be down to data providers not accurately knowing the information that is being asked for. For example, as part of the Care Inspectorate Annual Return, information is asked about Hospital Acquired Infection. Feedback to the review suggested that some care homes, particularly those without nurses, may not get passed on relevant health information when a resident is discharged from hospital to a care home, meaning data submitted by care homes may be incomplete.

To properly assess quality and completeness of data, organisations responsible for national data collections should undertake a full review of quality and completeness of each data source. For those data items / sources which are not considered robust, the organisations responsible for these data collections should:

• Work with the care home sector to improve or stop the data collection or variables within the data collection
• Investigate the potential to pause or stop some collections in order to divert resource to data and process development
• Investigate the potential to use data linkage to improve data quality and completeness (e.g. linkage of Scottish Care Home Census to National Records of Scotland’s deaths data)

Building trust in statistics

As noted previously, issues around data quality and completeness have led to some of the data collected not being reliable/ insightful enough to use. This leads to a situation where providers still have the burden of data provision, but this is not offset by any benefit as the data is not used. Where data quality is known to be an issue, it is important that data producers work with data providers and look to both improve the quality of their statistics and provide assurances to data users by being accountable and transparent, to increase the trust in the statistics.

The Code of Practice for Statistics sets out the standards for data producers and provides assurance to the public that the data collection, methods and publication uphold to best practices. In order to demonstrate they adhere to the highest data standards, national data collections which are pivotal in providing robust evidence for the care home sector and the public should adhere to the principles of the Code.

Addressing the problems relating to data quality and insight should be helped by the rationalisation, standardisation and communication recommendations.

Data quality, insight and relevance: Aims and recommendations

Aims: Improve the findability and accessibility of information about data collections, analysis and publications.

Recommendations: Bring together and publish information on care home data collected by national bodies in a central data catalogue. This will contain variable level data and show basic information (or metadata) about each data collection, such as its purpose, frequency and provide information on the benefits of the data collection.

Data producers to make available metadata for published datasets and share on open data platforms (e.g. Statistics.gov.scot) to increase visibility of care home data.

Aims: To work with the Care Sector to ensure that all data collections are timely, accurate and add value.

Recommendations: Data producers for national data collections to undertake a variable-level review of quality and completeness for each data source, and work with the care home sector to improve or stop the collection of data known to be of poor quality.

Data producers to consider the timeliness of data outputs and the potential to release less validated data as provisional outputs, if considered useful and not misleading.

Aims: Ensure we help readers to understand the key insights being derived from the analysis of data.

Recommendations: Data producers to use data users networks and other means to review products / outputs for the main data collections and ensure analysis is presented in a way that meets the needs of different data uses.

Data producers to work together to improve the coherence of published care home data in line with Office of Statistics Regulations (OSR) recommendations.

Aims: Work to increase trust in our statistics.

Recommendations: Data Producers to adhere to principles of the Code of Practice for Statistics for our key national data collections and publications.

Investigate the potential to use data linkage to improve data quality and insights (e.g. linkage of Scottish Care Home Census to National Records of Scotland’s deaths data).

Aims: Work with the wider sector to improve local context and improve insight and quality from data

Recommendations: Work with local partners to help share good practice and to provide additional support around making data collections more efficient, providing additional analysis and support for interpretating analysis.

Where appropriate, work with the research and innovation sector to improve insight & data quality.