Demand optimisation in laboratory medicine: phase two report

5 NDOG Output

This section details the overall activity and output of the NDOG in relation to the aims stated above. The NDOG met on 4 occasions through 2018/19. During this time, concepts around the identified activities and pilot work from phase I were taken further, in particular around the collection of data and development of an Atlas of Variation.

5.1 Atlas of Variation for Diagnostic Laboratories Background

The NHS England Diagnostic Atlas of Variation was first published in November 2013 and focused primarily on diagnostic tests from Laboratory Medicine. The Atlas demonstrated significant variation in some diagnostic requesting patterns. Although some of the variability may be attributed to differences in patient populations, disease incidence/prevalence or regional differences in information recording, these reasons cannot wholly explain the differences observed in the Atlas. Some of the variations in test requesting can only be attributed to variable requesting patterns, which may reflect an inconsistent approach to patient management across practices. 

This variation observed in the English Atlas of Variation is likely to be replicated across the United Kingdom and similar unwarranted variations in diagnostic test use would exist across NHS Scotland. As such, the collection of laboratory testing data from across Scotland was seen as being vital to determine and identify potential unwarranted variation both in Primary and Secondary Care. This data was not just for observational purposes but to identify and enable interventions to be put in place to improve and reduce under/over-requesting and equity of access to new tests.

Development

During phase I of the Demand Optimisation programme, a prototype Atlas had begun; using data on laboratory tests collected from NHS Grampian and Tayside in order to test proof of concept. There was significant development of this initial prototype in Phase II. A number of tests were targeted for data collection and incorporated into the Atlas. Using Tableau software, data was presented in dashboards that provided snapshots of laboratory requesting activity and allowed comparisons between the two NHS Boards and the GP practices within.

The clear demonstration of requesting variability provided the basis for the key recommendation from the Demand Optimisation report that data collection is a key enabler for all future work and that the Atlas should be expanded to include data from all disciplines from all NHS Boards for an overall Scotland view. The Atlas developed significantly throughout Phase II and, whilst there will always be scope for ongoing development, the initial version of the Atlas has been completed and is fully operational. 

Methodology

NMDNs are specialty focused networks, comprised of specialty specific health professionals, that support diagnostic services to continuously improve service delivery, and contribute to equitable provision of high-quality, clinically effective services. In collaboration with these key health professionals, a list of tests identified to be most likely to show variation within and across boards from each discipline was produced. The test data identified was collated from primary and/or secondary care and may not necessarily have been the most common tests carried out by each discipline, but the ones where unwarranted variation in requesting was suspected or was deemed important. The tests that were included can be seen in table 1.

Table 1: Tests identified by the networks for data collection

Biochemistry tests	Haematology tests	Immunology tests	Microbiology & Virology tests
Alb:Creatinine Ratio	B12	AGPCA	High Vaginal Swab
Bone Calprotectin	Coagulation Screens	ANA	Urine – Catheter
C-Reactive Protein	CRP	ANCA	Urine - Mid stream
CA 125	D-Dimer	APL-anti-cardiolipin	Urine – other
CEA	DRVVT	APL-beta 2 glycoprotein
Cholesterol	ESR	C3/C4
FSH	FBC	CCP
Free T3	Ferritin	CTDS/ENAS
Free T4	Folate	Ds DNA Ab
Glucose (incl. fasting)	Plasma Viscosity	Electrophoresis
HbA1C	Protein	IF
HDL Cholesterol	Rheumatoid Factor	IgA TTG
Liver Function Tests	Serum Free Light	IgE – Aspergillus
Protein	Thrombophilia Screen	IgE - Birch
PSA	Urine Bence Jones	IgE - Cat
Testosterone		IgE - Dog
TPO Antibodies		IgE - Egg
Triglyceride		IgE - Grass
TSH		IgE - HDM
Urea & Electrolytes		IgE - Milk
Vitamin D		IgE - Peanut
		IgE - Total
		IgE - Wheat
		IgG/ IgA / IgM
		IgM RF
		Liver autoantibody - AMA
		Liver autoantibody - ASM
		Liver autoantibody - LKM
		PR3/MPO
		TPO

Data on individual GP practice and non-GP requesting rates for laboratory tests were collected for an entire year from the Laboratory Information Management Systems (LIMS) of all NHS boards. Having this data at practice level meant it could be combined with information including GP practice list size, cluster groups, SIMD deciles and population demographics e.g. practice patient age distribution, to enable meaningful analysis and comparisons to be made.

The data was extracted from the LIMS by executing a standard query and populating an Excel spreadsheet for further analysis. The data was visualised in Tableau visualisation software to enable a variety of analyses and comparisons to be made. 

There has been the opportunity over phase II of the programme to demonstrate the interactive Atlas to a number of stakeholders including clinicians, laboratory managers, referring clinicians and Scottish Government (see Appendix B for a full list of engagement). This has helped to shape the content and views of the Atlas ensuring it meets stakeholders’ needs and is developed into a useful tool both for service users and laboratories.  

Results

5.1.1 Diagnostic Laboratories Atlas of Variation – Static Observations

Given the large number of tests identified to be of interest (Table 1), and the potential for many observations and correlations to be made between NHS Boards, GP clusters and individual GP practices, the following represent only some examples. All figures represent comparisons between NHS Boards (Box and Whisker style) at the top with a bar chart showing the spread across all GP practices.

5.1.1.1 Free T4 and Thyroid Stimulating Hormone (TSH)

The thyroid derived hormone Free T4 (fT4) is used for screening, diagnosis and monitoring of thyroid disease. The pituitary derived hormone TSH is used for screening, diagnosis and monitoring of thyroid disease.

Commentary

Large variation exists between the NHS Boards. This likely reflects the existence of different Thyroid requesting profiles offered by the different laboratories, with some offering both TSH and fT4, and others only TSH as first line. This difference does not appear to be present with TSH testing (see below). This remains clinically controversial but effort could be made to allow a consistent Thyroid requesting protocol across all Boards to be implemented.

Much variation also exists between GP practices as seen in the lower figure. This is also present for TSH requesting. This variation is likely to represent differences in requesting by individual GPs. Much of this may be due to unwarranted requesting, both under and over requesting. The Specialty networks could be directed to engage with and support clinicians to develop guidance that could be targeted to identified practices to promote optimal requesting.

5.1.1.2 Haemoglobin A1c (HbA1c)

HbA1c is used in the screening, diagnosis and management of diabetes.

Commentary

Minimal variation exists between the NHS Boards, except for higher requesting evident for Ayrshire and Arran. There is however significant variation between GP practices within some of the NHS Boards as is evident by the Box and Whisker vertical height. This may reflect differences in test use guidance made available to GPs or the varying adoption of HbA1c for diagnostic purposes.

Much variation also exists between GP practices as seen in the lower figure. This variation is likely to represent differences in requesting by individual GPs. This may be due to unwarranted requesting, both under and over requesting. Evidence would suggest that both under and over monitoring of diabetes using HbA1c can lead to sub-optimal outcomes for the patient. The Specialty networks could be directed to engage with and support clinicians to develop guidance that could be targeted towards identified practices to promote optimal requesting. A consistent approach to the use of HbA1c for diagnostic purposes could also be developed.

5.1.1.3 B-Type Natriuretic Peptides (BNP or ntProBNP)

Natriuretic Peptide measurements are used in the diagnosis and management of Heart Failure.

Commentary

Natriuretic peptides are relatively new tests to NHS Scotland. Their main use is in the triage of patients presenting with symptoms of possible heart failure. Their implementation has been obstructed due to the lack of suitable funding mechanisms to enable laboratory budgets to pay for the tests when the benefit is felt across the wider NHS, notably for echocardiography referral patterns.

Clearly many NHS Boards are still registering little use, which may reflect lack of availability, awareness or the test being offered as a Point of Care testing version within GP practices. This data therefore allows the monitoring of the uptake and availability of such tests.

It is notable that for the NHS Boards that do register that this test is available, there is both considerable differences in requesting rates between the boards and between the individual practices. This likely reflects differences in mechanisms for test request gatekeeping, clinical awareness of the test and potential unwarranted variation due to cross-systems reasons.

Specialty networks could be tasked with developing consistency of testing pathways and guidance across the boards and also attempt to improve future data accuracy by capturing any point of care testing activity.

5.1.1.4 Serum Vit B12 and Folate

Vit B12 and Folate are used in the investigation of anaemia.

Commentary

Significant variation exists in the requesting of both B12 and folate. This is evident both between and within NHS Boards. Much of this variation may be due to unwarranted under and over requesting of these tests.

Overall, the variation between all GP practices is large, as seen in the lower figure. The Haematology network could be tasked with engaging with and supporting clinicians to develop testing strategies for GP practices exhibiting outlier requesting behaviour.

5.1.1.5 Mid-Stream Specimen of Urine (MSSU)

The MSSU test is used in the diagnosis of urinary tract infections.

Commentary

Reasonable consistency is demonstrated between the NHS Boards in terms of requesting rates per 1000 patients, however large variation is evidenced in rates between GP practices both within each Board but also across the entire country.

The Microbiology and Virology Network could be tasked with developing requesting guidance that could be made available generally but also for those practices at either extremes of the requesting rate graph.

Overall points

It is clear that the availability of this kind of static data is very useful in identifying potential unwarranted variation in the requesting of laboratory tests related to over requesting, under requesting and lack of availability or awareness of specific tests. Matching this with a range of demographic data further supports analysis. It remains important that such observations are considered to assess their importance and to allow development of specific interventions aimed at reducing such variation. It would be the intention of the NDOG to work with the national Specialty networks within phase III of the programme to develop and implement such interventions.

It also needs to be emphasised that the data is not perfectly clean. Significant barriers remain in place that inhibit efficient, accurate data collection:

• Dysfunctional Laboratory Information Management Systems (LIMS).
• Lack of standardisation of nomenclature between NHS Boards and even within.
• Lack of standardisation of laboratory processes.
• Inconsistency in specific discipline location for certain tests.
• Lack of time for staff to manually collect data, especially given the competing pressures of the National Laboratories Programme.

Within Phase III of the NDOG programme, work is underway to further refine this process with collection of 2018 data. As an aside, there are moves to develop a national procurement process for a common LIMS with a linked standardisation workstream. While these are unlikely to deliver for many years, it is clear that the goal should be to move from manual extraction of data to electronic automated real time extraction directly from LIMS.

5.1.2 Interactive Atlas

In order to enable healthcare professionals (including those within primary care) to have access to the data within the Atlas of Variation and be able to interact with the different views of data; an interactive portal has been developed. The level of information contained within the Atlas allows comparator data be used such as cluster, board and national analysis.  Going forward into phase III there has been some interest in including test cost information which allows calculated financial comparator for GP practices per list size be made.  In order to take this forward, there would be some standardisation work required to ensure that there is consistency in cost calculation across NHS Boards.

Following stakeholder engagement (see appendix B for a full list of meetings) it was agreed that the end product was not to be overly complicated, was accessible and had the capability to deliver interventional change following observed unwarranted variation. As such, the decision was made to create a series of different views which would be linked to roles. Much has been achieved in terms of stakeholder engagement but it is envisaged that during phase III of the NDOG programme, this will be refined by collaboration of the NDOG with both specialty lab group and specific laboratory user groups and patient representatives. 

The developed interactive Atlas demonstrated the requesting patterns of laboratory tests at individual GP practice, cluster, region, NHS Board and Scotland wide. In addition, demographic data including SIMD deciles and population size were included to enable more appropriate analysis and comparisons to be made. 

The Atlas developed included four main dashboards, tailored to the specific user to view the most relevant information.

The dashboards generated were: -

• National Scotland wide overarching view 
• NHS Board primary care view per discipline 
• GP and cluster view per NHS Board
• Scotland wide and NHS Board practice requests 

In total, test request data from 94% of all practices in Scotland have been collated and added to the Atlas. Data was adjusted per 1000 population unless otherwise stated and variation in practice requests from the median were presented using box and whiskers plots. All the data are driven by the specific test chosen and available filters allow the dashboards to be tailored to the specific need. Filter options include viewing data by:

• Scotland-wide
• NHS region (South East, West, North)
• NHS board
• cluster group 
• practice 

The data may be colour coded by:

• age
• deprivation index
• outliers 

5.1.2.1 Atlas of Variation- National Scotland wide overarching view 

Upon entering the Atlas, the first dashboard comprises of two figures (

Figure 2); 

• a map of Scotland fragmented by NHS Boards
• a funnel plot demonstrating the selected test coloured by request incidence to highlight outliers. 

The figures are driven by the specific test chosen (‘test name’) and filters will allow for delving either by region (north, west or south east) or by NHS Board. The funnel plots can be colour coded to identify the outliers or to colour code all the practices according to the deprivation index or by age profile (‘colour overlay’). 

As per every dashboard, if a guideline for the specific test is available, a direct link is provided. In addition, further information for each section can be accessed using the ‘show info’ button.

Figure 2: Atlas dashboard at national level

This dashboard allows for an overarching Scotland wide view and provides a snapshot of the selected test requests across Scotland. Hovering over a specific NHS Board on the map of Scotland will show the requesting pattern of all the GP practices in the funnel plot. Similarly, hovering over a specific dot on the funnel plot, which represents a single practice, will highlight the practice information and the location on the map of Scotland. Using this dashboard may demonstrate tests where variation may not be prevalent (fig 3a), where variation may require further investigation (fig 3b) or provide at a glance the individual practices that are obvious outliers (fig 3c)

Figure 3: National dashboard demonstrating requesting variation of a specific test at practice level

Similar information can be viewed by changing the detail level to cluster or to board (Figure 4) 

Figure 4: National dashboard demonstrating requesting variation of a specific test at board level (a) and cluster level (b)

5.1.2.2 Atlas of Variation- NHS Board primary care view per discipline

Highlighting a board on the national level map or the Board/cluster view tab will direct the user to a dashboard providing greater detail of the selected test (Figure 5). The Board view includes three elements; 

• a coloured cluster map that highlights the cluster areas in the selected board
• a funnel plot that shows the selected test
• a practice table detailing all the practices in the selected board. 

As in other dashboards, the funnel plot may be tailored to compare the selected test to either the other practices in the board or to the other practices within its cluster. 

The practice table includes additional information, including the practice list size, total practice requests and practice requests per 1000 population. 

Figure 5: Atlas dashboard at NHS Board level

Delving deeper into the NHS Board dashboard funnel plot, the user, at a glance, can compare practice requests and compare this to the practice list size (Figure 6). An example below highlights the test request of 3 practices that are of a similar size (~12,500) in one board, yet the requesting rates differ in excess of 25x fold. This may highlight to the user that practice across this board may require further investigation.

Figure 6: Practice sizes with differing request rates

5.1.2.3 Atlas of Variation- GP and cluster view per NHS Board

Opening the GP dashboard will open the GP view, which includes the most pertinent figures most useful to a GP (Figure 7). The dashboard includes:

A GP practice histogram - Presenting the age distribution of the selected group 

A box and whiskers plot - Presenting the selected cluster spread of variation

A GP practice line chart - Plotting the selected practice (blue) in relation to the average requests of its associated cluster (lilac), board (green) and national figures (pink). 

In addition, the information box (blue fill) includes the GP practice the list size, the yearly requests and SIMD16 decile score. A useful slide bar chart below the information box represents the selected practice average request rate compared to the average request of its affiliated cluster.

Figure 7: Atlas dashboard at GP level

The GP dashboard can effortlessly demonstrate the test requesting patterns of a GP practice in comparison to its own cluster, board or National average. For example, concentrating on the histogram alone, Figure 8 shows the requesting patterns of two practices within the same cluster. The GP requesting pattern (blue) of figure 8a is consistently higher than its own cluster average (lilac), where as in the figure 8b the requesting is consistently lower. As a standalone, this suggests that there may be inconsistencies in the test requesting. However, when addressing the additional information, the population distribution age is overall ‘younger’ in the practice with the fewer test requests which may be a contributing factor. Additional information relating to the population group including SIMD 16 decile scale may further provide a more rounded picture. Overall however, the reasoning underlying the differences in test requesting cannot be wholly explained by the Atlas alone, it may only be used as a tool to highlight the differences and provide any additional information that will aid the user to reach an informed conclusion.  

Figure 8: Histogram demonstrating two practices from one cluster group with differing request rates (blue) over a 12 month period  

5.1.2.4 Atlas of Variation- Scotland wide and NHS Board practice requests 

This dashboard presents two elements (Figure 9):

• test request per practice within its own board and in comparison to other boards in a box and whiskers format. This demonstrates Board variation. 
• all the practices in Scotland by request, highlighting national variation.

Figure 9: Atlas dashboard for practice request comparison

As per the discussion in the static observation section above (section 3.1.1), this dashboard represents comparisons between NHS Boards in a box and whisker style plot and the spread of all GP practices in the lower bar graph. This provides a quick snapshot to determine the fold difference in variation at NHS Board level and at GP Scotland wide.

Minimal or no variation across the NHS Boards will portray the boxes aligned horizontally. The greater deviation from the horizontal ideal will demonstrate greater variation of the selected test across Scotland while the dots outwith the whiskers represent the outliers within each NHS Board. 

The practice histogram provides instant visualisation to the spread of the practice test request and an instant visualisation for highlighting unwarranted variation.

Future Development

A great deal has been achieved with the development of the Atlas, however in order to realise its full potential, the Atlas must be in regular use by referring clinicians and lab staff. 

Atlas development has been limited by individual laboratories’ capacity to provide data; consistency of data definitions, meaning that the accuracy of comparisons cannot be guaranteed; and lack of engagement with referring clinicians and capacity to engage. 

Over the course of the next phase, there is a clear need for the delivery of an Atlas of Variation with data from the subsequent year, collecting information across all lab disciplines. This will demonstrate the outcome of changes implemented from QI programmes in requestors’ behaviour, which will in turn lead to waste reduction and tackling unwarranted variation in line with the principles of values-based healthcare.

Specific objectives for the future phase include: -

• Data collection for the calendar year of 2018 to build upon the existing data collected in phase II
• Engaging with referring clinicians to continue to improve the Atlas of Variation and promote its capabilities within the GP community, including a supportive education programme to enable use of the Atlas
• Direct access to the Atlas for laboratory professionals, allowing them to interrogate their own data and develop their own QI strategies in response to this
• Exploring alternative options to streamline data collection for populating the Atlas in the longer-term.