National Demand Optimisation Group (Ndog) – Phase 3 Report

5 NDOG Output

This section details the overall activity and output of the NDOG in relation to the aims stated above. The NDOG programme was unfortunately paused early in 2020 due to developing impact of COVID-19, hence the scope of the programme has been limited with some objectives not fully met. Despite this, the full NDOG met on three occasions through 2019/20, with further regular meetings of a smaller NDOG core group throughout the period. During this time the Atlas was further refined and concepts around the identified activities and pilot work from Phase II were progressed.

5.1 Atlas of Variation for Diagnostic Laboratories

5.1.1 Background

The NHS England Diagnostic Atlas of Variation was first published in November 2013 and focused primarily on diagnostic tests from Laboratory Medicine. The Atlas demonstrated significant variation in many diagnostic requesting patterns which could not wholly be explained by differences in patient populations, by demography or by disease prevalence/incidence. It was expected that a similar observation in variation would be replicated across NHS Scotland.

Phase I Demand Optimisation reported a limited pilot of data extraction was possible and that similar variation was observed to that across NHS England previously. During Phase II of Demand Optimisation a list of tests was identified by the representatives of specialty focused networks and laboratory disciplines. The criteria for including a test was based on identifying the ones where unwarranted variation in requesting within and across boards was suspected or was deemed important, rather than the most common tests carried out. Data was initially collected from all NHS health boards for the 2017 calendar year for selected tests from primary and secondary care, and combined with additional demographic information. Data for 2018 was subsequently added in Phase III, with 2019 data under collection.

Using Tableau software, a prototype Atlas for Scottish Laboratories was produced and developed extensively throughout Phases II and III. The Atlas comprised of a number of summary dashboards that provided snapshots of laboratory requesting activity and allowed comparisons between NHS Boards and the GP practices within.

5.1.2 Detailed development and data extraction

The interactive dashboards were devised to provide laboratory and health professionals with a variety of ways to identify requesting variation, align to appropriate test use and to provide a starting point from which to develop targeted interventions aimed at reducing unwarranted variation. The importance of facilitating targeted interventions to drive optimal testing was clear but was in turn dependent on building on existing data collected and further refining the Atlas dashboards views for referring clinicians.

At the end of Phase II, the Atlas held data for the calendar year of 2017. It was agreed that building upon the existing data with data from the calendar year of 2018 was imperative for continuation into Phase III (see Appendix B for list of tests). A significant challenge remained around further data collection given the absence of automated real time extraction directly from Laboratory Information Management Systems (LIMS). A national procurement process for a common LIMS with a linked standardisation work stream is in progress, however it is unlikely this will be available for several years due to the scope of the development. As such, the extraction remains an arduous manual process undertaken by laboratory managers. To reduce the burden on individuals it was agreed that a tally of the yearly requesting would be acceptable rather than monthly breakdowns.

The data was extracted from the LIMS by executing a standard query and populating an Excel spreadsheet for further analysis. In total, test request data from practices across Scotland was collated for 2018 and added to the Atlas. As previously, data was adjusted per 1000 population unless otherwise stated. Comparative variation in requesting for each test between GP practices was displayed using simple bar charts, with variation across health boards displayed using box and whiskers plots.

During Phase III, the impact on colleagues' time for data collection was more evident. Data collection to populate the Atlas is pivotal for the programme and as such it was imperative that the process was as efficient and easy as possible. A meeting with the laboratory managers responsible for data collection was organised to identify any steps that could be put in place to further simplify the process. This led to the following recommendations:

• Data may be sent in the format that is extracted from the system with minimal/no requirement from the lab managers to adapt to fit in to a prescribed spreadsheet.
• Clear definitions of what to collect for each test to ensure all NHS Boards collect the same set of tests.
• One request for collection of all tests for all disciplines/specialties rather than individual requests per discipline.
• Monthly collections per financial quarter was acceptable as some health boards have or in the process of acquiring automated scheduling facility. A schedule of upcoming requesting dates would be advantageous.

Following feedback and amendment, data collection for 3 quarters of 2019 was also collected from six NHS Boards, covering 87% of all practices in Scotland with an additional four health boards providing data for the fourth quarter of 2019 and the first quarter of 2020.

5.1.3 Atlas dashboards

Throughout all Phases of the Demand Optimisation programme there has been significant engagement in the generation and development of the Atlas dashboards. In Phase III, there was increased engagement with primary care (see Appendix C for a full list of meetings) as the intention was for the Atlas to be piloted with referring clinicians within one NHS Board for utility and suitability in highlighting test variation. Using the simplicity of existing antibiotic prescribing feedback as a standard, the GP Atlas dashboards were refined to two specific dashboard views:

• GP practice variation of all tests
• GP comparator view, comparing practice with peer groups, cluster, NHS Board, region and Scottish average

GP specific variation of all tests

Upon opening the Atlas, the first dashboard includes a plot demonstrating the five highest and 5 lowest test requests (for the chosen practice compared to the health board average; Figure 2). This is not the highest or lowest number of requested tests but rather the 10 tests that the requesting pattern for the practice is least like their NHS Board average. Hovering over the blue bar of each test will bring up key information such as the national, board and GP practice average and a link to current guidelines if available. The option to view all the current 70+ tests included is available using the outlier filter mode.

The specific practice can be selected either by using the GP practice name or practice code using the filter at the top and a link to IMS is provided for comments, feedback or support. Furthermore, there is a link to NES shared decision making for additional resources.

This dashboard allows the user to easily identify the key tests where requesting patterns of the practice are least like the requesting pattern of their NHS Board overall. This can only be used as an indicator to direct the user to narrow their search from the list of tests. Clicking on the specific test on the dashboard provides more detailed information for this test in the GP comparator view.

GP comparator view

Clicking on a specific test in the previous dashboard brings up the GP comparator view, which provides greater detail for the test in question (Figure 3). A GP practice line chart plots the selected practice requests per 1000 patients (blue) over the previous 3 years. Feedback from primary care indicated that a number of different comparison options would further enhance the data when reviewing their practice requesting pattern. As such, the plot includes the average requesting pattern of the practice's associated cluster (turquoise), its peer group (green), NHS Board (lilac) and national average (red). In addition, an information box across the top of the screen (blue fill) includes the GP practice list size, the number in the peer group and the Scottish Index of Multiple Deprivation 16 Score Decile (SIMD16 decile score).

The GP comparator view demonstrates the test requesting patterns of a GP practice versus a variety of peer, regional and national benchmarks. While not specifically identifying inappropriate requesting per se, the tool highlights possible test areas for review. Additional educational materials, regional and/or national guidelines and specific user knowledge of the patient population and demographics are pertinent to draw conclusions.

5.1.4 Access

Following user feedback, it was agreed that for maximum ease and usability, online access to the Atlas should be open. However, there was uncertainty on whether in some small practices where very few tests are requested it was possible to determine the individual patients. To eliminate this, the raw number of all tests that were 5 or below and all those collected in future would be hidden with only the tests that are greater than 5 displayed.

A Data Protection Impact Assessment (DPIA) Screening was then carried out to identify any further issues. However, it was deemed that any information governance risk was negligible and so the programme was given permission to upload the Atlas online with no additional password protection.

5.1.5 Educational material

It was identified early in the programme that the Atlas is a valuable tool in highlighting unwarranted variation. However, as a standalone tool it lacks the educational support to facilitate and promote appropriate test use. Providing educational guidance in accompaniment to feedback of laboratory test requesting in not a novel concept. NHS Grampian piloted providing educational feedback with laboratory test requesting rates as part of a randomised controlled trial in 2006^[5]; this practice has now become routine with tailored feedback documents detailing requesting rates for laboratory tests sent to all GP practices across NHS Grampian every 2 months. More recently, antibiotic prescribing data is also now routinely sent to all GP practices presenting their antibody use in comparison to their NHS Board and across Scotland, with educational material highlighting the prescribing guidelines. This provides the GPs with all the tools to view their own practice and the resources to review their requesting/prescribing patterns.

To produce the supporting educational information, there was substantial collaboration with key representatives from the laboratory networks and primary care. Discussions concluded that the best format would be a one page 'flash report' for each test that included the specific practice's comparative requesting patterns, followed by key indicators on when to test and when not to test, with additional links to further resources. The educational material/guidance would ideally be devised on a national basis but would allow regional or board variation if required.

Interrogation of the interactive Atlas highlighted 3 tests that showed clear variation, namely thyroid function tests (TFTs), vitamin D and HbA1c. The first set of flash reports were produced, an example of the TFTs report can be observed in Appendix D.

Sending flash reports directly to every GP will ensure that referrers are given the option to receive specific educational material in their preferred manner and encourage users to access the Atlas to review their variation rates for their practice. However, implementing an electronic automated distribution system has its challenges and needs further exploration to be fully achievable. As an initial step, flash reports are being produced on an NHS Board basis to include a graph at the top showing the Scotland average and that NHS Board's average for the test in questions over a set time period. There will be links into the practice views.