Quality improvement and measurement: what non-executive directors need to know
Overview of what quality improvement and measurement is and how NHS non-executive directors can ensure that this approach is used by health boards.
Appendices
Appendix 1 – Quality Improvement methodologies
At the outset, it is important to acknowledge that QI methodologies are not the only types of change that bring about improvement. In some cases, wider changes will be necessary. These could include a change of legislation, or professional regulations. Similarly, the introduction of training to existing staff or introducing a new component to pre-registration courses so that new staff entering a system will bring a new skill will bring about change. QI methodologies work within the current legislative, professional and regulatory frameworks and challenging these is beyond the scope of these techniques. Parallel activity may of course work to make these changes.
Skill is required to discern which method is most applicable to the issue being addressed. For example, repeated actions that follow a particular sequence, such as the insertion of catheters, or the issuing of repeat prescriptions, are well suited to being improved through the Model for Improvement that requires rapid hypothesis building, testing and retesting. Redesigning pathways of care that are inefficient and unreliable may lend themselves to lean techniques. An example would be making the distribution of medicines throughout a series of hospitals more efficient.
It is important to recognise that QI usually takes time; the more people who are involved and more complex the environment the longer it will take to make and then embed improvements.
Board members are in a strong position to support wider cultural changes that will be necessary if these QI methodologies and the 'habit' of QI are to become part of the local culture.
The methodologies used most frequently in Scotland are described below.
Understanding the process – Process mapping
'Every system is perfectly designed to deliver the results it produces.'
Langley et al, 2009
Although technically not an improvement methodology, since a map is simply a description, process mapping has been included here, since developing a process map is often the first step in any improvement initiative. The sequence of steps in a process are identified and drawn. Quite often this act of investigating and describing a process will immediately show areas where there are inefficiencies or blockages. This can then lead to improvement action to 'unblock' and streamline the process.
To find out more about process mapping, visit:
https://improvement.nhs.uk/resources/process-mapping-conventional-model/
Model for Improvement
To learn more about the Model for Improvement, and to read about examples of how the Model for Improvement has been used in NHSScotland, visit:
http://ihub.scot/media/3256/a46965-20171124-handy-guide-to-the-model-for-improvement-v1-0-2.pdf
http://ihub.scot/about/empowering-improvement/
Lean
To read about how lean thinking is being applied in NHS England, visit the Lean Healthcare Academy:
https://www.england.nhs.uk/leadingchange/lcav-in-action/videos/
To learn how NHS Highland is using lean thinking to implement its Quality in Action work, visit:
http://www.nhshighland.scot.nhs.uk/AboutUs/HQA/QualityInAction/Pages/welcome.aspx
To learn how to reduce things that do not add value to patients, visit:
Six Sigma
To learn how Six Sigma can be combined with lean thinking to dramatically reduce waiting times, visit:
https://improvement.nhs.uk/resources/quality-service-improvement-and-redesign-qsir-tools/
For useful tips on how to use the 'five whys' to get to the root cause of problems, visit:
https://improvement.nhs.uk/resources/root-cause-analysis-using-five-whys/
To learn how to reduce things that do not add value to patients, visit:
Demand Capacity Activity and Queue ( DCAQ)
To learn more about Demand Capacity Activity and Queue, visit:
https://improvement.nhs.uk/resources/demand-and-capacity-overview/
The Vanguard approach
To learn more about the Vanguard approach, visit:
http://vanguard-method.net/the-vanguard-method-and-systems-thinking/
The following link shows how it has been used:
https://improvement.nhs.uk/resources/tcsl-programme/
Experience-based co-design
To find out more about experience-based co-design, visit:
https://www.pointofcarefoundation.org.uk/resource/experience-based-co-design-ebcd-toolkit/
Appendix 2a – The three reasons for measurement
Data and its measurement for improvement are used differently from data employed for judgement or for research. It is vital that Board members know the difference between these and the expectations on Board members for each type of measurement. In this video Mike Davidge explains how measurement for improvement is different to traditional measurements used in healthcare.
| Characteristic |
Judgement |
Research |
Improvement |
|---|---|---|---|
| Aim |
Achievement of target |
New knowledge |
Improvement of service |
| Testing strategy |
No tests |
One large test |
Sequential tests |
| Sample size |
Obtain 100% of available, relevant data e.g. percent of patients who have been seen within a predetermined waiting time |
'Just in case' data e.g. systematic analysis of an outpatients' invitation to attend notes to test the effectiveness of a new IT system |
'Just enough data', small sequential samples e.g. five case notes from yesterday – followed by five today to see if staff are actually undertaking and recording all three elements of a care bundle |
| Type of hypothesis |
No hypothesis |
Fixed hypothesis |
Hypothesis flexible, changes as learning takes place |
| Variation (bias) |
Adjust measures to reduce variation |
Designed to eliminate unwanted variation |
Accept consistent (random) variation |
| Determining if a change is an improvement |
No change focus |
Statistical tests, p-values |
Run charts or Shewhart control charts |
Adapted from: The Three Faces of Performance Measurement: Improvement, Accountability and Research. Solberg, Leif I., Mosser, Gordon and McDonald, Susan Journal on Quality Improvement. March 1997, Vol.23, No. 3.
Appendix 2b – Outcome, process and balancing measures
Not every activity in a change will result in a measurable outcome, so measures are developed to show the activity has taken place. These are called process measures.
For example, if a ward agrees that three patients are to be checked every two hours against the ' SSKIN care bundle' this is recorded as a process measure (which reflects how the system is working). A bundle is simply a set of actions that, when undertaken together, have been proved to address risks. In this case, the nurse checks the Surface, observes the patient's Skin, encourages her to Keep moving, ensures she avoids Incontinence and encourages Nutrition. The outcome measure would be the absence of a pressure ulcer developing (this reflects the impact the system is having on the patient). This outcome measure could be shown at a ward or care home level as the number of 'days between' a preventable pressure ulcer developing in their patients. Other examples of outcome measures are rates of falls, or hospital acquired infection.
Balancing measures enable a system to monitor any unintended consequences (good or bad) of the improvement effort and are particularly useful when there is a concern that a change might have an unintended negative consequence. In this example there could be a concern that patients may find the interventions disruptive and so the number of complaints received regarding pressure area care could be a useful balancing measure; there could also be concerns that staff may find the bundle burdensome and add to their workload, so staff satisfaction or time spent completing documentation may also be useful balancing measures.
Balancing measures are particularly useful when there is a concern that a change might have an unintended negative consequence. An example of this was in work to improve access to psychological therapies where clinicians were concerned that the focus on access might result in poorer quality treatment. A balancing measure focused on clinical outcome data was therefore developed.
Appendix 2c – Data presentation
This guide focuses on the interpretation of data so that action can be taken. Further information on the collection, analysis and presentation of data is available from the NHSScotland QI Zone website: Collecting data, Analysing data and Presenting data.
Appendix 2d – Showing data and variation over time
Data is often aggregated and presented over long time periods, yet shown as discrete points in time. If there are not enough points in time this can lead to poor interpretation. Presenting data continuously over time can provide a more accurate basis for decision making.
Run charts
Run charts are line graphs where a measure is plotted over time with the median also shown. Changes made to a process are also often marked on the graph so that they can be connected with the impact on the process.
Example of an annotated run chart
Courtesy of Professor Kevin Rooney, University of the West of Scotland and NHS Greater Glasgow and Clyde.
More information on run charts can be found at: http://www.healthcareimprovementscotland.org/previous_resources/implementation_support/guide_to_using_run_charts.aspx
Appendix 2e – Variation
It is important to understand that a certain amount of variation is normal. Some variation is intended, for example, care is designed for specific groups of patients with different needs. Unintended variation in healthcare is due to differences in healthcare processes that are not connected with patients' different needs. This can result in waste and harm, and commonly forms the focus for an improvement.
Statistical Process Control ( SPC) is a way of using statistical methods and visual display of data to allow us to understand the variation in a process. More information on this can be found at: https://improvement.nhs.uk/resources/statistical-process-control-spc/
Appendix 2f – Additional resources on measurement
- https://improvement.nhs.uk/resources/measurement-improvement-overview/
- The Seven Step Measurement Process
- The NHS Education for Scotland QI Zone: Setting measures to assess impact of improvement efforts
The NHSScotland QI Zone has a set of e-learning modules which can be found at:
https://learn.nes.nhs.scot/817/quality-improvement-zone/resources-elearning
The QI Zone has developed a Measurement Tool Decision Assistant which helps assess what the measurement needs are and suggests which measurement tools might be appropriate
Appendix 3 – Barriers to successful quality improvement
In a study of 14 quality improvement programme evaluations (Dixon Woods et al, 2012), 10 key challenges were consistently identified from the programmes:
Appendix 4 – Example improvement agenda for leaders
Agenda
1. Review aim statement and measures.
2. What are the important/unique/challenging aspects of the project that senior leadership should keep in mind?
3. Review significant accomplishments over the past month – focus on results and understanding the data.
4. What was the last test carried out? What will be the next test?
5. Review monthly data reports, and run charts where available.
6. Discuss team experience, identifying specific improvement issues.
7. Identify what problems/sticking points have been encountered in the last month that the team needs help with.
8. Determine the level of confidence that the team will meet its aims within the project timeframe.
9. Identify next steps.
Contact
Email: Sarah.Hildersley@gov.scot
There is a problem
Thanks for your feedback