Quality Improvement and Measurement - What Non-Executive Directors need to know

Improvement Framework

The 3-Step Improvement Framework for Scotland's Public Services has been developed to support people in creating the conditions for, and implementing, the improvements that will make a difference across our public services.

The framework sets out six questions (Figure 1) to be asked of every change programme to test whether the right conditions exist to support true improvement.

This guide uses these six questions to illustrate the role that NHS Non-Executive Directors can play in ensuring the right conditions exist to support QI.

Figure 1

1. Aim

'Purpose is the invisible leader.'

Toke Paludan MØller

Understanding an improvement aim

It is clear that for a team or group to work towards an aim, leadership that is inclusive, supports ideas on the ground, and inspires action is required.

Non-Executive Directors can ask how a particular improvement aim was formulated. If it was developed by a group having looked at a problem and who used data to identify and define it, and the problem is understood, then the chances of producing a successful approach to addressing the problem are high. The 'data' might be survey results from patients, or complaints from staff or patients; it does not have to be quantitative. An aim is most easily formulated where there is a clear evidence base to inform the actions that will achieve it; sometimes a judgement is necessary where the evidence base is not so clear. A Non-Executive Director can help question if in fact energy is being invested in making the right changes. We need to be clear we have the right aims that address what those who receive care most need.

It is important to remember that improvement aims are intended to generate unease with the status quo and to support a sense of urgency that a change is necessary. The change usually involves a redesign of a system or process. It does not involve exhorting people to work harder in the same way. An improvement aim is aspirational and not a target which tempts people to 'game' the system or which is used for judgement.

Organisations will have different starting points. Often, a 'best in class' organisation will find it harder to reduce a poor practice, for example, by 50%, than an organisation that is an outlier where the poor practice is more evident.

It is important to understand the importance of having an aim that is locally challenging and causes interest. Examples of the questions that a good aim provokes are:

• How did ward 10 achieve 366 'days between' a patient developing a preventable pressure ulcer?
• How did the GP surgery achieve a reduction of wasted appointments (do not attends) of 45%?

What Non-Executive Directors need to ask about the aim:

• What is the problem this aim seeks to address?
• Is there data that describes the problem?
• What is the vision behind this aim?
• Not all aims have to be quantitative. Is there a clear approach to how those concerned will know the aim is achieved?
• How has the aim been forged? Are unintended consequences likely?
• Do I believe in the aim?
• Does my experience suggest that this is an unrealistic aim? Is it appropriate to challenge this aim - or do I need to think and understand more about this?

2. Correct changes

'Would you tell me, please, which way I ought to go from here?'
'That depends a good deal on where you want to get to.'
'I don't much care where -'
'Then it doesn't matter which way you go.'

Lewis Carroll, Alice in Wonderland

Enthusing, involving and engaging staff

Evidence about successful QI indicates that it is not necessarily the method or approach used that predicts success, but rather it is the way in which the change is introduced (Health Foundation, 2011).

It is vital not to undervalue the importance of involving all relevant staff. Breaking down traditional hierarchies for a multidisciplinary approach is essential to ensure that all perspectives and ideas are considered, and all staff are engaged in all levels of improvement. Without involvement, we cannot expect commitment, ownership or further innovation through testing changes. As a Non-Executive Director, you can ask how staff are being engaged in the improvements reported on to the Board.

Involving patients and co-production

Patients, carers and the wider public are the only people who experience the whole healthcare journey from start to finish. They therefore have a significant role to play, both in designing improvements, and also monitoring whether these changes have delivered the anticipated impact.

Patients and their families frequently define quality differently from clinicians and managers (King's Fund, 2011). What they view as a problem or value within a service may be unexpected. They have a role to play not only in person-centred care improvements but in safety and effectiveness too. Therefore, leaders must ensure patients and their families are able to contribute meaningfully to their organisations' quality improvement programmes.

There is policy intent that this, in principle, becomes 'the way Scotland works'. An example is the second National Dementia Strategy (Scottish Government, 2013d), which is informed by the active participation of people with lived experience of dementia. The plan calls for mutual respect and equality in the relationship between the person living with dementia and their carer by ensuring that care is planned "from their perspective, not the perspective of service managers or clinicians" (Scottish Government, 2013d: 3). http://www.coproductionscotland.org.uk/resources/co-production-in-scotland-a-policy-overview/

What Non-Executive Directors need to ask about making the correct changes:

• Are we improving the right things for maximum impact?
• Is there evidence that there is a problem and something really needs to be improved?
• Does our data about patient and staff experience suggest this would be a good change?
• Do I believe this change is a good use of attention and resource?
• Will fulfilling the aim make a difference to care in this NHS Board area?
• When I walk in the area where this work is taking place and talk to staff, do they know about it?

3. Clear change method

'The measure of intelligence is the ability to change.'

Albert Einstein

When working to improve a situation, it is important to be clear on the change method being used and to consider whether it is appropriate to the issue being addressed. When working to address system level challenges, improvement often requires a combination of approaches. An example is the reduction in levels of smoking by individuals in Scotland. These changes to behaviour were brought about by a series of well-considered and planned methods. Education and training of key people charged with leading the change, the introduction of legislation and by-laws to prevent smoking in public places, the use of marketing and social media approaches to shift or nudge different habits in the population, and the promotion of smoking cessation support were among the many different tactics used to bring about a reduction in smoking.

Within this wider framework of change, specific QI activities can take place. An example is the work led by the SPSP Maternity and Children's programme. This used QI to develop an approach that focused on monitoring carbon monoxide (CO) levels in pregnant women, to improve a mother's awareness of the dangers of smoking to her unborn baby, with appropriate referral to smoking cessation services which support pregnant women.

The approach must be appropriate for the 'problem'. There are many QI techniques or methodologies, and Appendix 1 looks at some of the more common ones used in Scotland.

As healthcare is an extremely complex system, involving many human factors, it is important to consider which methodology will suit the intended improvement and the local context. It is also important to remember that improvement has a 'result' and is not only an activity. An understanding of why a particular improvement was identified and why a particular method was selected suggests fluency with the methodologies. If a Board is to lead by example this is a requirement. Changes may also have unintended consequences and your role as a Non‑Executive Director is to be alert for the impact on the whole system.

Some NHS Boards have adopted a particular approach to improvement and focus on building capacity in this one approach. There is debate in Scotland as to whether fidelity to one methodology or drawing on a variety of methodologies for a variety of problems is more appropriate. There is an argument that it is more straightforward to build an infrastructure around one approach, and train and build capacity in one methodology. There is a counter argument, however, that, with the complexity of health and social care, we need the flexibility to fit one method to the particular context to get the best possible outcomes.

In a large organisation, the narrative draw of one approach may engage staff powerfully and support culture change. Whether one approach or a variety of approaches is adopted, strategic support, integration, intent and planning for QI are important.

What Non-Executive Directors need to ask about a clear change method:

• How was the change method decided? Were other methodologies considered?
• Is the degree of attention to the problem proportionate and risk based?
• If the change happens, will the result matter? (So what?)
• Do I need to know more before I can properly understand this?
• Am I leading by example in introducing and/or supporting improvements at Board level?
• Do I have a preferred change methodology that blinkers me to others?

4. Measurement

Understanding data and measurement

'Every line is the perfect length if you don't measure it.'

Marty Rubin

Non-Executive Board members need to be aware that measurement:

Can be used for different purposes

Measurement for improvement differs from measurement for judgement and measurement for research. In one Board meeting, Non-Executive Board members will often be required to understand data for these three different purposes. It is important to know the different expectations on Board members for each type of measurement.

Data is commonly gathered from services so that managers and quality assurance and scrutiny organisations can judge the performance of those services against agreed quality thresholds and targets. An example would be an inspection report or a measure against a HEAT target which gauges performance. Data is also commonly collected by research projects that seek to develop knowledge of better ways of delivering services. An example would be data on different medicines to manage blood pressure in patients in the community.

Data for improvement cannot usually be used for comparisons between sites and against thresholds, and more commonly involves tracking processes and outcomes for the same site(s) over time. A typical example is a run chart for a ward showing the number of admissions where a care plan is in place within a particular timescale.

Measurement for improvement informs staff learning as the improvement process develops; data is therefore 'good enough.'

Rather than waiting for the results of an audit, an improvement approach might use a small sample of case notes used in one morning to see if an action has been taken, and quickly establish if not, then why not. This intelligence is fed into the picture of trying to make the improvement work later that day. For example, if case notes suggest an action did not take place and it is then identified that a new member of staff who did not know about the action was on the rota, then in future the induction for new staff can include reference to this. Further information on how measurement for improvement is different to traditional measurements used in healthcare is given in Appendix 2a.

At Board level, data for improvement at the local level is usually reported once it has been collected at scale or over a period of time, where there is a cohesive story. A typical point in reporting to the Board is when an improvement has been demonstrated and it then becomes a strategic responsibility to consider how to spread that improvement elsewhere.

Establishes if there is an opportunity for improvement

When we set out on an improvement project we need to understand if we are meeting the needs of those receiving care. Data can be used to understand how well the system is working in meeting these needs and help us focus energy on the right things. For example, staff might be concerned at the delays in movement of patients through an outpatient clinic; quantitative measures of time and qualitative measures through patient satisfaction surveys would confirm this needs improvement.

Helps prioritise improvements

Data can be used to prioritise improvements so that those that will bring the biggest change are made first. For example, the areas where measurement indicated a pattern of patient mortality informed the first priority areas for the Scottish Patient Safety Programme.

Illustrates improvement through a series of measures; outcome, process and balancing measures

Measurement for improvement should include a small set of measures to test the hypothesis that a particular intervention will bring about an improvement from a variety of angles. These are shown in Appendix 2b.

Needs considered interpretation

Board members will be required to understand data that are reported to them, rather than know about how data might be collected, analysed and presented. Further information on how data can be presented is provided in Appendix 2c.

Whilst a series of quarterly reports might be enough to establish a trend for some measures, data for improvement needs to show points plotted over units of time that allow us to interpret if a change introduced at one point has had an impact. This is typically shown in a run chart which can be annotated. An example is shown in Appendix 2c. Often staff in a hospital ward show run charts on their notice boards as a means not only of tracking but also communicating progress with an improvement process.

Non-Executive Board members also need to be aware that variation in a process may cause a 'point in time' data return to fluctuate between 'green' one month and 'red' the next if the variation is around the set target level. It is important to understand the variation over time rather than responding to the one data point, and support the Board to minimise these variations. More information on variation is shown in Appendix 2d.

Should be viewed in the wider context

It is not enough for a Board to simply approve QI data. It is a Board responsibility to see if there are links to other improvements elsewhere in the Board area, ensure the QI work is integrated into a Board-wide intent for QI and an infrastructure to ensure improvements can be sustained. It is important to understand the possible barriers to successful QI, shown in Appendix 3, since a Board member may be in a position to help remove these.

Board members are also in a position to encourage and support improvement by example, by making links through knowledge of what is going on elsewhere, and by supporting wider changes that could bring about parallel improvements. A Board needs to clear on the purpose of and be alert to the impact on staff if asking for measures. If Non-Executive Board members are invited to attend a local QI event, Appendix 4 shows questions to ask.

What Non-Executives Directors need to ask about measurement:

• What is the data for?
• Is it up to date?
• What does it say?
• Do I know how to look at this presentation of data?
• Do I understand it?
• Do I need to have a short session with someone who can interpret this?
• Do I know how this presentation differs from other data presented in Board papers?

5. Capacity and capability

'What we can or cannot do, what we consider possible or impossible, is rarely a function of our true capability. It is more likely a function of our beliefs about who we are.'

Albert Camus

'If I have the belief that I can do it, I shall surely acquire the capacity to do it even if I may not have it at the beginning.'

Mahatma Gandhi

Capability across a group of staff can be improved

Building capacity and capability for improvement within the NHS will bring huge benefits for patients, carers and staff, as well as increased quality and value. QI is most successful if frontline staff are engaged and indeed have been involved in developing the proposed improvement, and if they are then supported by those who have capability in QI methods, approaches, tools and techniques. In NHSScotland we have a series of courses which support NHS staff to develop their capability in improvement skills: http://www.qihub.scot.nhs.uk/education-and-learning.aspx.

We also have the means of identifying how many of these experts we need in a particular area; a workforce tool: http://www.qiwfdevtool.nes.scot.nhs.uk/.

Attending a course, however, does not ensure that someone is capable in QI; learning needs to be experiential, and rooted in real work. That is why most courses in QI require attendees to undertake an improvement project.

Capacity is related to time and ability to spot opportunities

There also needs to be the mental 'headspace' to formulate improvements and follow them through. Recent work on the Building a QI infrastructure programme found a consistent theme in NHS Boards was that middle managers are so busy doing the work that even if they know about QI and are committed to it (are capable of leading QI work), they often don't have the capacity to support QI in their areas. Often they are responsible for much of the measurement for performance on which the Board is judged and it takes skill and application in order for them to flex and encourage improvement: http://www.qihub.scot.nhs.uk/knowledge-centre/quality-improvement-topics/building-a-quality-improvement-infrastructure.aspx.

As a Non-Executive Director you can ask what plans there are:

• for developing staff in improvement skills across the NHS Board area;
• for supporting staff to actually use those skills they have learned, including creating the 'headspace' for proactive improvement work; and
• to retain those staff once they have learned the skills.

A lot of disconnected projects, each with their different measures and reporting mechanisms can be very demanding on staff and the system. QI is most likely to be effective if improvement projects are linked and supported at a system-wide level and all are focused on a few key aims for the whole Board area. This ensures good governance and avoids a 'scattergun' approach. There is a tension, of course, between wanting to encourage staff to think of and develop improvements, and ensuring that this enthusiasm and energy is focused on priority areas for the Board and where it will make most difference.

It is a Non-Executive Director's privilege to have an overview of the system and to support the integration of changes so that there is intent for, and a consistent approach to QI and activity is held together and sustained with an infrastructure. This helps ensure QI becomes part of the culture and is not only activity that is person or situation dependent.

What Non-Executive Directors need to ask about capacity and capability:

• Do the staff in the relevant area have the capacity to take on this improvement?
• Are they capable (ie do they have the skills)?
• What is the Board plan for improvement?
• How can I support the development of capacity and capability in my Board area?
• Does my Board have a philosophy or consistent approach to QI?
• Does my Board have a commitment to building up capability and capacity in QI?

6. Spread plan

'You're not gonna change any of them by talkin' right, they've got to want to learn themselves, and when they don't want to learn there's nothing you can do but keep your mouth shut or talk their language.'

Harper Lee, To Kill a Mockingbird

'All changes, even the most longed for, have their melancholy; for what we leave behind us is a part of ourselves; we must die to one life before we can enter another.'

Anatole France

Quality and sustainable change

Approximately two-thirds of healthcare improvements go on to result in sustainable change that achieves the planned objective (Health Foundation, 2011).

Local improvements, however, need to be acknowledged, celebrated and spread. Spread is 'when best practice is disseminated consistently and reliably across a whole system' and involves the implementation of proven interventions in each applicable care setting.

Understanding barriers to Quality Improvement

Non-Executive Directors need to understand that QI involves change and change is often resisted. An understanding of what may hold back change will help in developing QI in the first place and then supporting the spread of that change. A summary of barriers to change is shown in Appendix 4. Some of these relate directly to the potential of the Board to support or inhibit improvements, most notably the role of leadership and organisational context and culture. Non‑Executive Directors have a powerful role to lead by example.

Understanding barriers to the spread of quality improvement successes

There are barriers to spreading improvements. In healthcare a third of improvements are never spread beyond the unit where the improvement originated, a further third are embedded in their unit and spread to the organisation and the final third are spread beyond the organisation (UK NHS Institute for Innovation and Improvement, 2010). A common mistake in attempting to spread a success is to task those who have first introduced the change with the responsibility for spreading it. It is not their responsibility; it is a strategic responsibility.

Spread is not as simple as identifying an improvement in an area and telling others to go and do the same. A plan for spreading the change that is supported by Board leadership will be required. The plan needs to make sure that the conditions are created so that those in the next area in the spread plan wish to adopt the change. The relevant people need to be identified starting with those who are most likely to adopt the change and the ground prepared. Supporting others to make the change may require a combination of persuasion, marketing and communication skills and change management. On an encouraging note, the history of the NHS is the history of adopting innovations and positive changes. Antibiotics, anaesthetics and the concept of an outpatient department were once innovations. The current challenge is to ensure that innovations and changes that bring an improvement to the way healthcare is delivered are adopted.

How can leaders ensure that change is sustainable?

Sustainability is 'when new ways of working and improved outcomes become the norm'. There is evidence that sustainable change is more likely in certain contexts (Health Foundation, 2014). A model that involves patients and staff in co-developing, co-designing and implementing changes is more likely to secure a change than one driven by a 'command and control' or hierarchy model where a change is enforced.

A successful and sustainable spread of change requires understanding of the organisational culture, and knowledge of different units, areas and staff groups. The plan for spread must align with the vision and values of the organisation to ensure the work to spread the improvement is undertaken with conviction in each area. The role of senior leadership is to support alignment of improvements and provide the overview of the system to ensure that improvements are reliably implemented, spread and sustained.

The QI Hub has developed a resource to increase understanding of the 10 key factors underpinning successful spread and sustainability of quality improvement in NHSScotland. The Spread and Sustainability of Quality Improvement in Healthcare is available on the QI Hub website.

What Non-Executive Directors need to ask about the Spread Plan:

• How can we celebrate this success?
• Do we know where else this improvement is needed?
• What do we need to do as a Board to encourage the spread of this improvement to these areas?
• Is there anything the Board needs to do to unblock a barrier to spread? Or, what can we learn from this failure?
• Am I generous in acknowledging success?
• How do I support the Chief Executive and other leaders?
• How do I approach the failure of an improvement?
• How do I go about challenging or criticising? Am I supportive or do I only discourage?