Annual Report 2014 - Reporting on the Quality and Efficiency Support Team

Part 05: Outpatients, Primary and Community Care

ABOUT THE PROGRAMME

Susan Bishop
t: 0131 244 2019
e: susan.bishop2@scotland.gsi.gov.uk

drivers

There is increasing demand on health and social care services; in order to meet this demand it is necessary for more people to get the right care, from the right person, at the right time, in the right place.

We can create more opportunities for self-management, giving control to individuals over their own conditions. Shifting the balance of care and reducing unwarranted variation and waste allows resources to be better utilised for those most in need and achieve best value.

aims

The overall aim of the Programme is to move care closer to home and enable more people to receive the right care, from the right person, at the right time, in the right place.

This overall aim is supported by the following supplementary aims to:

• support teams, services and networks across outpatient services, primary and community care to:
  • work together to understand and diagnose system issues
  • systematically use quality improvement methodologies to deliver safe, person-centred and effective care
  • make best use of resources including the time of patients, the public and staff
• work in partnership with clinicians, managers, other improvement partners, patients and the public to generate new concepts, design and test new and innovative models of care and implement changes that improve the quality of service delivery
• increase the quality and use of data to help reduce variation, waste and harm

workstreams

• Transforming Outpatient Services - implementation of five change concepts as outlined in the driver diagramn (see page 55)
• Primary Care - designing and testing new models of service delivery to improve the flow of patient pathways through the health and social care system
• Community Care - designing and testing new ways of working to reduce unwarranted variation and waste, and to release time to reinvest in patient care

achievements 2014

Transforming Outpatient Services

• Patient Reminder Services Change Package to support NHS Boards to reduce the number of Did Not Attends (DNAs)
• Pathfinder projects to use advice instead of referral, optimise outpatient clinic resources, test digital technology prototype and use of alerts and video to support self-management
• National Return Outpatients Data Group to increase the quality and use of data for peer review to understand and reduce unwarranted variation and to inform and test improvements

Primary Care

• Design and delivery support for anticipatory care planning within General Practice and development of primary care indicators
• First phase of Managing Patient Flow - Shaping Our Future Practice (testing and extending the use of data and tools to improve patient flow within primary care)

Community Care

• Transition of the use of Releasing Time to Care to business as usual for NHS Boards and continuing development of the facilitator network
• Completion of phase one testing of Releasing Time to Care to support the process of integration of health and social care teams

priorities 2015

Transforming Outpatient Services

• Design new and sustainable person-centred models of outpatient services
• Adoption and spread of five high impact changes: use of advice only, clinical dialogue and referral feedback, centralised and e-triage, improved booking practices and use of reminder services, direct access to diagnostics and musculoskeletal redesign
• Increase the quality and use of data for peer review to understand and reduce unwarranted variation and to inform and test improvements

Primary Care

• Supporting delivery of General Practice access indicators
• Second phase testing of the Managing Patient Flow - Shaping Our Future Practice project

Community Care

• Releasing Time to Care for integrating health and social care teams
• Collaborative design of Telehealth and Telecare Improvement Programme

http://www.qihub.scot.nhs.uk/quality-and-efficiency/outpatient-primary-and-community-care.aspx

Programme Case Study

Patient Reminder Services

Background and context

In 2013-2014 just under 5 million 'consultant led' outpatient appointments were recorded. 1.5 million of these were new outpatients. Traditionally outpatients are classified as patients who are referred to a consultant or other hospital-based specialist for further assessment, advice, diagnosis or treatment. The current 'traditional' model for the provision of outpatient services is that patients will be referred and subsequently offered appointments with hospital based consultants at clinics usually based within hospital premises. Overall demand for traditional 'consultant led' outpatient services is increasing. However, it is increasingly recognised and evidenced that not all outpatients need to be seen in this traditional manner.

The Transformation of Outpatient Services (TOPS) programme of work supports NHS Boards and local Health and Social Care Partnerships to move care closer to home and to enable more people to get the right care, from the right person, at the right time.

TOPS aims to identify suitable alternative options for outpatient services, which improve quality and safety of care and to support clinicians to transform the service.

It supports teams working together with patients and the public to understand and diagnose system issues, to design and innovate and to use continuous improvement to deliver high quality person centred care and best value for money.

Problem

The overall average 'Did Not Attend' (DNA) rate for new outpatient appointments across NHS Boards in 2013-14 was 9.8 per cent. Extrapolated to the 4.6 million total outpatient appointments, this equates to 460,000 lost outpatient appointments across NHSScotland in one year. These wasted appointments and the resulting, irretrievable loss of capacity and specialist resource adversely impacts on patient waiting times. It also equates to an estimated £61 million net total unutilised outpatient appointment cost to NHSScotland, which is potentially lost to the service and cannot be reinvested.

The existing and expanding evidence base (encompassing research and experiential) demonstrates significant advantages to the individual patient, to specialties, and also to the wider NHS organisation in systematically implementing and utilising patient reminder services (PRS).

Aim

The TOPS programme of work aims for all patients to be seen in the right place, at the right time and by the right person.

In order to achieve this overall aim there are five high impact change concepts (see driver diagram below), one of which is to 'Reduce DNAs through using patient reminder services'.

Refer to image below.

The aim of the PRS change concept is to:

• Help patients to keep appointments, meaning that more people will arrive for their planned outpatient appointment
• Help reduce DNAs and release outpatient clinic capacity
• Provide advance notification of cancelled appointments that can be reused by other patients on waiting lists
• Enable recycling of released capacity to reduce overall outpatient waiting times and realise the subsequent health benefits to patients by reducing waiting times
• Reduce 'new' outpatient waiting lists
• Ensure best use of staff and resources through optimising clinic utilisation and where appointments need to be cancelled, the released appointments are made available for other people to use thus more efficient use can be made of available resources
• Provide safe, effective and person-centred appointment booking
• Improve patient flow and capacity management
• Reduce health inequalities - through utilising DNA propensity software, we can identify patients or patient groups most likely to DNA and communicate with them in the most effective way to improve attendance.

Action taken

Extensive evidence and literature review was undertaken (encompassing research and experiential), leading to the identification and development of each of the high impact changes.

Between 2012 and 2014, QuEST commissioned three NHS Boards to use three different approaches to transforming outpatient services, one of which was rapidly testing changes and contributing to the development of change packages that help spread reliable improvements. Through this process PRS (incorporating telephony and digital systems and services) have been tested in a range of NHS Boards, sites and specialties.

During 2013-14, Chief Executives of all NHS Boards agreed to support rapid adoption and spread of five evidence-based, high-impact change concepts including implementation of PRS.

QuEST, in collaboration with NHS Boards and partners developed the PRS change package, as a resource to support NHS Boards to implement and spread the PRS change concepts. The change package is designed as a web-based resource which describes in detail the steps needed and how to plan for, adopt, implement and monitor effectiveness of PRS. It includes direct links to evidence, resources, and examples of good practice, information and contacts for teams to use and also provides information and resources to enable planning for spread.

The PRS change package and readiness assessments were issued to NHS Boards via a Chief Executive Letter (CEL).

Baseline readiness assessments have been completed for all NHS Boards.

Development and production of PRS evaluation matrix, core data set and productivity tool for use by NHS Boards.

Collaborative development of 'Once for Scotland' PRS proof of concept proposal.

Commissioning and development of national PRS procurement framework.

Patient experience

The following patient benefits are anticipated:

• patients will be seen in right place, at right time by the right healthcare professional
• increased health benefits to patients by reducing waiting times
• improved patient experience, fewer delays and increased patient satisfaction
• safe, effective, person centred appointment booking

Staff experience

From a staff perspective the following indicates some of the benefits that can be realised:

• professional satisfaction of seeing patients, at the right time
• opportunity to reduce harm to patients by reducing waits
• release of clinical capacity, enabling optimal use of clinical time
• ability to innovate, design and deliver outpatient services which meet the health needs of the future

Efficiency savings and productive gains

By achieving the overall aim of reducing DNAs to 7 per cent by 2016 (which represents achievement of the 2013 upper quartile) and

taking activity projections into consideration (an increase of about 3 per cent by 2016) around 41,151 new outpatient appointments alone could be released back into the system with a productive opportunity of between £4.3 million (total direct cost) and £5.5 million (net total cost).

Benefits realisation through utilisation of released capacity and consequential cost reduction through a decreased requirement for additional clinics and minimal use of waiting list initiatives.

Release consultant time and improve utilisation of released clinician/clinic capacity for which there is an indicative productivity gain.

Optimise utilisation of clinic resources and maximise opportunity of filling cancelled slots.

Sustainability

The identification of key enablers (local and national) in relation to all aspects of the Programme and the engagement and collaborative working with patients, NHS Boards, and partners to develop easily accessible comprehensive resources for use by NHS Boards will support implementation, spread and sustainability.

Key enablers across the TOPS programme of work include:

• Chief Executives of all NHS Boards agreed to support rapid adoption and spread of the five existing interdependent, evidence based change concepts.
• All fourteen territorial NHS Boards, with their partners, are beginning to take the necessary strategic actions to create the contextual, cultural and leadership conditions to enable staff, practitioners and patients to achieve their 2020 vision.
• Some NHS Boards have already formalised outpatient transformation as a strategic priority and integrated or adopted TOPS high impact changes into their local programme.

Planning for spread and sustainability are integral elements within each of the change packages and as such significant evidence and resources are made available.

• A key point is to truly understand capacity and demand for new and return outpatients.
• Development/commissioning of national PRS procurement framework.
• Development of PRS evaluation matrix, core data set and productivity tool.
• Supported implementation and evaluation of the PRS change package.

Lessons learned

The following key lessons have been learned:

• The development and implementation of the PRS change package focuses on improved quality of service for patients, but clear efficiency savings and productivity gains can be realised.
• Recognise the value and learning from what does not work well, as well as what does.
• Provide non-prescriptive guidance to NHS Boards to enable implementation and sustainability at local level.
• Keep the patient at the centre of every discussion and involve them in implementation.
• Ensure that the right clinicians and staff are involved from the start - this should include medical, nursing, primary care and medical records staff to ensure buy-in, stakeholder engagement, and identification of win-win solutions.
• It is critical to ensure joint working at Scottish Government level to enable connections between initiatives to be made, ensure consistent messages and increase visibility of the Programme.
• Plan for implementation, spread and sustainability from the start.

Next steps

A further evaluation of implementation of the PRS change package will be undertaken.

All NHS Boards implementing PRS will use common criteria and a core data set to evaluate the impact of PRS, which will enable useful qualitative and quantitative cross NHS Board comparisons.

The TOPS Programme team in collaboration with NHS Boards and other partners will review the PRS change package and update to include current research and experiential learning and newly developed resources.

Chief Executives of all NHS Boards will continue to support the rapid adoption and spread of the five existing interdependent, evidence based change concepts.

Transforming Outpatient Services

NHS Board Case Study

NHS Ayrshire and Arran

Co-production for the Design and Delivery of Personal Footcare

Background and context

The demographic changes facing Scotland are well known; the number of people aged over 65 is projected to increase by 22 per cent by 2020 and by 63 per cent by 2035.

Good personal footcare such as toenail cutting helps older people to remain active, well and independent for as long as possible (Managing Falls and Fractures in Care Homes for Older People, Social Care and Social Work Improvement Scotland (SCSWIS) and NHSScotland, 2011).

In the past NHS podiatry services provided personal footcare. Within NHS Ayrshire and Arran the consequence in reshaping the service around need was that many older people were no longer eligible for NHS services yet still required help with their personal footcare to keep well and active (NHSScotland National Personal Footcare Guidelines, 2013).

Problem

Demand on service and capacity to deliver.

Aim

• For NHS and third sector to work with the public to develop and deliver an asset-based personal footcare service.
• To develop a sustainable model that utilises skills existing in the community.
• Through the above to enable NHS podiatry services to be able to meet demand and deliver an enhanced service.

Action taken

Using the experience of the third sector in delivering community projects a range of delivery models was explored. A social enterprise model was supported by the main features which were:

• recruit people with experience in health care and/or volunteering
• recruit a coordinator for admin and team lead role
• pump prime then charge small amount for service (£5)
• provide training and mentorship model
• pay people to deliver footcare to ensure motivation

Results

• First 10 months 1,200 people seen by third sector = 3,200 NHS Podiatry appointments saved
• This equates to two WTE NHS podiatrist time reinvested
• This has led to a reduction in waiting times from 16 weeks to three weeks in podiatry musculoskeletal service
• This has enabled the development of a foot protection programme for people with diabetes
• Third sector see people within two weeks compared to 10 week wait in NHS
• Person-centred - 100 per cent of people booking appointments between 6-9 weeks compared to 'enforced' 12 -14 week return times in NHS
• Social enterprise model with full sustainability within three years

Patient experience

• 99 per cent highly satisfied with the service, due to appointment availability and frequency
• 99 per cent of the public attending are agreeable to paying £5
• 100 per cent found it very easy or easy to make an appointment
• 100 per cent found the staff very helpful
• 100 per cent would definitely recommend the service to a friend
• Service able to be delivered within heart of communities in locations easily accessible
• Five NHS Board areas have been to visit the project; two with their third sector partners with a view to reproducing model in their areas

Staff experience

• 18 volunteers trained age range 21 years - 73 years
• 100 per cent of volunteers rated the training and experience to date as positive
• 12 volunteers delivering personal footcare as interested in contributing to their community, six volunteers are delivering footcare with an aim to gaining full time employment
• Two volunteers have gained full time employment due to this volunteering experience

Efficiency savings and productive gains

Productive gains are:

• First 10 months 1,200 people seen by third sector = 3,200 NHS Podiatry appointments saved
• This equates to two WTE NHS podiatrist time reinvested
• This has led to reduction in waiting times from 16 weeks to three weeks in podiatry musculoskeletal service
• This has enable the development of a foot protection programme for people with diabetes
• Third sector see people within two weeks compared to 10 week wait in NHS

Sustainability

The on-going costs for the third sector personal foot care service are demonstrated by the east project projections which shows sustainability by year three.

Lessons learned

This is an example of a co-produced and sustainable model of care delivery which has had a direct and positive impact on the public and on NHS services.

Lessons:

• How to work in partnership with individuals and communities to deliver and improve services.

NHS Board Case Study

NHS Ayrshire and Arran

Musculoskeletal Service Redesign

Background and context

Within NHS Ayrshire and Arran, musculoskeletal (MSK) services have been redesigned in order to consistently deliver an evidence-based, streamlined and timely service for patients. The MSK service is community-based using a single point of contact to access the services of a core range of allied health professionals (AHPs), with there being no need for re-referral between specialties.

Problem

MSK conditions account for one in four GP consultations in Scotland. Historically in Ayrshire and Arran, patients with MSK conditions were referred into acute services, in part driven by lengthy waits for AHP services. Unsurprisingly more than 40 per cent were discharged from their first orthopaedic appointment with no need for further follow-up, many then added to the waiting list for AHP services. Lack of clarity on the patient pathway resulted in uncoordinated activity, often with expensive duplication and intervention which added no meaningful value to management.

Aim

To improve access to MSK services with patients being seen by the most appropriate clinician at the right time closer to their home with care delivered by an integrated team of AHP practitioners.

Action taken

A series of redesign events, engaging all stakeholders and utilising lean methodologies informed the MSK pathways of spinal, shoulder, elbow wrist and hand, hip, knee, and foot and ankle. Pathways inform and prompt best management from initial presentation, need for referral, triage, appointing, management and onward referral as indicated. Patients can self-refer via the national Musculoskeletal Advice and Triage Service (MATS) hosted by NHS 24. Reconfiguration of electronic referral process has enabled a single point of access, a single waiting list and efficient electronic triage for a core team of AHPs including physiotherapists, occupational therapists, podiatrists and orthotists. Implementation of TrakCare patient management system also supports clinical outcome recording and evaluation. Advanced practitioners who previously worked in acute services now deliver clinics across all primary care sites, a successful shift in the balance of care. A robust process of clinical supervision and escalation ensures patients can access specialist opinion, further investigation, including diagnostic tests, and referral on to acute services as and if required, with there being no need for referral back to GPs. This avoids over-medicalisation, reduces risk of chronicity and promotes a culture of self-management and enablement. On-going developments will improve management options for patients, for example, through engagement with third sector and voluntary organisations. The identified need for alternative management options has driven improvements in pain pathways enabling patients with persistent pain to access specialist management encompassing a multidisciplinary approach and pain management programme.

Evaluation includes business measures, e.g., orthopaedic waiting time, quality measure, EQ5D5L, person-centred, including staff and patient surveys, patient stories and focus groups. Safety evaluation has included clinical presentation mapping as well as review of any adverse incidents and complaints. Continuous improvement is supported by regular programme board meetings, incident log and route cause analysis.

Results

Implementation of the low back pain pathway alone resulted in significant improvement in orthopaedic outpatient capacity. Low back pain previously accounted for 40 per cent of advanced practitioner capacity and now accounts for less than 10 per cent. Over the first six months this accounted for 945 saved orthopaedic appointments. The success of the low back pain pathway provided confidence for stakeholders to support the roll-out of all other pathways.

"In general, MSK pathway is better than hospital consultants and GPs at delivering well thought out cases for surgery"

(Consultant Orthopaedic Spinal Surgeon)

The clinical pathways have demonstrated a significant impact on orthopaedic waiting times. The figure below clearly demonstrates the impact on the orthopaedic waiting list. Since implementation in October 2012, a 20 per cent reduction in total orthopaedic demand has been sustained.

Significant improvements in surgical conversion rates have been reported with pre MSK rates of 25-35 per cent and post redesign 74-80 per cent.

Patient experience

Patient focus groups and stories have been integral to the redesign from the outset. Patients have reported improvements in ability to access services, with no need for repeat visits to GPs and patients felt more informed and involved in decision making. Patient story recordings are available.

Staff experience

Following the implementation of the low back pain pathway a staff survey and focus groups helped inform the on-going pathway and service developments.

In May 2014 further staff focus groups were conducted.

Efficiency savings and productive gains

The redesign was delivered cost neutral but by continuing to save approximately 150 Orthopaedic clinic appointments every month an estimated minimum financial saving of £325,000 per annum has been achieved. However consultant time was able to be effectively reinvested in theatre time to facilitate improvements in Treatment Time Guarantee performance.

Sustainability

On-going work will develop improvements in management options for patients to deliver MSK management that fulfils the changing needs of patients and our communities. Test of change initiative identified that 27 per cent of the MSK patient population have a low risk of on-going symptoms and best care is supported by self-management. With less than 4 per cent of this patient cohort opting back in for treatment, significant improvements in capacity have been achieved. This along with other co-production initiatives should ensure the long term sustainability of the redesign.

Lessons learned

The MSK redesign has come about with a radical change in working practice. The process has been supported by the programme board which continues to meet regularly. Critical to success has been the engagement of stakeholders which has been achieved in a number of ways. A clear communications strategy has supported the workstream, with regular activity reports and updates, including face to face contacts, provided for all stakeholders. Our greatest challenges were around the implementation of patient management system and without dedicated commitment from eHealth team this would not have been achievable.

NHS Board Case Study

NHS Lanarkshire

Anticipatory Care Planning in NHS Lanarkshire

Background and context

The essence of Anticipatory Care Planning (ACP) is to help people with long term conditions to have the confidence, control and choice that comes with knowing what might happen, spotting small indications of change and being ready to do the right things with the right support from the right people. It should be tailored to the stage of the patient's condition and as such exemplify person-centred and holistic care, and respect for the individual's goals, wishes and choices (NHSScotland 2013). It's also about collectively managing risk by working with individuals to help them adopt a 'thinking ahead' approach to have greater control in the event of a flare up of their condition or carer crises.

Within the context of palliative care where the person's condition is expected to deteriorate, the term anticipatory care fits under the umbrella of advance care planning. The aim of advance care planning is to develop better communication and recording of decisions, thereby leading to provision of care based on the needs and preferences of patients and carers.

Problem

The results of an NHS Lanarkshire review of Health and Social Care pathways for the over 65 year olds (June 2013) showed that there was little evidence of anticipatory/advanced care planning and that had this been in place as an alternative, a number of hospital admissions could have been avoided.

An evaluation of Phase 1 of the ACP project in Lanarkshire also showed:

• A lack of relevant knowledge and skills by health care staff, patients and carers to effectively undertake self-management (red flags/alerts) and condition management (GPs) steps to avoid unnecessary acute admission
• Unnecessary admissions which could have been avoided if symptoms or changes in health condition or patient needs had been met earlier thereby de-escalating any deterioration at an early stage
• The above factors causing undue demand and therefore increased workload on acute services and staff with subsequent impact on efficiency and costs and negative impact on patient and carer experiences
• Evidence from patients and carers of a negative patient and/or carer experience following a change or deterioration in health, with dissatisfaction being expressed with regard to lack of dignity and respect shown to individuals and ignorance of preferred wishes and choice

Aim

To firmly embed ACP into practice using improvement methodology approach.

ACP is a collaborative, person-centred self-management approach to facilitating control and choice in personal health. In Lanarkshire, we introduced ACP to improve patients' experiences by respecting individuals' preferred wishes and goals for their care, reducing avoidable admissions and reducing length of hospital stay.

To establish project key performance indicators (KPIs) and track patients in, through and out of hospital, using care homes in the Monklands Hospital catchment area as a focus for the initial work.

Action taken

• The development and introduction of an ACP, guidance notes on the use of ACP for health and social care professionals and an information leaflet for families
• The introduction of ACP throughout the 84 care homes in Lanarkshire, within community nursing, within GP practice, our acute hospitals, carer networks and patient advocacy groups
• Introduced training for ACP champions within care homes to ensure sustainability, identification of ACP champions in the acute division and wider carer groups
• Promotion of ACP at a variety of stakeholder events, multi-agency learning and sharing events, and the introduction of ACP within post graduate educational programmes
• Tracking and follow up patients referred to hospital via the ERC (Emergency Response Centre) who either have, or could benefit from an ACP
• DVD developed to demonstrate the interaction between a service user and service provider during the anticipatory care planning process - used for education purposes/wider promotion of ACP process

Results

A significant increase in the number of ACPs being shared by care homes in Lanarkshire

Please refer to graph below.

Increase in sharing and communicating relevant care home ACP information with GP practice from baseline

Improved communication of patients preferred wishes/key information available to health care staff at point of need or at change in care needs.

Voice of the customer and employee feedback:

• "Residents who have completed an ACP with their family, have done so in the knowledge that their wishes are valued and respected" (care home staff)
• "An ACP is very beneficial to patients, carers and those that are involved in their care" (care home manager)
• "Our resident came back from hospital recently and the ACP had been updated by staff on the ward this is an example of good communication with regard to person-centred care" (care home manager)
• "We have engaged with service users and their families in an attempt to promote the advantages of ACPs in a more positive way...We have initiated a programme of ACP training within our home which is proving to be successful." Feedback from ACP questionnaire (care home manager, North Lanarkshire)
• "ACP is recognition of empowerment for all our residents. We need to ensure choices and preferences are recognised" (care home manager, Lanarkshire)
• "Anticipatory Care Planning made me think about my own health and writing down my preferences and wishes gave me a feeling of control. This enabled discussions with my partner with regards to my health needs and the relevant signs and symptoms to look out for which may indicate a change in my condition" (service user)
• "Completing an ACP is like a wakeup call to take ownership of my own health and inform others with regard to my wishes/choices" (service user/patient advocacy)

Efficiency savings and productive gains

The table below shows data from the audit of residents from participating Care Homes (n = 73) in the Monklands Hospital catchment area who avoided hospital admission as a result of an ACP being in place i.e. remained in their preferred place of care. The audit began in January 2014 and is on-going on a monthly basis.

	No. of patients	*Ave LoS (days)	Cost per day (£)	Occupied bed days	Cost avoidance
Jan 14	38	5	350	190	66,500
Feb 14	20	5	350	100	35,000
Mar 14	16	5	350	80	28,000
Apr 14	18	5	350	90	31,500
May 14	30	5	350	150	52,500
Total	122	5	350	610	213,500

*NB: Average Length of Stay - 5 days at £350 per day
(http://www.isdscotland.org/Health-Topics/Finance/Costs)

Sustainability

Whilst there has been very significant progress in embedding ACP within NHS Lanarkshire, it is acknowledged that further work is required to ensure that the necessary infrastructure is in place to sustain the ACP ethos. The necessary elements to ensure that ACP is sustainable have been identified and are being implemented in phase two of the project. ACP champions have been identified and trained in both the care home, community and secondary care sector and they will be a cornerstone for ensuring sustainability in the future.

Lessons learned

• Remember to factor in time - cultural change will not happen overnight
• The goals of staff/patients/carers/stakeholder groups all vary even though the overall objective is shared - adapting to change needs to take cognisance of individual and group priorities
• Resistance to change/barriers - can be overcome once benefits are realised
• Momentum for change builds once benefits become demonstrable - difficult to plan ahead for resource needed to deliver the improvements once late adopters come on board (i.e. difficult to predict pace of change and demand for support can outstrip the initial planned capacity to deliver it)
• Relying on external partners to collect and report performance data to quantify benefits is problematic.