TEC programme data review and evaluation: options study

2.0 Logic model development and data review findings

This section describes the outputs from stages 1-4. We present the logic models in diagrammatic and narrative form, discuss the evidence base, and make recommendations for future evaluation priorities in each area. A total of 95 documents were submitted to Just Economics for review. Of these 54 contained evaluation data, and these were synthesised (see Appendix 1 for further details of the data classification and review).

The TEC programme has a strong commitment to evaluation and this is reflected in the number of evaluation studies undertaken. As a group, these studies point to a solid evidence base, especially for two of the workstreams; telecare and HMHM. In other areas, the evidence base is still 'emergent'. This is to be expected, however, given that workstreams are at various stages of implementation and many of the outcomes are expected to take place beyond the life of the programme. In addition, whilst there are still some gaps for evidence in Scotland, the programme is building on a large international evidence base in areas such as telecare and HMHM, and these areas were chosen based on the strength of that evidence base, making evaluations of some outcomes by the TEC programme unnecessary. ^[1]

The evidence base also partly reflects the difficulties associated with the measurement and evaluation of technology in health and social care. Technology-based interventions create a unique set of evaluation challenges, which can conflict with the requirements of experimental research designs. Experimental research designs, in the context of rapidly-evolving technological change, may also fail to provide data in a timely fashion (Glasgow et al. 2013). For this reason, our emphasis in the data review is on pragmatic, relevant, and well-designed evaluation studies (see the measurement framework in Section 5 for further discussion of these criteria).

2.1 TEC programme logic model and evidence base

The TEC programme is designed to function as 'more than the sum of its parts'. At the heart of its strategic vision is catalysing the transformation of health and care services such that technology becomes integrated as 'business as usual', thereby allowing more delivery of services at home and in community settings. The logic model set out below aims to capture how the various programme elements seek to build the local and national conditions to deliver on this vision.

2.1.1 Programme logic model

The logic model for the programme is intentionally forward-looking (see Figure 1). It reflects what the programme is aiming to do in the coming years and the programme legacy, if those objectives are achieved. It should also not be seen as static, but rather as an evolving set of activities that respond to new evidence as it emerges and changes in context ( e.g. technological change).

The programme operates at two levels. At a national level, the programme seeks to create the conditions to support adoption of new technologies and scaling up. This includes building the infrastructure, tools and approaches that are required on the ground, and tackling existing technological, social, organisational, and cultural barriers. At a local level, it seeks to grow TEC activities and initiatives through dedicated funding to organisations implementing or trialling TEC initiatives.

There have already been considerable successes for the programme, with an additional 66,000 clients benefiting from technology-enabled care (to February 2018), sharing of learning across settings, and progress on infrastructural issues, such as the analogue to digital switchover. ^[2]

Going forward, the TEC Programme Board has identified the following four areas in its Strategic Priorities for technology enabled care 2018-21 in support of the new Digital Health and Care Strategy for Scotland:

Transforming local systems – supporting exemplars that are seeking to transform local health and social care systems using digital technology to shift local systems upstream to prevention, self-management and greater independent living.

Developing approaches once for Scotland – developing approaches that have been shown to be effective, supporting scaling up across Scotland and addressing barriers that require national level action.

Preparing for the future – identifying and testing new approaches that offer the potential to achieve change at scale.

Building capabilities and supporting improvement – championing, supporting, gathering and promoting the evidence of what works, to develop the culture and skills that recognise and use digital TEC including through developing business cases, supporting strategic planning and delivery.

These strategic priorities aim to increase the number of people using fit-for-purpose technologies and for those technologies to either a) free up capacity within the system without any risk to care quality or b) improve health and social care outcomes (or, indeed, both simultaneously). Ultimately, the goal is to enable citizens to enjoy longer, happier and healthier lives in their own homes, thereby reducing demand on current services and freeing up capacity.

Achieving these strategic priorities occurs via several pathways. These include the use of Attend Anywhere for remote clinical and care consultations and decision support, improved condition control and self-management via remote monitoring and digital platforms, more technology-enabled care options being used in care homes and better access to specialist services, especially for those with long term conditions and those experiencing health inequalities.

The benefits of increased roll-out of technology-enabled care extend beyond users of these services. The health and social care workforce, in the short-term, is expected to feel more supported in their roles and less burdened. Longer-term, we would expect to see improvements in well-being, either through removal of routine tasks, reductions in workloads, or reduced stress from working in a more efficient system.

It is also expected that the programme will positively impact on the well-being of unpaid carers. Technology-enabled care is expected to reduce some routine tasks and the need for certain face-to-face appointments, thereby reducing the workload of some unpaid carers. Further, it is expected that some technology-enabled care options, such as telecare and HMHM, will provide reassurance to carers. Wider social benefits have been identified for interventions at scale. These include improving the viability of vulnerable services and communities, reducing CO2 emissions and economic benefits from the improved productivity of a healthier workforce.

Another objective of the TEC programme objectives is to reduce health inequalities and there are various means by which the technologies can play a role within this. However, for all the technologies, excepting telecare, digital exclusion is a potential barrier to achieving this. That is, whilst technologies like video conferencing or HMHM can bring services to people who currently have more limited access, they also require minimum skills and infrastructure to be fully exploited. Given that those who experience health inequalities are also more likely to be digitally excluded ( e.g. older people, disabled people, people in remote locations and those on low incomes), there is a risk that technology-enabled care will exacerbate inequalities. Further, it is important to distinguish between inequality of access and inequality of outcome. Although clearly correlated, better access to services is not the sole determinant of greater equality of outcome and both should be priorities for measurement.

A signal that technologies are being exploited to their full potential is that technology-enabled care options become the default within decision-making in health and care teams. This is a key objective for the programme and underscores the importance of advocacy work to increase uptake. This is vital as some of the main systemic benefits of technology-enabled care will only be realised at scale. For instance, a motivation for incorporating technology into health and social care is to reduce the demand for mainstream services by enabling people to self-care, be cared for remotely, or to improve their lifestyles and behaviours such that they require fewer health or care interventions. From the perspective of health and social care services, it is possible to place a monetary value on outcomes, such as the reduction in hospital admissions, reduction in care home admissions and the logic model includes a longer-term outcome around reducing the per capita costs of services for those with long-term care needs. However, all the technologies that form part of the programme are interconnected, and exhibit strong network effects, individually and as a group, meaning that maximum value is only achieved at scale.

There are three ways in which this happens. First, there are the network benefits to users of the technologies. A network effect is an economic concept, which describes the positive effect that each additional user or good or service has on the value of that good or service to others. The internet is a good example where a greater number of users increases the value to other users. Even with a small number of users the internet can still be valuable, but its value increases the more users come online. In an analogous way, whilst technology-enabled care users can personally benefit, they benefit more when (for example) all health and care staff can be contacted via a digital portal or VC.

Second, economic benefits and savings to the health and social care system will not be fully realised until a critical mass of people are using the technologies. For example, if an intervention like telecare is shown to prevent hospitalisation, the marginal benefit of each additional bed day saved will be quite small as the hospital and individual wards will still need to operate at full capacity. As the number of users increases and the pressure on hospitals reduces, the marginal value of the bed days saved increases. However, even then, given the demand for health and social care services, the result is likely to be a redeployment of care service to an area of unmet need rather than the realisation of a 'cashable' saving.

Third, integration of technologies with each other has the potential to increase value to users and the government. For example, in time, a user may be able to upload their home health monitoring data to the digital portal where it can be accessed by their GP, who may schedule a video conference call to discuss results and where lifestyle information relevant to those results will be available. We would, therefore, expect the social value of the technologies to increase over time with greater integration. Identifying the points at which a critical mass is reached is challenging and outside of the scope of this study. However, some forecasted economic studies reviewed as part of this evaluation have attempted to do this and are discussed in section 3. These have focused on technologies, such as telecare, that are further into the implementation life-cycle.

The logic model diagram for the programme is set out below (see Figure 1). Given the forward-looking nature of this logic model, we have not included an evidence matrix for the programme. Progress on achieving the overall programme objectives is covered in the discussion around implementation in Section 4. A set of programme-specific recommendations will be provided in Section 5.

Figure 1: Overarching logic model

2.3 Telecare

Telecare refers to continuous, automatic and remote monitoring of users by means of sensors to enable them to continue living in their own home by minimising risks such as falls, gas leaks or flooding. A key element of the programme in Scotland is an effective triage system, which ensures that emergencies are prioritised. Telecare is one of the most widely used technology-enabled care devices in Scotland. According to the most recent data, there are nearly 130,000 local authority-provided telecare systems in operation. ^[3] It is estimated that a further 50,000 are in receipt of Telecare from housing associations and social care providers. Among those aged 75 and older, 20% are using a telecare device or service.

2.3.1 Telecare logic model

The logic model (see Figure 2) set outs several ways in which the implementation of telecare can both improve outcomes for clients, families, carers and health and social care professionals, and be cost-saving.

Firstly, insofar as the telecare service operates an effective triage system to ensure that resources are well-targeted, care and support services should operate in a more timely and cost-effective manner. To this end, the workstream has sought to integrate the telecare response into the wider responder network. Better targeting of resources ensures peace of mind that emergencies will be dealt with appropriately and that telecare users are safe in their own home with fewer falls, fires and the reduced risk of complications that arise from them. This, in turn, is expected to enable people to remain in their homes for longer, leading to a reduction in hospital/care home admissions, fewer incidents of delayed discharge and reduced need for costly overnight/sleepover/wakened care. As discussed above, avoiding hospital may be very beneficial to an individual telecare user, whereas the value to society is likely to be low initially but increasing over time as usage widens and telecare becomes integrated with other technologies.

The imperative of preparing for the analogue to digital transition was identified early in the programme and this has been a major focus of the telecare workstream. The aim has been to ensure that telecare clients receive a seamless service throughout the switchover period and that opportunities for integration of telehealth with telecare data and technology is facilitated. This aspect of the work has been monitored in terms of the rate at which people transition but not subject to an evaluation at this stage, so is not included in this synthesis.

A final benefit set out in the telecare logic model, which is shared with the other workstreams, is to increase the choice available to clients and to deliver more person-centred and responsive care and support services. All these outcomes should combine to improve the quality of life and well-being of services users, especially those with conditions, such as dementia, frailty and/or those experiencing health inequalities.

A key measure of the success of the telecare workstream is the extent to which it becomes a mainstream option within community provision. This is not just about the number of users but also about the attitude of health and care teams. It is expected that, as telecare moves to the mainstream, telecare packages become the first point of referral at discharge, or for those in the community in need of care and support. The production of robust evidence and data is central to this. The programme seeks to understand the data landscape better and progress the opportunities that telecare services can bring to clients and their carers. As a result, there has been an emphasis on demonstrating the economic and quantitative benefits of telecare to overcome barriers regarding trust in its use. The workstream has also sought to exploit the opportunities provided by Big Data, for example using demographic data to identify new potential users.

Figure 2: Telecare logic model

2.3.2 Evidence base for telecare

There was a relatively strong evidence base for telecare, with a total of eight evaluation studies with quantitative data and three with qualitative data (see Appendix 2 for a summary of evidence sources for each workstream). This reflects the emphasis on producing evidence to support scaling up. A comparison of the data and outcomes identified in the telecare logic model is available in Appendix 3. ^[4]

Across all the studies, a range of outcomes were evidenced, including:

• enhanced dignity, independence and quality of life for clients
• increased confidence of vulnerable clients to be more active
• increased health and well-being of carers
• reduced unplanned hospital admissions and prevention or delay of admission to care homes.

There is also evidence that telecare has specific benefits for people with dementia, providing that it is introduced early enough in the development of the condition.

Both the qualitative and quantitative data highlighted wide variation in the provision of telecare across Scotland. There are a range of providers, often using different equipment. This presents challenges in moving to a universal telecare approach. It also makes it more difficult to generalise the findings of research which focus on a specific telecare solution, provider, equipment and so on.

A wide discrepancy in charges for the telecare services was also reported in both the quantitative and qualitative data. The qualitative studies repeatedly identified charging and the way this is implemented ( e.g. framing of the invoice letter; monthly vs. quarterly invoicing) as a potential factor influencing dropout rates. The recent report from Deloittes (2016) recommends that the impact of charging requires further investigation. Although a policy, rather than programme issue, it is a barrier that still needs addressing. Future studies could also seek to identify the causes of user drop out (other than going into a care home or mortality). Of further benefit would be research on ways to harmonise telecare provision to reduce service variation. This could look at examples of where local authorities have merged operations, including challenges they encountered in standardising their approaches and technologies.

2.4 Video Conferencing

The initial focus of the video conferencing workstream was on expanding traditional video conferencing beyond the NHS, across public services and into community settings. This had some initial success but was delayed for technical reasons. In 2017, the integration of video conferencing into health and social care has been driven primarily through the Attend Anywhere system. Alongside this, work continued supporting those areas with a more traditional technical set up. Attend Anywhere is a bespoke platform that allows health and care staff to offer video calls as part of their day-to-day operations. There has been considerable success with rollout. The system is functional in 13 out of 14 Health Boards and being used by 8 third sector organisations and in care homes

2.4.1 Video Conferencing Logic model

The logic model for video conferencing (see Figure 3) is derived from the development of general remote consultations/access, rather than Attend Anywhere specifically. Implementation is still in the early stages, and programme and delivery staff are working to improve the reliability and functionality of the infrastructure upon which it depends to support a greater number of connections between organisations and systems. There are two main uses of video conferencing for health and social care: firstly, for consultations directly with citizens and, secondly, for providing specialist input/support to health and care staff.

The primary beneficiaries of this technology are health and social care clients and their carers. Video conferencing could lead to shorter waiting times for appointments and better access to specialist services, thereby improving client outcomes and well-being. Time may be freed up for carers by reducing the need for them to bring clients to appointments. There are also circumstances where video conferencing may be preferable to a face-to-face consultation. This is the case for so-called 'hard to reach' clients. These include prisoners and people with anxiety disorders, frailty or disability, which impedes them from leaving the house, or people living in remote or rural locations who may be discouraged from using services due to the cost and inconvenience of travel. These groups may be experiencing service inequalities, which video conferencing has the potential to improve. For remote communities, video conferencing could also improve the viability of services that are at risk of closure by enabling them to access specialist input and extend their service offer. This, in turn, could impact the sustainability of those communities in the longer-term.

As with other areas, a substantial part of the workstream activities relate to the promotion of the technology through advocacy and evaluation, alongside tackling barriers to its adoption. The aim is to integrate video conferencing into health and social care as well as wider public services, where VC meetings could improve communication and collaboration between professionals and reduce the need for travel. Longer-term, these technologies could impact on the workload and work/life balance of health and care professionals, thereby improving recruitment, retention and well-being.

The programme has recognised the benefits of aligning approaches within HMHM to support video conferencing consultations. As discussed, earlier video conferencing along with along with other technologies exhibits strong network effects. This has implications for cost savings and for CO2 emissions, which likewise suffer from network effects and only become material at scale.

Figure 3: Video conferencing logic model

2.4.2 Evidence base for video conferencing

There were five reports containing quantitative information and four with qualitative information on VC. These reports reflect the historical focus of the workstream and are broader therefore than the Attend Anywhere system, which is still in the early stages of implementation.

Several of the quantitative studies reported solely on usage. Two of the reports were robust outcomes evaluations examining the use of video conferencing to provide specialist input to remote areas. Outcomes demonstrated included:

• Better pharmaceutical management
• Improved access to specialist services
• Reduced hospital admissions and length of stay
• Cost, time and travel savings for staff and clients
• Staff having greater confidence in the care they are providing
• Family and clients having greater confidence in the care they are receiving, although this is based on very small samples and/or staff reports

The technology – in these studies not the Attend Anywhere system – was described as highly reliable and both settings had been provided with the necessary equipment, infrastructure and training to effectively use video conferencing. For some staff, regular video conferences with the remote specialist increased workload ( e.g. preparing files for the meetings) but this did not lead to resistance as staff could see the substantial benefits that were derived from these meetings.

A third evaluation study looked at the use of video conferencing technology by Edinburgh GP practices. In this setting, the response from staff to video conferencing was more mixed. While generally regarded as preferable to telephone consultations, there were concerns about the adequacy of the technology, impact on workload and the purpose for which video consultations could be used ( e.g. sufficiency of diagnostic information compared to face-to-face consultations).

Evaluations to date have primarily focused on using video conferencing to gain specialist input and not on the routine use of video conferencing for consultations with clients or for wider relevance on remote access in public services. However, these are important potential uses of video conferencing and should be considered in the future. Although staff and users are asked to complete a survey after an Attend Anywhere consultation, these have not been reviewed as part of this research. The evidence base in this area could also be enhanced by collating and analysing the Attend Anywhere surveys and by undertaking more in-depth studies of how family members and clients experience video conferencing including outcomes of interventions etc. Finally, the impact on services/professionals is a further area that merits additional research.

The evidence base, mapped against the video conferencing logic model, is set out in Appendix 3. This mapping, as noted above, reflects video conferencing use more generally rather than use of the Attend Anywhere system specifically.

2.5 Digital Platforms ( DP)

This workstream is concerned with supporting the development of an integrated digital platform that enables people to access their health and care records, engage with the health service and manage their own healthcare. This is an ambitious goal that is expected to eventually lead to an independent programme of work. The role of the TEC programme has been to support proof of concept on development work for the National Health and Social Care portal, including commissioning a business case and testing components of the portal. The TEC Programme also supported two key citizen-facing programmes, Living it UP and ALISS (A Local Information System for Scotland). Living it UP was a significant aspect of the Digital Platform work in the first three years, along with a focus on developing the core platform.

One of the key objectives of the National Health and Social Care platform, once operational, is to facilitate integration with other current and future tools and technologies to create seamless interoperability between technologies. This is a key area of development for the new Digital Health and Care Strategy and it is expected that such a digital platform will provide:

• A portal where people can access their own records, manage their own healthcare and seek advice/support about their condition
• A safe and secure means for people to access reputable online services
• An access point for people to book and attend video conferencing consultations (using Attend Anywhere),
• An opportunity for people using HMHM to upload their monitoring or telecare data for oversight by a clinician, carer or support service

The development of such a portal is in the early stages and to date only parts of the system have been developed and/or tested.

The logic model in Figure 4 relates only to the health and social care portal ( i.e. the first bullet above). Participants at the logic model workshop for this workstream decided to focus on this aspect as the benefits of the remaining elements are already covered to some extent by the other workstream logic models and the additive benefits of integration into the digital platform are difficult to ascertain at this stage of development. However, it would be beneficial to undertake some further work exploring the likely benefits of the fully functional platform.

For the health and social care portal, the system would allow people to view their records, scheduled appointments, prescriptions and test results as well as be given advice and support relating to their risk factors. There are several potential benefits of such a system. First, it supports objectives around achieving person-centred care and greater co-production of healthcare. This should improve people's knowledge and understanding of their own health conditions and treatment. In turn, this would be expected to lead to greater adherence to treatment programmes and motivate people to adopt positive lifestyle/behaviour changes that impact on health and well-being. This should also improve satisfaction with the system. Client oversight of their own healthcare should reduce the number of clinical and medication errors, which should improve user safety. This should also reduce the litigation liabilities of the NHS (which are currently substantial). Better management of health and health conditions through the portal should also mean fewer working days lost and higher productivity.

The system should create time savings for both the client and the professional by reducing the time taken up with conveying information, results and so on. This should lead to greater efficiency within the health service. The requirement to reach a critical mass of users has already been discussed. Whilst benefits to individual users may exist at low uptake, as the number of users increases so does the value to those users and wider society.

Finally, where clients access their own information, or a service, through the portal, this may reduce the need for carers to carry out these activities, thereby reducing the care burden and potentially positively impacting on the well-being of carers.

Figure 4: Health and social care portal logic model

2.5.1 Evidence base for Digital Platforms

Given that this workstream is largely concerned with supporting a technology that is yet to be developed, it understandably does not have an existing evidence base. For this reason, we have also not provided the usual matrix mapping the evidence base against the logic model.

However, evaluations of several other digital portals, carried out as part of the TEC programme, provide some insight into the benefits that a National Health and Social Care platform might provide. The evaluation of the Living It Up platform, which has now been integrated with NHS Inform, provides the most comprehensive assessment of the outcomes that can potentially be derived from the use of digital platforms for those managing chronic conditions and for their carers. The study used a controlled prospective design to undertake an SROI evaluation. Benefits identified in the study included:

• Greater sense of control and self-management
• Reduced social isolation due to making local connections ( e.g. through volunteering, finding local activities or resources)
• Finding out information about their condition (which they had not received from health professionals) and how to manage it

The data review for this workstream also included two process evaluations of ALISS and a journal article examining readiness for digital health platforms at macro, micro and meso-levels (Lennon et al. 2017). The process evaluations set out the steps taken to develop ALISS but do not contain outcomes information. The journal article draws on interviews with key implementers (n=125) and a small number of users (n=7) to highlight several issues related to rolling out digital platforms, including the potential sense of disempowerment felt by health staff, inequalities of access because of different levels of digital/ IT skills, and fears around data safety, especially where platforms are run by non- NHS bodies.

Unlike some of the more developed workstreams, evaluations to date have mostly focused on technical or process issues. In the future, Digital Platforms would benefit from a greater focus on outcomes measurement of systems such as ALISS.

2.6 Home and Mobile Health Monitoring ( HMHM)

HMHM is the use of digital remote monitoring technology to enable patients outside of hospital to receive, record and relay clinically relevant information about their health and wellbeing. In March 2015, prior to the start of the TEC programme, there were 485 users of HMHM. This had increased to 10,780 by December 2017, with 12 Health and Social Care Partnerships now undertaking work in this area.

2.6.1 HMHM Logic model

Figure 5 shows that most of the activities under this workstream relate to supporting the take-up of HMHM through service redesign, awareness-raising and the production of evidence reports. The benefits of self-management are not just related to condition control and adherence to treatment but should improve the satisfaction and well-being of people living with long-term conditions. This outcome is expected to improve the likelihood that people can live independently for longer and have a higher quality of life whilst doing so and reduce reliance on traditional health services including admission to hospital. Although it is not clear that this will reduce the burden on carers, it is expected that carers will feel more supported in their caring role and that this will enhance their quality of life. It may also reduce the need for carers to attend some appointments, thereby providing respite.

From the perspective of the health and care system, network effects apply as with the other technologies. This means that, although capacity can be freed up within the system, the marginal savings are likely to be low in the early stages of implementation. Over time, however, these could translate into a more cost-effective health system.

Although the benefits to people with long-term conditions is a major driver of HMHM roll out, HMHM can also be used with other clients and services. For example, there are positive examples of its use to support remote decision-making within midwifery services, post-surgical services, and rehabilitation services. These examples suggest that HMHM could potentially be relevant to any user of the health service.

One of the TEC programme objectives is to reduce health inequalities and HMHM can play a role within this. However, as with other areas such as video conferencing, existing digital exclusion may act as a barrier to achieving this.

For clinicians, the expectation is that HMHM will help to support decision-making, thereby improving job satisfaction and a sense of efficacy in their roles. There is also potential for HMHM to reduce the amount of routine activities within clinical roles once economies of scale have been achieved.

Figure 5: HMHM logic model

2.6.2 Evidence base for HMHM

This workstream had the largest number of reports of any workstream, reflecting in part the extent to which the technologies underpinning HMHM are becoming embedded in clinical practice (compared to video conferencing and digital platforms) and the specific focus of this workstream on developing an evidence base.

The reports provide a solid basis for identifying some of the key outcomes, as well as challenges around the use of HMHM (see Section 4 on implementation). The main outcome for patients centres around a feeling of reassurance from the increased connectedness that HMHM offers. Having a clinician in the 'loop' was vital to achieving this outcome. There was both quantitative and qualitative evidence for improved health through a combination of increased motivation to self-manage their condition and more responsive care ( e.g. titration of medicines in response to readings), although the findings are less impressive for COPD than some other long-term conditions. These have been confirmed by a recent NHS England summary of HMHM impacts (Wolters, forthcoming). As well as better self-management and condition control, the report found some evidence for reduced hospital admissions, optimised face-to-face contact and access to services. This was also the finding from a recent study on the scale up of blood pressure monitoring (McKinstry et al. forthcoming). Although initial take-up of the service was from affluent practices, this changed over time, suggesting it could play a role in reducing health service inequality. The study also found a reduction in the number of face-to-face appointments over the course of the study.

Challenges centred mainly around the use of the technology ( e.g. connectivity, having necessary tech skills) and overcoming the initial trepidation of staff and patients around its use ( e.g. staff fears around impact on workload; patient fears around the ability to use the device/technology). There was some evidence of reduced health visits and hospital admissions. In the short-term, however, for some patients the introduction of HMHM increased demand for services by uncovering new health and/or care needs.

The literature also points to some implementation challenges for HMHM, though these require further exploration. The first of these relates to the possible negative effects of patients being able to readily access information about their condition in the form of graphs and trends that they do not fully understand. However, this can be addressed with the right support for users. A second negative outcome relates to the increased demand on staff in the early implementation stages because of switching to a new system and embedding this in usual work practices. This is a common problem with technology deployment and will be discussed later as part of the discussion on implementation.

This evidence for this workstream is the most well-developed but it could benefit in the future from further research on implementation as HMHM is applied to additional conditions, and for multiple conditions. A further challenge is evidencing prevention. This is an important part of the evidence base for HMHM, particularly for people with long-term conditions. The contribution HMHM directly makes to self-management, especially for different populations and interventions, and the type of service models that should be adopted to optimise this at scale are further areas for research.

Finally, there is some evidence to suggest that carers are key to ensuring effective use of HMHM by the patient and that they may also experience benefits in the form of reduced anxiety and an increased sense of reassurance. Again, this merits further research. The evidence base, mapped against the HMHM logic model, is set out in Appendix 3.