Evaluation of Integrated Resource Framework Test Sites - Research Findings

This report presents a summary of the main findings of the evaluation of the development and implementation of the Integrated Resource Framework (IRF) in four test site areas in Scotland. The study was commissioned by the Scottish Government in March 2010 and undertaken by Fortuno Consulting and Partners. The IRF is a mechanism developed by the Scottish Government and partners to support shifts in the balance of care through integrated mapping information for health and social care and new joint financial mechanisms between Local Authorities and NHS Boards.

Main Findings

• For the first time partners in the IRF test sites attempted to map cost and activity data across health and social care. Analysis of this data started to improve the evidence base on which planning decisions are made.
• The IRF test sites used the mapping data to examine equity, efficiency, variation and quality but encountered difficulties in engaging GPs and hospital clinicians in discussion of the data, and also in linking outcome data into the analysis.
• IRF test sites worked with national support to improve the detail and accuracy of joint cost and activity information for health and social care. NHS hospital data on cost and activity is centrally gathered and well developed. Work remains ongoing to address the data protection and standardisation issues which limit the extent to which social care and community care costs can be accurately presented at detailed patient level.
• The IRF enabled senior managers in NHS Boards and Local Authorities to coordinate joint working and empowered health and social care staff to reflect on how to work together to improve care pathways and care provision for local populations.
• The success of new ways of integrated working was linked to the extent to which relevant stakeholders were: represented in planning structures; incentivised to engage in analysis of data; and empowered to influence planning decisions.
• Within the timescale of the evaluation, the IRF did not provide evidence of integrated work resulting in the release of resources or of significant changes to fixed costs.
• Integration of health and social care requires clarity of purpose and outcomes. The benefits of integration must be agreed and the evidence base for this strengthened. Strong leadership commitment should be matched with an informed empowerment of staff, patients and carers.
• Integration of health and social care requires commitment to an appropriate scale and scope. Careful consideration needs to be given to which services should be integrated and the timescale over which integration should take place.
• Integration of health and social care requires alignment of all available drivers - policy, legislation, structures, information, incentives, and outcomes - to create momentum for change.

What is the IRF?

The Integrated Resource Framework (IRF) has been developed jointly by the Scottish Government, NHS Scotland and Convention of Scottish Local Authorities (COSLA) to enable partners in NHS Scotland and Local Authorities to be clearer about the cost and quality implications of local decision-making about health and social care. The IRF aims to help partnerships to understand more clearly current resource use across health and social care, enabling better local understanding of costs, activity and variation across service planning and provision for different population groups.

Two broad areas of work have been pursued under IRF: (i) Explicit mapping of cost and activity information for health and adult social care to provide a detailed understanding of existing resource profiles for Partnership populations; and (ii) Work by NHS Board and Local Authority partners in test sites to develop protocols that describe agreed and transparent methods to allow this resource to flow between partners.

Applications from NHS Boards and their Local Authority partners to participate as test sites in the IRF resulted in the selection of test sites in four NHS Boards and twelve Local Authorities in Ayrshire and Arran, Highland, Lothian and Tayside. These sites received £400,000 between 2010 and 2012 for mapping work, organisational development, and project management.

Evaluation aims and methods

The overall aims of the evaluation were to: monitor progress in the test sites; assess the impact of the work of the test sites; feed evidence back into the process of change itself; and draw out implications from the findings for other partnerships moving towards financial and resource integration, both within and acrosssx, health and adult social care services in Scotland in the future.

The evaluation took place in three phases between April 2010 and March 2012 and used five main methodologies to gather information: review of IRF documentation in test sites; reviewing processes and discussions at IRF Project Team and Programme Board meetings; interviews with key strategic partners; an email survey of delivery staff at each test site (carried out just after the baseline phase and repeated at the final phase of the evaluation); focus groups with operational (patient and client facing) staff.

Findings
Joint mapping of cost and activity

There has been progress in mapping health and social care data. All test sites mapped cost and activity across health and social care. It was reported that this was the first time that mapping information across the partners had been undertaken and that this was helpful in building a more complete picture of activity patterns and the associated costs. Flexibility to design local approaches to mapping enabled this to best fit the expectations and uses of each test site. However, it also made broader comparison more difficult.

A large proportion of hospital costs can be mapped at an individual level using centrally collected Scottish Morbidity Record (SMR) and prescribing data. However, detailed information on community health and social care costs is not yet standardised or centrally collected, raising questions of credibility and confidence in how data are produced and applied.

Mapping at a patient level has raised significant data protection challenges. In order to address, this Local Authorities, NHS Boards, and GP practices require clarity on the use to which such data will be put, and the steps which will be taken to ensure anonymity and confidentiality. The IRF cost and activity data did not include public health activity, which was reported to have a substantial effect on demand into the system.

The IRF mapping information was used to facilitate more open and transparent discussion of integrated working across health and social care. Test sites attempted to use the mapping to look at unwarranted variation, identify potential service improvements and examine how services could be provided more efficiently. The potential of the mapping data to help define, measure and monitor outcomes was recognised but remains underdeveloped.

Staff involved in Community Health Partnerships (CHPs), locality planning groups, or IRF steering groups, have a responsibility to plan, organise and manage services in an efficient, effective, and equitable way. Consequently, information to support partnership working and integrated service planning was of obvious and immediate value to this group of stakeholders.

Engaging with wider staff groups whose main role was direct delivery of care (GPs, hospital clinicians, social care managers etc.) proved difficult. The immediate demands of caring for the current needs of patients and service users make it more difficult to find time to step back and look at the impact on planning future services. The IRF programme was designed with engagement of GPs and clinicians at its core. However, the experiences reported by the test sites indicated that local partners did not make a clear enough case for affecting the work of these individuals nor how these individuals can affect the outcomes.

While some of the mapping data is recorded in public record, and therefore in the public domain, there is greater potential to interrogate and then share the mapping data with service users, or more generally with the public.

Trialling new models of integration

Progress with trialling new models of integrated working was linked to existing knowledge of the services and of realistic appraisal of the extent of partnership. Test sites reported the importance of balancing the funding level and timescale alongside local drivers and barriers in order to define realistic opportunities for better integrated working. Test sites made more progress when they took a pragmatic approach to integration by building on existing local knowledge and relationships. These sites restricted their scope to a manageable geography, population, number of service partners and ambition of impact. An alternative approach taken in Highland indicated that more ambitious change could be introduced where local leadership was willing to extend the timeframe for implementing change

The test sites did not implement new financial mechanisms within the expected IRF timeframe. However, during the pilot phase, the Highland Partnership took a decision to bypass the process of trialling small scale pilots of change as a means to testing out the potential of integration. Instead a commitment was made to full scale implementation of new financial mechanisms and governance arrangements using a Lead Agency model.

The IRF test sites found it difficult to get meaningful representation or engagement with groups such as GPs or hospital clinicians. There are four key factors which could be used to incentivise greater engagement with delivery staff: patient outcomes; professional standing; financial reward; and workload balance. Where these were present and recognised then engagement in service change was more likely.

The IRF raised partners' awareness of variation in activity and in costs. However, it also brought into sharper focus the difficulties of improving efficiency and addressing unwanted variation when there is no direct financial mechanism to link demand and supply across the health and social care system.

The IRF helped to support the case for shifting the balance of care by continually highlighting the idea of opportunity cost in discussions on the best use of resources. However, there was a lack of evidence that the IRF has influenced transfer of resources, and fixed costs remain a significant challenge.

The commitment at a strategic leadership level from Chief Executives in NHS Boards and in Local Authorities was important where large scale integration was considered. Non-partisan support from elected members was also reported to be valuable and worthy of investment.

The test sites recognised the importance of networks and relationships in creating a natural momentum for integration. When introducing new models of working the test sites encountered a number of barriers and used strong relationships and a shared commitment to create solutions or work around these problems.

The test sites reported the value of work amongst partners to articulate and understand the shared aims and outcomes that gave purpose to integration. It was acknowledged that variation in perceptions of risk could undermine this.

Conclusion

The IRF enabled four NHS Boards and 12 Local Authorities to gain a clearer understanding of health and social care costs and activity. It has identified some of the limitations of current data, but also started to address some of these through national support of local innovation. It encouraged partners to experiment with new ways of jointly planning services and delivering care. Over the past two years, it has supported national policy direction to shift the balance of care and has contributed to the development of a growing culture of more integrated service delivery. Implementation of new financial mechanisms to facilitate resource transfer proved a lengthier undertaking than expected, and the value of new financial models is as yet untested.

The evaluation of the IRF has shown the importance of: jointly defining the purpose of integration; ensuring sufficient scope, scale and time for integrating services; and harnessing all available drivers to encourage strong leadership, engage stakeholders and deliver change. This learning should inform further development of the IRF and may well be relevant in the consideration of emerging policy and legislation about the future integration of health and social care services.

Recommendations

The full report outlines 11 recommendations which focus on: (i) clearly communicating the purpose of and commitment to integrated working; (ii) reviewing planning structures to incentivise engagement, improve representation, and ensure that those planning services have the necessary skills to interpret data and negotiate decisions; (iii) continuing to address the limitations in the data and share good practice on how this can be used; and (iv) examining the definition of fixed costs and exploring ways of releasing/transferring cost savings.