Evaluation of Integrated Resource Framework Test Sites

4 USING THE MAPPING INFORMATION

4.1 The previous chapter described progress made by the IRF test sites in developing aggregated cost and activity data for health and social care. It also outlined the challenges which the mapping process has encountered and the steps being taken nationally and locally to overcome these. The IRF programme was designed to enable partners to use the mapping data to inform the planning and provision of health and social care services. This chapter reviews the range of uses, and the variety of users, of IRF mapping data.

Uses of the mapping data

Openness and transparency

4.2 The process of working together to collate, test and refine this data was reported to have given NHS and Local Authority partners a better understanding of their own information as well as that of their partners. It was also reported that the process of producing the mapping information enabled partners to recognise the value of greater openness and transparency in sharing data.

"The IRF has facilitated a greater focus on public sector spending on individuals. While the council had knowledge of commitments to each individual, having a wider focus including NHS investment has been helpful."

Iterative improvement of information

4.3 The credibility of mapping data is improving. The first iteration of the mapping data (based on 2008/09 data) attempted something new and innovative and therefore encountered a number of challenges to its credibility and the extent to which it accurately described cost and activity. This was, in part, due to uncertainty in looking at new information for the first time and also linked to an awareness of the number of contextual factors that could influence the data. By exploring the limits of the data and refining the mapping process used, the subsequent iterations are reported to have provided a better discussion of health and social care cost and activity data.

4.4 The first attempts at mapping focused mainly on delivering an overview of spend and activity across NHS and Local Authority healthcare at a fairly strategic level. Once this had been produced it helped inform partners of the total resource spend for one historical period. It was not used to influence budget setting and did not enable any trend analysis. Test sites that repeated the mapping for 2010/11 reported that this was helpful in improving the quality of the analysis as well as providing a sense of change over time.

4.5 Partners reported that repeating the process on an annual basis would improve analysis of trends and allow a degree of monitoring of high level indicators. (For example the Lothian data was reported as showing a reduction in trauma and orthopaedic expenditure in the over 65s which is attributable to a decline in volume and/or complexity of care, rather than a reduction in prices). However, it was also reported that the value of the mapping data would be much greater if in addition to historical analysis, it was used to positively influence future budget setting and service redesign.

Different levels

4.6 Mapping took place at a number of levels reflecting local priorities and systems in each partnership area. As stated above, it aimed to inform discussion on how resources have been spent and to aid thinking about how they might be spent in the future. Mapping of total resources spent by NHS and Local Authorities on care provision was carried out at the following levels:

• NHS Board area - giving a more accurate assessment of the total resource available and the current level of associated activity. This was reported by test sites as being broadly achievable and of value to high level strategic planning.
• CHP area (usually aligned with Local Authority boundaries) - informing discussion on variation in spend and activity amongst areas (including comparison with NHSScotland Resource Allocation Committee (NRAC) apportionment). This was reported by test sites to be broadly achievable and was useful for informing CHP planning.
• GP practices - opening up opportunities for discussing how GP decision-making varies across a locality. Test sites reported a variety of attempts to create and share this data but encountered two significant challenges at this level. Some data analysis - such as variation in activity and referral patterns - was reported as accurate but its credibility was challenged on its limited ability to reflect local context or patient outcomes. Other data analysis - such as NRAC apportionment - was challenged by partners in the test sites on the basis that comparison became less accurate at this level of population.
• Patient populations based on age or disease groupings - informing discussion on how resources could be spent more efficiently or more effectively for particular clinical/care pathways or patient groupings. The pilots looked at drilling down mapping to this level and reported that it took time to do this for populations of tens or hundreds of patients and service users. It was possible to use this data to inform discussions which included a wider range of qualitative information and local knowledge.
• At a patient level - providing the potential to build up cost and activity information on the basis of locality, age, care setting or disease groupings. This approach was challenging for partnerships to attempt on their own and was initially only fully pursued in a systematic way by NHS Lothian. The Scottish Government and ISD are now working with Tayside on patient-level mapping to build on the progress made by Lothian. The Scottish Government and ISD plan to extend this to other partnerships during the course of 2012.

Different purposes

"IRF data is useful...because it makes you start to ask relevant questions about what it means, and why it happens, and start to think through those questions. But this needs to be facilitated"

4.7 The mapping information was used in a variety of ways. The three main opportunities identified for the mapping data during the IRF programme were: addressing unwanted variation; improving outcomes; and increasing efficiency.

• Addressing unwanted variation. The plethora of ways in which care can be provided, along with the personalised needs of each individual, mean that variation is necessary and beneficial in providing flexible services. However, the IRF acknowledged that certain elements of variation warrant attention and the mapping data was used to examine some of these. For example, mapped costs at a CHP level were compared against an NRAC apportionment to see whether different geographical communities were receiving an equitable share of health and social care resources. Similarly, comparisons of GP referral behaviour, emergency admissions or prescribing could be made using existing NHS data, allowing partners to consider the reasons for variation and possible ways to address these.
• Improving outcomes. The mapping data could be used to help define, measure, and monitor outcomes. Production and discussion of a more complete set of activity data for NHS and Local Authority services enabled analysis of which activity is linked to desirable outcomes (defined by service providers' and service users' perspectives). For example, if place of care is important to patients, the mapping data can be used to monitor this and make a case for redistributing resources to support service improvements. This use of the mapping data was found to be more relevant at the level of the IRF pilots where mapping at the level of a smaller population can be linked more closely with patient/service user outcomes.
• Increasing efficiency. The IRF test sites started to identify ways of delivering health and social care services more efficiently. Two broad approaches to improving efficiency through integrated working were identified as: (i) assume the level of work as fixed and attempt to redesign services to reduce the cost of delivering this work; or (ii) assume the cost base is fixed and attempt to redesign services to increase the amount of work that can be delivered.

4.8 Within each of the purposes set out above, stakeholders reported that there were three possible levels at which the IRF mapping data could be used: a can opener; a dial; or a feedback loop:

• Can opener. This was where data was used to enable partners to start a dialogue on the use of NHS and Local Authority resources. In this situation the level of detail was often less important since its primary role was to bring about general acknowledgement that a particular area of health and social care was important and could benefit from further discussion and analysis. The key function was reported to be the production and sharing of data in a way that started partners talking about a new area or an existing area in a new way.
• Dial. Where an issue, care pathway or population has already been identified as important by partners (can opener), the next stage may require more detailed and precise data to inform discussion on how to address or improve the provision of services in this area. The key function of this data was to identify the particular indicators which would allow analysis of the interplay between related elements of health and social care services and identification of potential alternatives.
• Feedback loop. This use of the data assumes that the issues have been raised (can opener), and the relevant indicators agreed (dial). The specific value of this use of mapping information was to track costs and activity over time in order to allow partners to monitor the effect of any changes to service provision and inform further revisions.

Examples of mapping use in the test sites

4.9 Mapping data was used as a "can opener" by strategic stakeholders to better understand how resources are currently used. Analysis by partners of the initial mapping data was used to define the activities and costs which were worth tracking over time. In this way, the baseline mapping data enabled partners to formulate the questions with which to interrogate further iterations of the mapping. For example Lothian partners' interrogation of three years' of mapping data (2008-11) shows a reduction in admissions of over-85s, and they are now assessing whether this corresponds to an increase in adult social care provision.

4.10 Research and information managers at the City of Edinburgh Council (CEC) interrogated the IRF data set for 2008/09 and used it as a "dial" to identify key indicators which they wish to track over time: acute hospital episodes for patients receiving intermediate care; patterns of emergency hospital admissions of older Edinburgh residents by age group; Accident and Emergency attendance by persons receiving domiciliary care; type of social care service used by persons also receiving NHS care; and use of health and social care services by sub-groups of the older population. Repeating this analysis for subsequent years (as data becomes available) was reported as an important step towards informing strategic planning and the redesign of care pathways. The analysis format developed by CEC is now being used by East and Mid Lothian partners as they learn to interrogate their IRF data sets.

4.11 Similarly, in Tayside, the mapping data for three consecutive years is being used as a "feedback loop" to look at longitudinal trends within CHPs and to compare activity and expenditure between CHPs. Equity of resource use across the three CHPs is being examined for the period 2008/09-2010/11 for selected indicators including: emergency adult admissions; occupied bed days due to long term conditions; and emergency admissions from care homes.

More detailed information on cost and activity

4.12 Detailed mapping data was developed for populations based on specific localities. For example, in NW Perthshire mapping data is being used to help define the scope of services which will be included in their consumption fund (see section 6), with the possibility that this scope will extend beyond health and social care. Highland initially started to pilot a similar approach to creating detailed mapping of cost and activity for district level populations which could then inform future joint service planning. Under the Lead Agency model which is now the focus of integration in Highland, there are plans to develop this approach across nine newly formed districts.

4.13 Mapping data was also developed in more detail for populations based on specific disease grouping and care pathways, such as COPD patients in Ayrshire and Arran. In a similar way, some pilots "drilled down" mapping information on particular age groups such as young people with complex needs in North Ayrshire. These were used to enable partners to discuss potential cost pressures, geographical variation, and possible service improvements.

4.14 Mapping data at the patient/client level is being used to identify typical pathways through health and social care services. For example in Lothian, bespoke software has been used to look at the care pathways of patients/clients whose final destination is a package of residential care.

4.15 Detailed mapping identified potential improvements to services. Pilot projects mapped groups of tens or hundreds of patients and were able to analyse care packages ranging from hundreds of pounds up to those costing over one hundred thousand pounds per year. Analysis of this mapping allowed partners to look at:

• Clustering of patients, in order to compare this with location of resources.
• Location of care activity.
• Variation in costs between independent service providers.
• Appropriateness of existing care services and exploration of viable alternatives.

Users of the mapping data

4.16 The quality and relevance of the information provided was noted as affecting the extent to which stakeholders engage with its findings rather than comment on any perceived shortcomings in its production. With such a large volume of information, generated by different systems and based on different assumptions, there is a potential for data to quickly become complex and confusing. The experience of the IRF test sites indicates that a level of sophistication is required in understanding the audience with whom mapping information is being shared and the purpose for which it is being discussed. The experience of the test sites indicates that repeated presentation of data, in a consistent format targeted at the particular stakeholder grouping, increased familiarity with the process and reduced anxiety about mapping content. Appendix 3 gives a checklist of important considerations for sharing mapping data which have emerged from the evaluation.

Sharing mapping data with service planners

4.17 The mapping produced information that had not previously been available or shared amongst partners involved in health and social care. As noted previously, the process for compiling this information was also new and experimental, leading to a level of uncertainty in the quality of the resulting information. Sites recognised the importance of engaging with a range of stakeholders in order to ensure that the data was accurate enough to allow comparisons to be meaningful.

4.18 The developmental nature of the mapping information left it open to challenge and misinterpretation. As a result, some partners were reticent to discuss this information with wider audiences before all stakeholders had a chance to agree its validity and account for any mitigating factors in large variation. Test sites reported the requirement to balance the need for public accountability and scrutiny of any new data against the need to allow the data to be iteratively developed, so that its validity was well enough accepted to ensure that discussion was not sidetracked by spurious outliers.

"There has been good work around looking at costs down to a GP practice level but we haven't capitalised on this since there is a reluctance to bring this information into the open. There may be good reasons that explain the variation but we need to put this on the table and talk about it."

4.19 Test sites shared data at IRF meetings and were proactive in sharing this through CHP structures. These forums were seen as safe settings in which to analyse and constructively challenge drafts of the data in order to make it more reliable under wider scrutiny.

4.20 The IRF has been most successful in improving dialogue in structures and settings where partnership working was already on the agenda. Test sites reported greater difficulties in engaging with stakeholders who are not already involved in existing partnership structures. Staff involved in CHPs, locality planning groups, or IRF steering groups have a responsibility to plan, organise and manage services in an efficient, effective, and equitable way. As a result, information to support partnership working and integrated service planning was of obvious and immediate value to this group of stakeholders.

Sharing mapping data with service providers

4.21 The IRF recognised the limitations of providing health and social care with finite resources and increased demand. One of the key challenges that the IRF sought to address was how to empower and enable service providers to balance their remit to provide the best possible care for individual patients and service users against their responsibility to consider the care needs of the wider population. However, engaging with staff whose main role was direct delivery of care (GPs, hospital clinicians, social care managers etc.) proved difficult. The immediate demands of caring for the current needs of patients and service users make it more difficult to find time to step back and look at the impact on planning future services.

4.22 From the outset the IRF programme acknowledged that GPs were key stakeholders whose understanding and analysis of the mapping information would be essential to its ongoing usefulness. To this end, the sites used a variety of forums to engage with GPs on the mapping data including taking it to local groups of GPs, organising larger events to which GPs were invited and gaining input from GPs through existing partnership groups. However, sites reported little return for their efforts to engage GPs. The reasons given for this included:

• Professional groups are - to a greater or lesser extent - represented on committee structures. However, neither the extent of current representation, nor the flow of information between representatives and their professions, are clear at present.
• Events were organised which provided focused and protected time to engage with GPs or clinicians. These events were reported to have raised awareness but did not provide an ongoing engagement with the mapping information and were thus of limited value.
• There are significant costs involved in taking frontline staff away from patient contact. Test sites reported difficulties in getting locum time to allow engagement from local GPs.

4.23 The main difficulty encountered in persuading frontline staff to step out of a care-providing role to spend time thinking about service planning and resource realignment, was summarised in what was described as the "so what?" question. In essence, this recognises that staff have competing demands on their time and therefore need positive answers to two key questions before they will feel able to commit to any new process of change: (i) "does this affect me?"; and (ii) "can I affect this?". The experience reported by the test sites was that local partners did not make a clear enough case that the work of these individuals would be affected, nor how these individuals could affect the outcomes. Therefore the "so what?" question remained largely unanswered.

'This (mapping data) is probably not a lot of value at my level (frontline delivery). It's interesting and good to look at, but at a higher level, they should be looking at it hard"

4.24 The IRF has been supported by - and has itself supported - other policy agendas. However, to implement each policy or strategy requires staff time. Although it was generally reported that the policy direction of travel is consistent, the actual demands on frontline staff to discuss, plan and implement each policy initiative can be seen as competing.

4.25 Analysing financial and activity data is not a core skill for many of the frontline professions involved in delivering health and social care. Some stakeholders reported that training would be required in order to allow GPs, hospital clinicians and social care managers to make full use of the mapping information. This would facilitate a much more useful and progressive analysis of: (i) which issues may require data refinement; (ii) which require additional data; and (iii) which should result in actions to improve services.

4.26 As discussed above, it was difficult for GPs, hospital clinicians and social care managers to make time to look at mapping data since it is outwith the normal scope of their work remit. Training would itself require a time commitment from these professionals but may be needed for meaningful engagement with integrated cost and activity mapping.

Sharing mapping data with service users

4.27 Throughout all three rounds of the IRF evaluation, stakeholders reported the importance of public perceptions about the location, timing and quality of care. While some of the mapping data is recorded in public record, and therefore in the public domain, the IRF test sites did not focus significant effort on sharing the mapping data with service users or more generally with the public. This underlines the need to draw attention to the data which is in the public domain and develop strong skills in interpreting and presenting it.

4.28 Policy initiatives such as Self Directed Support and Anticipatory Care Planning were seen as key drivers in improving the extent to which patients/service users and their families are included in service planning. There was widespread recognition of the potential value of involving service users in the planning of their own care, while at the same time acknowledging that this could create an increased level of complexity in the management of integrated services. Test sites reported that the more detailed mapping information (at patient level or disease grouping) could be used to inform individual care planning discussions.¹⁴

Key findings

• The IRF mapping information was used to facilitate more open and transparent discussion of integrated working across health and social care. Test sites attempted to use the mapping to look at unwanted variation, identify potential service improvements and examine how services could be provided more efficiently.
• Staff involved in CHPs, locality planning groups, or IRF steering groups have a responsibility to plan, organise and manage services in an efficient, effective, and equitable way. Consequently, information to support partnership working and integrated service planning was of obvious and immediate value to this group of stakeholders.
• The potential of the mapping data to help define, measure and monitor outcomes was recognised but remains underdeveloped.
• Engaging with wider staff groups whose main role was direct delivery of care (GPs, hospital clinicians, social care managers etc.) proved difficult. The immediate demands of caring for the current needs of patients and service users make it more difficult to find time to step back and look at the impact on planning future services.
• The IRF programme was designed with engagement of GPs and clinicians at its core. However, the experiences reported by the test sites indicated that local partners did not make a clear enough case for affecting the work of these individuals nor how these individuals can affect the outcomes.
• While some of the mapping data is recorded in public record, and therefore in the public domain, there is greater potential to interrogate and then share the mapping data with service users, or more generally with the public.