Developing an Outcomes Model for Disabled Children in Scotland

3 Outcomes Models: Framework and Context

Context

3.1 An emphasis on measuring outcomes in children's services has become enshrined in Scottish Government policy. "In 2006 the Scottish Government stated that less time should be spent on measuring what goes into services and how money has been spent, and that more time should be invested in what funding achieves for individuals and communities. This was followed under the Concordat locally, by the overarching Single Outcome Agreement (SOA), which set out a new relationship between central and local government, allowing for more flexibility at the point of delivery.

3.2 However, implementing an outcomes policy and approach, affects more than just local authorities and Community Planning Partnerships. Cook and Miller (2012) distinguish between outcomes for individuals, and outcomes for services, organisations or nations. They stress that the personal outcomes need to drive outcomes activity. They place these personal outcomes in relationship to the others illustrated in Diagram 1, which expresses them as different 'levels'.

Diagram 1: Different levels of outcomes from Cook and Miller 2012

Outcome level	Focus	Examples
Individual/personal	Defined by person: what is important to them.	I want to be able to get back to the bowling club, so that I can spend time with my friends.
2. Service/project	Defined by a project or service as a key focus to work towards with people.	We work with older people to improve their ability to get out and about.
3. Organisational	Defined by local authority. NHS Board or provider as a key area to work towards. Will increasingly be required to be defined across organisations.	Improve the social inclusion of the older people we work with.
4. National	Defined by government to focus activity across sectors and organisations.	We live longer, healthier lives. Our people are able to maintain their independence as they get older and access appropriate support as they need it.

3.3 Their work is very helpful in locating (though perhaps not relating) the outcomes currently being defined in different areas of Scottish services. It also suggests a number of issues, in particular about the relationship between the different levels, and raises a number of problems when applied to the situation of disabled children⁴.

3.4 Miller (2011) cites the following benefits to all the outcome levels in this framework:

• For people who use services and their families, being involved in defining the outcomes they want to achieve can be empowering and result in increased relevance and support.
• Working with individuals to develop outcome-focussed plans, and reviewing the outcomes achieved, can help achieve clarity of purpose.
• An outcomes approach can help organisations to reconnect with their value base and ensure that they are focussed on the differences they make to people's lives as well as the activities undertaken.

Do the levels relate?

3.5 Personal outcomes are not achievable in isolation, nor can they be monitored outwith the context of the other outcomes. Individuals receiving services interact with staff from agencies. The agencies are part of local authorities, health boards or third-sector organisations, each with their own outcomes policy and each of which must report to government and/or funding agencies.

3.6 Ideally, for a complete system of outcomes, the lowest levels - those closest to the individual receiving a service - should feed upwards. The achievement of personal outcomes should inform and further the achievement of national outcomes.

3.7 However, policies about desired outcomes often feed down from the top, reinforced by legislation, regulations and/or financial systems. They are not necessarily congruent with the outcomes being sought by individuals receiving services. This tends to be true for children, in general, and disabled children in particular.

3.8 Looking at how the levels relate involves complex sets of interactions involving management, working procedures and reporting mechanisms, within and between levels.

Personal outcomes

3.9 Cook and Miller (2012) look at how the personal outcomes approach should be implemented in practice, but not the wider issue of how these relate to other outcome levels.

3.10 They contextualise the personal outcomes approach: "There is a widespread recognition that systems need to shift from an exclusive focus on what they do to include consideration of what difference they make to the people using services and support. A personal outcomes approach can support that. This means working with the person to identify what is important to them or what they want to achieve, and then working backwards to identify how to get there. It means supporting the individual to be as independent as possible, while paying attention to their quality of life... working like this means a shift from 'ticking boxes' to engaging with people."

3.11 They describe personal outcomes as: "what matters to people using services, as well as the end result or impact of activities and can be used to both plan and evaluate activity."

Implementing a personal outcomes approach

3.12 Cook and Miller identify three components of the 'Talking Points' approach:

• Engaging with the person, to identify what is important to them in life and planning how everyone is going to work together to achieve these outcomes. This means that conversation is important, with listening being an important part of that process.
• Recording the outcomes in a support plan, which is shared with everybody concerned.
• Using the information to ensure that what matters to the person is used to influence service planning and implementation.

3.13 Miller (2012) added: "It cannot be assumed that service users' views on their outcomes will correspond with those of organisations and practitioners". This is critical since it refers to the interface between an individual's views and the outcomes outlined in any pre-defined model.

3.14 Cook and Miller also identify three types of outcomes:

• Maintenance or quality of life outcomes: Aspects of a person's whole life that they are working to achieve or maintain, including being well.
• Change outcomes: Focus on short-term removal of barriers that relate to the improvements in physical, mental or emotional functioning that individuals are seeking from any particular service intervention or support.
• Process outcomes: Relate to the experience people have seeking, obtaining and using services and supports, which focus on how services are delivered. (Cook and Miller, 2012, and Miller 2011)

3.15 The outcomes above are interrelated and should not be considered in isolation. "Specific services may emphasise particular types of outcome but ... research with service users demonstrated that the process, or how services engage with people, is inseparable from, and shapes the outcome." (Miller 2011)

3.16 There are also outcomes for unpaid carers, which emphasise the importance of carers being included as partners in decisions about the person. This is a central issue when considering outcomes for disabled children.

3.17 Finally, this approach "involves a shift away from the way services currently do business ... so staff need time for the conversations involved and the organisation needs to support them in doing this ... the service needs to use the information about outcomes to make the improvements required ... so management needs to be outcomes focussed too ... strong leadership is required to live through the change at every level".

Applying this approach to disabled children and young people

3.18 Applying this framework to disabled children is complex for a number of reasons. The three dominant reasons are identified below:

• Multiple parties involved: Given that there are multiple people involved (from parents/carers to professionals), there are also multiple perspectives, preferences and priorities at work. For instance, agencies and professions usually have their own (sometimes complementary, sometimes contrasting), procedures, time-scales, information-sharing systems, training, and work cultures. This is the everyday working reality for these professionals and the outcomes devised for an individual child has to fit into these multiple frameworks.
• Multiple needs: Disabled children will frequently have more than one disability, condition or situation with which they must deal. This usually means that they relate to different agencies or frameworks, bringing with them a range of sometimes disparate outcomes.
• Engaging with the child: Disabled children may have foundational needs that must be addressed before others can be effectively resolved. This may involve communication issues or other issues affecting their ability to interact easily with the people with whom they are working. Disabled children of different ages, maturity, and personality do not always see eye-to-eye with some (or many) of the adults with whom, they are involved. This makes decision-making about whose preferred outcomes will prevail a complex and potentially contentious matter.

Challenges in implementation

3.19 Miller (2011) lists a number of challenges when measuring outcomes:

• Clarity of purpose: "It is important to be clear about the purpose of measuring outcomes. Is it primarily for improvement purpose or to make judgments?"
• Measurable or meaningful: "One of the policy priorities in service improvement is that the results should be measurable. Recent research highlighted the limitations of quality measurement including the tendency to miss areas where data are not available, and to miss less quantifiable aspects of quality. Evidence reveals the adverse effects of prioritising external reporting, particularly in the form of targets."
• Hard or soft outcomes: "Soft outcomes give a fuller picture of the overall value and success of projects... the most vulnerable users were viewed as missing out because they were less likely to achieve quick and measurable outcomes."
• Challenges of attribution: This addresses the "challenges of establishing cause and effect, or attribution... this is further complicated where there is multi-agency involvement".
• Variation in service users: "Variations in the characteristics of service users leads to challenges in interpreting the data... Responses can be influenced by service user characteristics unrelated to the quality of care", which is particularly pertinent when measuring outcomes for disabled children or young people.

A variety of approaches

3.20 Miller (2011) identifies a number of approaches to developing outcomes models including "Theory Driven Evaluation", "Logic Modelling" or "Choosing or Designing Outcomes Tools." It is not proposed to expand on these here but they are identified where used in the summary of outcomes activity taking place in Section 5.

3.21 A briefing from Community Care Providers Scotland and others, An Outcomes Approach in Social Care and Support: An Overview of Current Frameworks and Tools (2010) outlines a number of approaches that have been developed. Further information about some of these has been included in Section 5 and Appendix B of this report.

3.22 Multi-agency involvement with a child's outcome planning adds an additional layer of complexity as it raises the issue of "attribution". It is very challenging to clearly identify what factors have achieved particular outcomes. It can be more helpful to think about contribution rather than attribution, in aiming to identify the contribution of each agency involved towards achieving the outcomes. This will be helped by the idea of a "single plan" for each child, the child's plan. Different agencies may require their own more detailed plans depending on their role or statutory obligations but the single child's plan should embrace these specific issues.

SUMMARY of FINDINGS

This section distinguishes between outcomes for the individual (personal outcomes) and outcomes for services, organisations and nations. These different outcome 'levels' are interrelated and are not achievable in isolation.

Ideally, the achievement of personal outcomes should inform the achievement of outcomes from services, organisations and nations.

A personal outcomes approach requires a systematic shift so that the focus is on working with individuals to identify what is important to them or what they want to achieve, and then working backwards to identify how to get there.

A number of challenges are identified in applying this framework to disabled children. These include:

• A child's multiple needs and the number of professionals and agencies involved, as well as the views of parents/carers.
• Communication or other issues that affect a disabled child's ability to communicate with people.
• Whilst soft outcomes provide a fuller picture, they are harder to measure. If measurable results are required, there may be concern that the most vulnerable users might miss out on equally important outcomes that are most challenging to measure.