Beyond survival: resource for organisations working with child abuse survivors

Part Two: Setting and Measuring outcomes

2.1 The outcome framework

We hope the framework is useful for:

• Helping people and projects to plan and measure their progress.
• Showing the unique contribution of survivor support projects to higher level impacts, policies and strategies.
• Showing the parallel but intersecting journeys that survivors, families and communities (including services) are on. It shows that for survivors to achieve their outcomes, services also need to learn, develop and change.
• Giving survivor support projects/organisations a systematic way to measure – and communicate - their outcomes
• Potentially, the framework could also be used as a touchstone for developing policy and legislation, for example making sure outcomes and aspirations are aligned.

2.2 Things to note

The outcome framework shows a ‘theory of change’, with outcomes presented in a sequence. Real life is less linear than this - links between cause and effect are more complicated - and progress isn’t always straightforward. It is also easy to oversimplify individual lives and stories into a framework like this. It is important to remember the model is a simplified representation of the outcomes people or services might achieve in a particular order, rather than a prescriptive ‘road map’. The framework also contains several ‘assumptions’ and beliefs:

• The ‘areas of activity’ listed are illustrations of activities provided by survivor support services, not an exhaustive list.
• Survivors and services might have parallel journeys, summarised by the outcome headings in bold.
• With better awareness of abuse, trauma and survivors, services will be better able to support and serve them appropriately.
• Trauma Informed education can be integrated in the community.
• Policy and practice will both be improved by involving and learning from survivors’ experience
• Long term, survivor support should be integrated in other relevant policy and service areas – survivor-specific policy would not be needed.

The framework outcomes contribute to National Health and Wellbeing Outcomes 1, 3 and 9 :

Outcome 1: People are able to look after and improve their own health and wellbeing and live in good health for longer

Outcome 3: People who use health and social care services have positive experiences of those services, and have their dignity respected

Outcome 9: Resources are used effectively and efficiently in the provision of health and social care services

And national outcomes:

• We live our lives safe from crime, disorder and danger.
• We have strong, resilient and supportive communities where people take responsibility for their own actions and how they affect others.
• Our public services are high quality, continually improving, efficient and responsive to local people's needs.

2.3 Why setting outcomes improves sustainability

There are lots of good reasons to set and measure outcomes.

• The people you work with need to know how they will benefit from your support. They also need opportunities to look back, reflect on their progress and plan their next steps.
• Staff and managers need to know what works – and what doesn’t. We all evaluate our work every day, even if we don’t call it that. Using outcomes keeps us focused on making a difference and learning.
• Funders, commissioners and policy makers need good evidence about the impact of their policies and funding. Increasingly, they don’t just want to know what difference has been made, but what’s been learned.

For all these reasons, organisations that measure their impact effectively are likely to be more sustainable than those who don’t. See the Mind Mosaic case study for an example of the ways this can help.

2.4 How to set and measure outcomes For more information and resources visit www.evaluationsupportscotland.org.uk

Outcomes are about change, the difference your work makes. You can start setting or reviewing your outcomes by answering a few simple questions:

• What is the problem, need or situation that you are trying to address? Your outcomes should reflect that. In our framework, the starting point is that survivors’ needs are often unrecognised or unmet. So, several of our outcomes are about survivors being better understood and supported.
• Who is changing, what is changing and how is it changing? An outcome should answer all three questions. For example: survivors (who) access to information and support (what) is increased (how).
• So what? List your current activities, and for each one ask ‘So what?’. Why does that activity take place, what’s it trying to achieve? Your answers should lead you towards useful outcomes. For example: We produced a sustainability guide (activity). So what? So…

Organisations’ understanding of sustainability will improve.
[Who] [What] [How]

You might find the following templates helpful.

2.5 Template outcome framework/logic model (with thanks to Inspiring Scotland)

This is a template outcome framework (sometimes called a logic model) like the one we used above. This format can be particularly useful for showing how a number of different activities (or services, partners etc.) contribute to a set of shared outcomes. It also helps you to work out what to measure and when – most people find short term outcomes easy to measure, and long term ones too hard to prove. So a framework like this can help join the dots by showing the links between the two.

Measuring the short and medium term outcomes helps show your contribution to longer term or higher level outcomes. In our example, we cannot achieve the Scottish Governments national outcomes on our own, but we can show how our work contributes. Increasingly, organisations (and evaluators) are less concerned about ‘attribution’ (how do we know our services made the difference) and are more interested in ‘contribution’. Imagine the outcomes are links in a chain – what would happen if your links were removed?

2.6 Template evaluation plan

One of the downsides of outcome frameworks is that they only show you what to measure and when – they don’t show you how. So most organisations create a separare evaluation plan. This helps break the outcomes down into a series of measures (indicators – see 2.7 below), each linked to appropriate evaluation tools and methods. A plan like this makes your evaluations much more systematic.

2.7 Working with indicators

Indicators are useful for measuring ‘soft’ outcomes, breaking intangible ideas down into specific measures that describe what the outcome looks like in practice.

For example, what would the outcome ‘Improved access to information and support’ look like in practice? Potential indicators might include:

• The extent to which individual survivors know where to go for help
• The number of local services providing trauma-informed support
• The number of appropriate referrals your service makes to other services
• The number of appropriate referrals your service receives
• The availability of information in appropriate languages and formats

Indicators are useful in lots of ways:

In evaluation planning they can help you to:

• Involve survivors to define success for themselves (individually or in groups). What would it look like or feel like when they achieve their goals? (For some survivors the first goal might be to get them to a place where then can begin to envisage the possibility of setting goals for themselves or even to begin to view that as a legitimate or possible activity).

• Work out exactly what to measure and when – ideally giving you a baseline or starting point too, so you can measure progress over time
• Developing or selecting evaluation tools for gathering evidence, specific to what you want to measure.

In analysing information they help you to:

• Link the evidence you have gathered back to your outcomes, in a systematic way
• Work out the extent to which outcomes have been achieved. Outcomes aren’t usually black and white. Good quality, convincing reports are honest about which indicators have been evidenced – and which have not.

In reporting they help to:

• Structure reports, keeping them focused on what matters.
• Presenting a picture of what the outcome ‘looks like’ with examples from real life.

Example activities and indicators for the framework outcomes

These examples are intended to help you think about the sorts of measures you could use for each of the framework outcomes. Remember that the outcome might look very different in different settings, so it’s best to develop your own specific indicators. Survivors should be able to help you with this – individually or in groups, they are likely to know exactly what the outcome would ‘look like’ or mean for them. Involve survivors in developing indicators if you can.

1. Improved access to information and support

Example activities:
Promoting services e.g. in GP surgeries, schools, colleges, social media etc.
Providing a database of services
Early assessment and appropriate signposting
Shared understanding and regular communication with other agencies

Example indicators:

• The extent to which individual survivors know where to go for help
• The number of local services providing trauma-informed support
• The number of appropriate referrals your service makes to other services
• The number of appropriate referrals your service receives
• The availability of information in appropriate languages and formats

2. Individual barriers to accessing support are removed

Example activities:

• Awareness raising e.g. via social media, positive campaigns
• Providing appropriate information in public spaces
• Self-management and self-advocacy workshops

Example indicators:

• I know I am a survivor
• I accept I am entitled to support
• I don’t stigmatise myself…
• I understand my experience and start managing its impact

3. Increased knowledge of choices and support for wellbeing and work

Example activities:

• One to one support and signposting
• Maintaining up to date awareness of local resources
• Helping survivors to access other services that can help to identify and get support with their needs e.g. advocacy, CAB

Example indicators:

• Number of people who report awareness of appropriate services
• Number and range of supports accessed by people
• Reported levels of confidence in accessing future supports
• Choices and decisions are reflected in personal action plans

4. Services have better access to trauma-informed information about supporting survivors

Example activities:

• Input to meetings and events
• Sharing publications and articles with other professionals
• Connecting partner organisations with relevant supports e.g. NES Trauma Framework

Example indicators:

• Number of services attending training
• Number of services attending relevant partnership meetings, number of meetings attended
• Reported knowledge of participants

5. Services are better able to recognise trauma - and survivors

Example activities:

• Service-based workshops on responding to disclosure
• Support line for external agencies’ staff
• Promoting and training professionals in the NES Trauma Training Framework

Example indicators:

• Workers recognise the signs to look for
• Workers feel confident to approach the subject
• Number of enquiries specialist services get from other agencies seeking information

6. Services make it easier for people to get support - removing barriers, reducing stigma

Example activities:

• Providing information on impacts of abuse on physical wellbeing as well as psychological
• Proactive referrals, not just signposting
• Open access and walk-in referrals

Example indicators:

• Services make necessary adjustments to systems and procedures to simplify and improve access to services
• Survivors report feeling safe and welcome

• Increased follow-on referrals
• Survivor engagement with partner organisations

7. Services are more able to provide choices and support for wellbeing and work

Example activities:

• Scoping services, gathering recent data on other services and their focus
• Survivor journey mapping – your own service and those they come into contact with
• Person-centred and survivor-centred practice forums

Example indicators:

• Services mention survivors and trauma informed work in their publicity information

• Services make necessary adjustments to systems and procedures to simplify and improve access to services

• Survivors report feeling safe and welcome Enhanced range of services available

8. Improved opportunities and services that meet personal outcomes

Example activities:

• Coordinating partnership and network events e.g. Trauma and Addiction Partnerships
• Scoping services, gathering recent data on other services and their focus

Example indicators:

• Number of trauma informed services
• Number and variety of survivor-specific services
• Geographical availability of support

• Number of services attending relevant partnership meetings

9. Reduced isolation

Example activities:

• Peer support groups/events
• Workshops on improving personal capacity and relationship building
• Volunteering opportunities

Example indicators:

• Ability to build relationships
• Survivors report feeling less alone
• Survivor involvement in formal or informal peer support
• Time spent with others, for example in group activities
• Taking part in community events, services and activities independently

10. Increased mutual trust and understanding between services and survivors

Example activities:

• Staff training in childhood abuse and its impact
• Welcoming and addressing clients appropriately, taking time with them, showing acceptance
• Signing up to appropriate standards, e.g. LGBT Charter awards

Example indicators:

• Level of mutual comfort with disclosure
• Services make necessary adjustments to systems and procedures to simplify and improve access to services
• Survivors report feeling safe and welcome Services and clients understand and uphold confidentiality

• Number of other services reporting increase in uptake from survivors

11. Services are better able to serve survivors and make appropriate referrals

Example activities:

• Involving survivors in developing services and approaches
• Carrying out evaluation and quality control
• Hosting an open day for partner organisations

Example indicators:

• Services make necessary adjustments to ensure survivors feel safe and welcome

• Number or survivors accessing other services
• Number of other services reporting increase in uptake from survivors
• Improvement in service waiting times

12. Improved partnership responses

Example activities:

• Evaluation and quality control processes
• Providing opportunities for partnership working and sharing experience
• Establishing information sharing processes and protocols
• Reciprocal inter-agency training

Example indicators:

• Demonstration of coordinated approach to survivor support
• Level of waiting time for appropriate support
• Appropriate information sharing where express permission is given

13. Increased prevention

Example activities:

• Safe (confidential) opportunities for disclosure
• Training and supporting other organisations to develop policies and practices on prevention
• Public awareness and media campaigns, including survivors’ voices
• Support for people with thoughts/behaviour that could lead to abuse

Example indicators:

• Services and people can give examples of recognising and responding to grooming
• People can give examples of what to do if they don’t feel safe
• Professionals know how to have a conversation about their or others’ concerns
• The wider public knows the early signs of risk and know where to go for help
• Length of time between negative experience and disclosure
• Professionals’ ability to challenge organisational practices that are not helpful to prevention

For more information on prevention activities and outcomes, there is a Child Abuse Prevention Framework written by practitioners for practitioners, to help explain and evaluate work to prevent child abuse happening. It was created by a working group facilitated by Scottish Government and Evaluation Support Scotland. The framework is not a finished product and it can be developed for your own work. http://www.evaluationsupportscotland.org.uk/media/uploads/prevention_framework_for_website_prevention_page.pdf

14. Development of coping strategies and resources for life

Example activities:

• Workshops to develop self-management and self-advocacy skills
• 1:1 support on developing enhanced self-awareness
• Groups and 1:1 support to develop alternatives to harmful coping mechanisms

Example indicators:

• Awareness of self-management skills
• Awareness of self-advocacy skills
• Ability to choose appropriate responses in specific situations, e.g. using logic to overcome emotions that don’t belong in present situations
• Ability to express needs and aspirations

15. Increased resilience

Example activities:

• Practising self-management tools and techniques in role plays in safe groups
• Providing therapy
• Helping survivors to plan and measure progress towards their outcomes

Example indicators:

• Use of self-management skills
• Use of self-advocacy skills
• Ability to choose appropriate responses in specific situations
• People have the information they need to make informed decisions

16. Increased social confidence

Example activities:

• Befriending and peer support groups
• Volunteering opportunities
• Running activity groups, helping people identify and start/restart hobbies and interests

Example indicators:

• Ability to express needs and aspirations
• Level of participation in activities and conversations
• Ability to form relationships
• Taking part in community events, services and activities independently

17. Reduced impact of trauma

Example activities:

• Supporting self-advocacy and self-management, helping people to identify, develop and use healthy coping strategies
• Support to attend appointments e.g. GP, hospital, mental health services
• Producing toolkits of resources and coping strategies

Example indicators:

• Increase in reported ability to manage the ups and downs of life
• Ability to choose appropriate responses in specific situations, e.g. using logic to overcome emotions that don’t belong in present situations
• Ability to express needs and aspirations
• Ability to set and work towards personal goals
• Levels of self-worth and self-care: ‘I am a good person, who deserves to be heard and valued by myself and other people’

18. Increased access, social inclusion, and contribution to community life and activities

Example activities:

• Develop and implement tiered approaches to community service provision
• Targeted services for people identifying as having experienced childhood abuse
• Universal services for people who have experience of childhood abuse but who can’t or aren’t ready to identify and seek specialist support

Example indicators:

• Level of participation in community activities
• Survivors achieve their own goals for improving their situation or quality of life
• Involvement in community volunteering opportunities
• Contributions to ‘service user voice’ activities; capturing impact and resilience journey

21. Reduced costs through improved partnership responses and increased prevention

Example activities:

• Participation in relevant local and national fora to learn about and influence policy
• Identifying which national and local strategies your service contributes to
• Developing local (or national) policy, not just waiting for others to take the lead
• Sharing/promoting opportunities for joint funding bids
• Exploring options for ‘sharing’ access to specialist staff

Example indicators:

• We are viewed as valuable partners by policy and decision makers
• Policy and decision makers report finding our evidence valuable and credible
• Commissioners and policy makers know how we contribute to their strategies
• Survivors and groups report consistent, satisfactory involvement with policy making processes
• Trauma informed practice is recognised as a crucial element of other policy strands (social care eligibility and provision, health outcomes, adverse childhood events, etc.).