Follow-Up Evaluation of Self-Directed Support Test Sites in Scotland

1 INTRODUCTION

Study Background

1.1 While the term 'Personalisation' has been used to describe the approach applied in social care to increase self-determination, choice and control in England (Glendinning et al, 2008), in Scotland the focus has been on promoting self-directed support (SDS) defined in the recent Social Care (Self-Directed Support) (Scotland) Bill. The terms 'personalisation' and 'SDS' are sometimes used interchangeably, while elsewhere, they are used to distinguish a broad approach to social care (personalisation) from a set of particular mechanisms for implementing personalisation e.g. using a process that identifies an individual budget (SDS).

1.2 In 2009, Scottish Government selected 3 local authorities to act as test sites - Dumfries & Galloway, City of Glasgow and Highland - to trial targeted activities addressing 3 key issues identified by past research, as part of its investment in promoting SDS. A 2-year evaluation of the test sites was commissioned by Scottish Government to inform national strategy and the development of SDS legislation, and this found SDS to be an 'evolving concept' and one that was clearly interpreted variously in practice (Ridley et al, 2011).

1.3 The 10-year national plan for SDS (Scottish Government, 2010) aims to bring SDS into the mainstream of social care and increase the number of people directing their own support, including the number doing so via Direct Payments (DPs). If enacted in 2013, the Social Care (Self-Directed Support) (Scotland) Bill will make offering SDS the duty of local authorities. Evaluation of the SDS test sites set up 2009-2011, demonstrated that local authorities face a number of challenges in implementing SDS and achieving transformational change (Ridley et al, 2011). Another study (Rummery et al, 2012) of the macro level financial and economic evidence on costs, benefits and impacts of increasing SDS in Scotland, concluded that while implementation remains an area of contention, the cost of further uptake does not differ significantly between SDS and more traditional services.

1.4 A number of commentators, including ADSW (2009), argue that the change needed will be transformational for the work of those at the frontline of providing support and services, as well as for the structures and systems of service delivery in local authorities and other sectors. Directors of Social Work from the 3 test site local authorities giving evidence to Scottish Parliament in May 2009 identified what they called the "seismic change" required, and referred to the inevitable slow progress in terms of numbers opting for SDS initially (Scottish Parliament, 2012).

1.5 Evaluation of the local authority test sites established that those accessing SDS packages were positive about the choice and flexibility offered, although it was unclear whether this was solely as a result of SDS or the greater levels of support and funding made available during the test sites (Ridley et al, 2011). Access to SDS increased particularly among people with learning disabilities, the main target group of at least 2 of the test sites. The study found that the test sites took longer than anticipated to get established - recruiting staff, raising awareness, training and setting up SDS systems, and providing support - and that numbers of people accessing SDS were relatively low across the 3 areas. It concluded that significant time and investment in infrastructure was required to implement such a major policy initiative.

1.6 The fieldwork for the evaluation was completed in March 2011 just as the test sites were building momentum and as new SDS systems were starting to see progress. By the end of the 2 year period, the 3 local authorities had resolved to move towards full implementation of SDS with support from senior management. Follow-up work was commissioned by Scottish Government to examine progress and developments in implementing SDS in the test sites in the intervening 12 months i.e. from 1^st April 2011 to 31^st March 2012. This report presents and discusses the findings from the follow-up evaluation of the test sites.

Evaluation Aims & Objectives

1.7 The overall aim of the follow-up work was to assess the continuing and longer term impacts of the interventions employed in the SDS test sites in the original project. The objectives were to:

• Assess the continued uptake and impact of the interventions used to improve uptake of SDS in each test site.
• Identify the activities to further promote and increase awareness and knowledge of SDS, particularly amongst care users, carers and the workforce.
• Identify system wide change within the test sites and what can be learnt from this change.

1.8 Further, given that the evaluation report raised a number of questions around the sustainability of the systems introduced by the test sites, it was of particular interest to explore what progress had been made in each of these areas in terms of:

• The extent to which the local authorities had moved towards mainstreaming SDS and the leadership provided by senior management in achieving this.
• Whether the creative and innovative ways of working developed by the SDS teams within the test sites had been maintained and further developed.
• The extent to which test sites had been able to implement a shift towards greater involvement and co-production of care and support.

1.9 The original evaluation suggested that the reach of SDS packages was limited mainly to people with learning disabilities, and that no one from a black and minority ethnic (BME) group had benefited. It was therefore important to investigate how local authorities had subsequently tackled such inequalities in access to SDS. Another issue raised was the absence of independent advocacy from the support available to those accessing SDS during the test sites. As well as evidence of brokerage arrangements, the follow-up evaluation sought evidence of support to independent advocacy organisations and about how their role may develop in the context of SDS.

1.10 Given the wider financial context of resourcing social care, it was also important to gain some understanding of how this might be impacting on SDS. For example, how SDS was being presented in this context to potential service users and carers; what information was being given to service users about SDS; how care managers perceived the impact on support arrangements; and assessing whether packages were agreed and/or sustainable post test site.

Evaluation Design

Methods

1.11 Building upon the design and data from the original evaluation of the test sites, the study collected information about developments and progress over the time period of 1^st April 2011 to 31^st March 2012. Four main methods were used to build as comprehensive a picture as possible within a short timeframe as follows:

• Interviews with local stakeholders in each area (including staff and managers from the local authority, third sector organisations, advocacy and service user organisations).
• Cohort information collating data about access to SDS and types of SDS packages.
• Information from local documents including Social Work Committee reports, leaflets and promotional materials, including a 'mystery browser' exercise of local authority SDS websites
• Questionnaire survey of care managers and other social work staff involved in community care assessment.

Populations & Samples

1.12 A brief summary of the samples of different stakeholders and information obtained for the evaluation is given below.

Stakeholder interviews

1.13 We aimed to interview 12-15 key stakeholders in each test site, and worked with local authority SDS leads to identify the most relevant local authority and external stakeholders to approach. In addition to local interviews, we interviewed 2 members of the SDS Team at Scottish Government. We aimed to speak to those directly involved in and managing SDS teams and local authority senior managers including Directors of Social Work, service user organisations, advocacy organisations, and third sector service providers. A total of 67 individuals participated in 42 interviews during May to July 2012, that is, 65 local stakeholders and 2 members of the SDS Team at Scottish Government. Mainly individual face-to-face interviews (8 were requested by telephone) were conducted, and were a mix of individual and group interviews. The higher number of participants in Glasgow, and also Dumfries & Galloway, was due to greater numbers of service users and providers being involved in group interviews:

Table1.1: Number of stakeholder interviews and participants in the 3 test sites

Type of stakeholder	Dumfries & Galloway		Glasgow		Highland		TOTAL
	No of interviews	No of participants	No of interviews	No of participants	No of interviews	No of participants	No of interviews	No of participants
Local authority (incl SDS Teams, DP Officers, Commissioners, & senior managers)	8	14	7	9	7	9	22	32
Service user organisations (including support organisations, group interviews with users)	1	1	2	10	1	1	4	9
Independent advocacy (including carers)	2	2	3	3	2	2	7	7
Voluntary sector providers	3	4	3	9	2	3	7	17
Total	14	21	15	31	12	15	40	65*

*Total number of interviews does not include 2 interviews with Scottish Government staff

Cohort information

1.14 The 3 local authorities were asked to complete a shortened database form (cohort form) used in the original evaluation to gather information on demographics and packages of SDS recipients during the test site period - i.e. from 1^st April 2011 to 31^st March 2012. As there are different definitions about what is counted as SDS we have summarised information from the local authorities about the populations counted.

1.15 Dumfries & Galloway supplied information for all those who had chosen SDS as their route and completed a support plan. Most had this agreed by a resource allocation panel. Information from Glasgow was derived from its client group database and distinguished those who had opted for SDS and had an SDS assessment only; those who had been assessed and had a support plan; as well as those whose support plans had been agreed including the SDS option(s) chosen. The cohort information for Highland included all who had opted for the SDS route, whether these were new SDS clients or reviewed DPs that were following the SDS approach (completed SDS assessment and support plan).

Documentary analysis

1.16 Each local authority was asked to provide relevant documentation. These were obtained electronically and included joint community care plans, SDS leaflets and promotional materials, newsletters, service change plans, action plans, press releases, committee reports and anything else that the local authority felt would help us to better understand developments since the test site. We also utilised a brief 'mystery browser' exercise where we searched each Council's website for information about SDS.

Questionnaire survey

1.17 A questionnaire survey, using the SurveyMonkey internet tool, surveyed the views of staff with responsibility for community care assessment and care management in all 3 local authorities. While Dumfries & Galloway and Highland provided comprehensive lists of email addresses for all relevant staff in adult care, older people's teams, and children and family teams working with children with disabilities, Glasgow provided a list of staff who had participated in personalisation training. This limits our ability to make any direct comparisons, especially in relation to increased awareness about SDS amongst staff. The survey was initially sent during July 2012 and two reminders were issued.

1.18 Out of 500 potential respondents, we received 213 replies (93% of which were fully completed). This equates with a response rate of approximately 43%. Some caution needs to be applied in interpreting the responses from Highland given the considerably lower response rate from this site. The number of non-respondents who stated the survey was not relevant to them was also highest from Highland.

Table 1.2: Survey questionnaire responses by local authority

Local Authority	Potential no. of responses	No. of Responses	% response rate
Highland	175	40	23%
Dumfries and Galloway	127	61	48%
Glasgow	198*	112	57%
Total	500	213	43%

*31 email addresses from Glasgow were non deliverable

Analysis

1.19 Notes were taken at all individual and group interviews. Data were analysed using qualitative methods to identify themes and patterns (Coffey & Atkinson, 1996). This process was largely driven by the research aims and objectives. Data was initially organised under five main categories corresponding with the study specification. These were: awareness and promotion; access and systems; uptake; organisational and structural issues; and the impact of financial cuts. Data was analysed and themes collated by the team collectively.

Study Limitations

1.20 The original evaluation was hampered by the relatively short time period (2 years) given the delayed start of the test sites. This follow-up study therefore attempts to capture the longer term impact of the test sites. It was however limited in scope and there were many interesting lines of possible inquiry that could not be pursued within the limits of the project timeframe (6 months) and the budget. Most significantly the evaluation lacks inclusion of consultation with service users and carers in measuring impact and outcomes. Nevertheless, this study provides insights into the continued development of SDS in the 3 sites from the perspectives of stakeholders from different sectors.

Report Structure

1.21 The findings of the follow-up study are presented thematically in the next 5 chapters, which also draw out differences between the 3 local authority sites wherever possible. Chapter 2 begins by looking at what interviews with key stakeholders and local reports told us about the structures and organisational issues experienced since the test site. This includes whether or not the specialist SDS teams were retained and any changes in their role as well as perceptions of the leadership role of senior management. Chapter 3 summarises the data from interviews, documents and the survey on promotion and awareness raising of SDS by the local authorities, while Chapter 4 presents the evidence post test site on access and uptake including from the cohort forms, commenting on ways that the sites had addressed equality issues and scale of implementation. Chapter 5 then describes progress and issues with implementing the systems and processes of SDS across the local authorities, and Chapter 6 looks at perceptions, particularly those of care managers and other professionals, of implementing SDS at a time of severe financial constraint on public services. The final section, Chapter 7, draws together some conclusions from the findings and the implications for policy and practice.

Terminology

1.22 As stated earlier, the original evaluation found SDS to be an evolving concept, and the terms SDS and personalisation to be used both interchangeably and distinctly. In this report we have used both SDS and personalisation in ways that reflect how the terms were used in the local sites. We have also used the term 'care manager' as shorthand when referring to survey respondents. While this included staff with the job title of care manager, it also included social workers, team leaders/managers, care coordinators, and practitioners who had responsibility for community care assessment and care coordination.