Self-directed Support Implementation Study 2018: report 2

Appendix 1: Literature review methodology

Literature search

Given the wide range of terms that can and are used to describe schemes similar to self-directed support in Scotland, a broad search strategy that was designed to have high sensitivity was developed.  As many of the terms also relate to personalised health care, the strategy was expected to have low specificity which would mean a large number of erroneous results that would need removing at the study selection stage.

The reference list from a literature review from 2016^[13] was used as a means to check that the strategy was performing adequately and identifying expected studies.  The final search strategy is presented in Table A1.1 below.

Table A1.1: Search strategy 

HMIC Health Management Information Consortium <1979 to November 2017>
1     (individual adj3 service fund$).ti,ab. (5)
2     individual budgets/ (256)
3     (individual$ adj3 budget$).ti,ab. (127)
4     (personal$ adj4 budget$).ti,ab. (303)
5     ((personal$ or individual$) adj3 care planning).ti,ab. (50)
6     ((personled or person led) adj3 commissioning).ti,ab. (0)
7     (individualised care or individualized care).ti,ab. (139)
8     ((care or commissioning) adj5 (intermediary or intermediaries)).ti,ab. (7)
9     flexible commissioning.ti,ab. (0)
10    ((personalisation or personalization) adj4 (care or commissioning or budget$)).ti,ab. (82)
11    ((selfdirected or self directed) adj3 (care or support or commissioning or budget$)).ti,ab. (85)
12    ((selfmanaged or self managed) adj3 care).ti,ab. (4)
13    independent support.ti,ab. (9)
14    (cash adj3 care).ti,ab. (74)
15    or/1-14 (854)

The search strategy was translated so that it could be run in Medline, Embase and Social Policy & Practice and HMIC.

Eligibility criteria for inclusion in the review (PICOS)

Population

Any person eligible for support from social care, their carers or their families; social care service providers; the social care workforce; and government (local, state or national).

Interventions and comparators 

Any interventions that enabled choice or control over the social care provided to a supported person was eligible for inclusion.  This included (but not exclusively) interventions that were described as:

• Self-directed support;
• individual funding plans;
• personal budgets;
• direct payments;
• cash-for-care; or
• brokerage or advocacy services.

Outcomes

Any evaluation outcomes were eligible for inclusion (quantitative or qualitative).

Study design

Evaluations of local or national interventions either at a pilot stage or already implemented were eligible for inclusion. Studies that did not state they were evaluations but reported research that was evaluation in nature were also eligible. Studies that just described an intervention were excluded as were simple audits of data.

Studies published as abstracts of conference presentations were not eligible for inclusion. Literature reviews were used only as a source of references to primary studies.

Only papers published in English were included.

Table A1.2: Summary of eligibility criteria

	Inclusion Criteria	Exclusion Criteria
Population	Supported people of any age and with any need, their carers or their families Service providers Service brokers Advocates Local Authorities
Interventions and Comparators	Self-directed support Individual Budgets Personal Budgets Direct Payments Cash-for-care Any intervention enabling choice and control for supported people	Interventions related to health care only
Outcomes	Any outcomes	Studies not reporting any outcome data
Study design	Full evaluations or reviews of monitoring data	Literature reviews Audits Descriptive case studies Opinion pieces Conference abstracts News Comments Editorials Letters

Study selection and extraction

The purpose of the review was not to list in detail methodological approaches taken, but to highlight approaches that could inform an evaluation of self-directed support in Scotland.  The focus was on selecting evaluations of similarly embedded national programmes rather than on pilot or local evaluations although any useful learning from such evaluations was still to be captured.

As such, a pragmatic rather than systematic approach was taken to study selection for extraction. All evaluations of national interventions were extracted but only the primary evaluation report was extracted and not secondary reports covering specific issues.  The exception was if there was a methodological element or concern in a secondary report not covered in the primary report.  Remaining studies of pilot or local programmes were reviewed in alphabetical order and extracted provided they gave information on methodological approaches or concerns not already extracted from a paper already reviewed.

A data extraction template was developed in MS Excel. Date items extracted were:

• Country;
• Year of research;
• Name of intervention;
• Description of intervention;
• Quantitative outcomes;
• Quantitative methods;
• Key qualitative research questions;
• Qualitative methods;
• Limitations in methods identified by author; and,
• Noteworthy elements of evaluation for evaluability assessment.

In all cases, only data of relevance to an evaluation of SDS in Scotland were extracted.

Results

Results of the searches

The literature searches were conducted on 11-12 January 2018 and retrieved 6,120 unique records.

Studies identified and selected

As suspected, the search strategy had low specificity resulting in 5,972 records excluded at abstract screening. The full texts for 148 records were retrieved and were assessed for relevance. One hundred and twenty-nine documents were excluded at full text stage.

Of the 19 included studies (1-19), ten were extracted as evaluations of existing programmes at a national level (1-10).  As self-directed support is an existing programme that is now embedded nationally, these evaluations were the most informative in understanding the issues involved in monitoring and evaluating self-directed support.

A further six studies of pilot programmes (11-16), two studies of local evaluations of national programmes (17-18) and one study of a local evaluation of a local pilot (19) were extracted.

A list of extracted references is included in Appendix 2.