Volunteering for All: national framework - literature review

Appendix Two: Literature Search Strategy

Our proposal document for the Volunteering Outcomes Framework literature review project set out a number of ‘Example search string components’. A series of preliminary searches were undertaken with a view to refining these terms, developing search strings, and making recommendations for the search strategy. In particular, the objectives of the preliminary searches were:

• To test different relevant search terms to get a broad understanding of how many potentially relevant items might be identified in the bibliographic databases searched under different search conditions.
• To consider how best to manage the balance between sensitivity (identifying all relevant items but at a risk of identifying large numbers of items not relevant to the current study) and specificity (using more constrained search strings and limitations but at the risk of not identifying potentially relevant items).
• To develop an optimal search strategy which searches the evidence base comprehensively and produces a manageable number of potentially relevant items in a resource-efficient way.

A number of searches were conducted between 13 and 23 August 2018. The initial focus was on using ‘Web of Science – all databases’ to test search terms, strings and inclusion criteria. This is a gateway service which includes a number of large bibliographic databases and from experience often produces large numbers of potentially relevant items. To test refinements made, searches were then carried out across each of the remaining three bibliographic databases services to be used for the review (ProQuest, EBSCOhost, ScienceDirect).  

Developing the search strategy – Recommendations

Below documents the steps we undertook to develop our search strategy approach and the recommendations for our approach that were agreed.

Developing the ‘volunteering-related term’

An initial set of searches aimed to refine and develop the ‘volunteering-related term’ for use in search strings. These began by using ‘Topic’ (TS) as the field tag. ‘Topic’ searches for items in which the specified terms appear in Titles, Abstracts, Keywords and Indexing fields such as Systematics, Taxonomic Terms and Descriptors. The search strings use Boolean searches.

The publication date limiter was initially set to ‘All years’. In the service description this defined as ‘1864-2018’, and in practice means from the earliest date at which items were included in individual bibliographic databases.

• Recommendation 1: The final composite volunteering term should be ((volunteer* NOT “healthy volunteers” NOT “human volunteers”) OR charit* OR unpaid OR unsalaried OR “civic engagement”)’ in order to increase search specificity.
• Recommendation 2: The composite volunteering term should be limited to the ‘Title’ field to keep number of items identified to manageable levels within the review.

Considering publication date limitations

Initial searches considered the results of date changes using the unrefined volunteering-related composite term and using ‘Topic’ as the search field. A range of publication date limits were tested to allow an understanding of how the numbers of potentially relevant items might change under different restrictions. The results obtained show that the numbers of potentially relevant items identified were unmanageable in the context of this review without the refinements to the volunteering-related composite terms recommended above AND the use of a restricted publication date inclusion criterion. The final publication date inclusion criterion tested, 2007-2018, allows for identification of relevant publications over the last decade, but many of these referenced key texts from before 2007. 

• Recommendation 3: A publication date inclusion / exclusion criterion should be used. Only items published after 1 January 2007 should be included.

Developing the general ‘outcome-related’ composite term

Development of other potential composite search terms was primarily based on the recommended refinements to the volunteering-related composite term, search field and inclusion dates for publications noted above. However, the first search was undertaken using the individual outcome-related term ‘result*’ prior to other refinements. This search identified an issue with identification of very large numbers of potentially relevant items as a consequence of the term appearing as a heading in the standard format of abstracts in many scientific journals. The remaining searches examined other proposed elements in the general ‘outcome-related’ composite term.

A decision was made to retain ‘Topic’ as a search field for terms other than the volunteering-related composite term as this provided for greater sensitivity than more restrictive search fields and the results obtained suggested a manageable number of items for subsequent screening.  

• Recommendation 4: The final composite beneficiary-related terms should include all of the individual terms identified. The final composite micro-beneficiary term should be ‘(volunteer* OR helper* OR participant* OR beneficiar* OR recipient*)’; the final composite meso-beneficiary term should be ‘(group* OR organisation* OR (community OR communities))’; and the final composite macro-beneficiary term should be (societ* OR public OR national OR Scotland OR Scottish)’.
• Recommendation 5: The search field used with the composite beneficiary-related term should be ‘Topic’ to allow for greater sensitivity of searches where the volunteering-related composite term is restricted to ‘Title’.

Developing the micro-, meso-, and macro-specific ‘outcomes-related’ composite terms

Searches to test the micro-, meso-, and macro-specific ‘outcomes-related’ composite terms suggest that a manageable number of potentially relevant items are returned from searches which use all the individual search terms suggested in these categories to make three level-specific outcomes-related composite search terms. 

Additional searches for items identified in both searches using the general outcomes-related composite terms and the searches using micro-outcomes and meso-outcomes composite terms suggest a degree of overlap and duplication between search results. The number of unique results left for screening following removal of duplicates will therefore be lower than individual search totals suggest.

• Recommendation 6: The final composite specific outcomes-related terms should include all of the original individual terms. The final composite micro-outcomes term should be ‘(volunteer* OR helper* OR participant* OR beneficiar* OR recipient*)’; the final composite meso-outcomes term should be ‘(group* OR organisation* OR (community OR communities))’; and the final composite macro-outcomes term should be (societ* OR public OR national OR Scotland OR Scottish)’.
• Recommendation 7: The search field used with the composite specific outcomes-related term should be ‘Topic’ to allow for greater sensitivity of searches where the volunteering-related composite term is restricted to ‘Title’.

Comparison of ‘Web of Science – all databases’ and other bibliographic database services

There are a number of key differences between the four bibliographic database services that were used in this review. There are minor differences in the ways in which publication date limits are set across the services, but these are trivial.

One of the main issues relates to differences in the available search fields, with none of the other services using ‘Topic’. However, it is possible to construct an equivalent search using available search fields in ProQuest and EBSCOhost. ScienceDirect does not allow this, but has a composite search field of ‘Title, abstract and keywords’ that can be used instead.

EBSCOhost presents issues with overlap of searched databases (primarily MedLine), and with the inclusion of databases of news articles and other sources of ‘grey literature’. Test searches suggested that where these are left in, the numbers of potentially relevant items identified by the refined searches detailed above are infeasibly high. 

• Recommendation 8: To exclude specific databases and to limit the search to ‘Document type: academic journal’ to address this issue. 

Finally, ScienceDirect does not support wildcard searches, meaning that truncated terms used in searches on other services will need to be replaced by multiple relevant whole terms for this service. This will create longer search strings, which gives rise to a second issue, that ScienceDirect does not support the use of more than eight terms in a search field. The proposed solution would see the composite terms used in other services turned into multiple search terms where necessary, with all permutations of part terms searched to ensure equivalence of results.

Summary of recommended search strings and inclusion / exclusion criteria to be applied

The following general inclusion / exclusion criteria were applied:

• Exclude any items with a publication date before 1 January 2007
• Include only items where full text publication language is English
• For EBSCOhost – limit ‘Document type’ to ‘Academic Journal’

The following searches were carried out using each of the bibliographic database services:

• TI=((volunteer* NOT “healthy volunteers” NOT “human volunteers”) OR charit* OR unpaid OR unsalaried OR “civic engagement”) AND TS=(outcome* OR benefit* OR impact* OR improv* OR effect*)
• TI=((volunteer* NOT “healthy volunteers” NOT “human volunteers”) OR charit* OR unpaid OR unsalaried OR “civic engagement”) AND TS=(volunteer* OR helper* OR participant* OR beneficiar* OR recipient*)
• TI=((volunteer* NOT “healthy volunteers” NOT “human volunteers”) OR charit* OR unpaid OR unsalaried OR “civic engagement”) AND TS=(group* OR organisation* OR (community OR communities))
• TI=((volunteer* NOT “healthy volunteers” NOT “human volunteers”) OR charit* OR unpaid OR unsalaried OR “civic engagement”) AND TS=( societ* OR public OR national OR Scotland OR Scottish)
• TI=((volunteer* NOT “healthy volunteers” NOT “human volunteers”) OR charit* OR unpaid OR unsalaried OR “civic engagement”) AND TS=(employment OR education OR cohes* OR equit* OR ((social OR human OR cultural OR political) AND capital) OR resilience OR reputation* OR financial OR capacity)
• TI=((volunteer* NOT “healthy volunteers” NOT “human volunteers”) OR charit* OR unpaid OR unsalaried OR “civic engagement”) AND TS=(((mental OR psychological OR physical) AND health) OR training OR employ* OR career* OR ((social OR human OR cultural OR political) AND capital) OR functioning OR rehabilitation OR “quality of life”)
• TI=((volunteer* NOT “healthy volunteers” NOT “human volunteers”) OR charit* OR unpaid OR unsalaried OR “civic engagement”) AND TS=( collectiv* OR communitarian* OR cohes* OR equit* OR ((social OR human OR cultural OR political) AND capital) OR resilience)