The Opportunities and Challenges of the Changing Public Services Landscape for the Third sector in Scotland: A Longitudinal Study (2009 - 2013): Year 4 and Final Report

Appendix A: Methodology

The methodology involved qualitative research within 21 third sector organisations based in Scotland. The methodology involved two key components: (1) in-depth case studies with eight third sector organisations and; (2) three focus groups involving 13 additional third sector organisations.

A qualitative longitudinal approach was used to ensure that the complex and fluid experiences of participants over a four-year period were reflected in the report. A qualitative approach ensured that interviewers could explore important and sensitive issues in depth with research participants. The importance of understanding and reporting changes over time within third sector organisations and how those changes are embedded in patterns of social, economic and political change has meant that a broad range of issues were explored during interviews between researchers and participants.

Case studies for Year 1 (baseline) were carried out between December 2009 and May 2010 and for the focus groups between April and June 2010. The results were reported in the Year 1 report.

Case studies for Year 2 were carried out approximately one year after the first visit with organisations (between January and June 2011). Focus groups were carried out at six monthly intervals following the baseline meeting, between October and November 2010 and April and May 2011. In addition, a workshop was carried out in June 2011 at Edinburgh University Business School, which is also used as evidence for Year 2.

Case studies for Year 3 were carried out between January and May 2012. Focus groups were carried out between October and November 2011 and between January and March 2012.

Case studies for Year 4 were carried out between January 2013 and May 2013. Focus groups were carried out between December 2012 and February 2013 and between March and July 2013.

Following discussions with the Scottish Government and the Research Advisory Group, a framework was developed for the selection of research participants. This was designed to ensure the establishment of a purposive sample of organisations working in different:

• policy areas (with a mix of social care, healthcare, and employability/economic development/regeneration providers)
• geographies (based in different locations across Scotland)
• scales (with a mix of larger and smaller organisations included)
• and to include some social enterprises

The selection of focus groups was based on similar lines with individual focus groups bringing together organisations with strong agendas in the following areas: (a) equalities; (b) social care and health care, and (c) employability/economic development/regeneration.

Potential participants were identified through a database of 685 possible organisations provided through the Scottish Council for Voluntary Organisations (SCVO). The final selection of possible organisations was made in order to achieve the balance required by the framework (above). All organisations were then contacted and invited to take part in the research for a period of three years as either: (1) a case study carried out once a year; or (2) to participate in a focus group carried out twice a year. Most first choice organisations were happy to participate, with the few who declined being replaced by other suitable organisations. In this way, the baseline sample of organisations was obtained.

The case studies included face-to-face interviews with staff at different levels of the organisation. These included: Chief Executives; other senior officers/managers; research/policy officers; business/planning managers; operational and line managers; front line staff delivering services. The selection of staff for interview was decided in consultation with the main contact from the organisation (usually the Chief Executive or another member of the senior management team) and actual staff interviewed varied depending on the size of the organisation and availability of appropriate functions. A list of interviews carried out within each organisation for each year can be found in Appendix C. A copy of the main interview schedule used in Year 4 is attached in Appendix B.

In Years 2, 3 and 4 the main contact in each organisation was approached and asked for an interview and permission to follow up staff contacted in the previous year/or staff were approached directly. If original staff were not available for interview, where possible, other staff covering a similar role were identified and interviewed. A very small number of staff either did not reply to invitations to be interviewed, or there were difficulties in arranging available interview dates within the research timeframe.

13 organisations were divided into three focus groups. Each focus group pulled together organisations with strong interests in particular areas. These were: (a) equalities; (b) social care and health care, and (c) employability/economic development/regeneration (Note that these categorisations were not applied rigidly and there was some overlap in the activities of organisations).

One representative from each organisation (usually the Chief Executive or a member of the senior management team) attended the focus groups. Where an organisational representative was unable to attend the focus group, telephone interviews were conducted. A common discussion framework was used - see Appendix B.

In order to protect the anonymity of individual respondents who took part in the research, quotes have been labelled with generic job titles (e.g. Senior Manager, Manager, and Officer). A brief description of the type of organisation is also provided after each quote. Additional background information on the participating organisations is provided in Appendix D. This is intended to give context to the overall report and individual quotes without revealing the identity of participating organisations. All organisations were happy to be identified as taking part in the research (although not necessarily to have particular opinions credited to them). All participating organisations approved the approach to anonymisation that has been used.

Data from each year was analysed using the Framework analysis method. This is a 'matrix-based method for ordering and summarising data' (Lewis, 2007: 550). Key themes from findings are identified and divided into sub-topics. Matrices are then drawn up in Microsoft Excel, each representing a different sub-topic.

The first stage of the qualitative longitudinal research was to apply the Framework and identify significant topics and sub-topics. This provided a flexible common core framework that enabled 'comparability over time and between projects' for which 'the use of common data collection tools and reproducible modes of analysis are suggested' (Holland, 2007). Thematic analysis has been carried out on individual focus group and case study data collected in each year. This analysis provided the basis for the longitudinal analysis as well as providing more detailed cross-sectional data, including quotes, which are used in the report.

For the longitudinal analysis, later data were then added to the earlier data within the Framework enabling accounts provided by different respondents at different points in times to be compared.

Individual focus groups and case studies were subjected to a longitudinal analysis comparing data collected in the waves. This was done for each group/case study in Microsoft Excel. This data was then integrated (in Microsoft Excel) across all groups and case studies. In this way, a summary analysis of key changes between the waves was enabled and forms the basis of this report.

The longitudinal qualitative nature of the research presents an unparalleled opportunity to track the dynamic of change over time. However, this also presents challenges including issues of attribution, policy changes and attrition.

Attribution is being able to attribute changes to a specific cause. However, third sector organisations in Scotland operate within a complex and changing policy context, with policy emerging from different levels. The UK and Scotland level are particularly important for policy, but also policy emerging from (and interpreted through) local authorities, regulatory bodies and Europe form part of a complex background. There is also often a time lag between policy announcements and the actual impact on third sector organisations since these can be mediated via other bodies (e.g. local authorities in particular). Where possible the report identifies the key links, but it would be too simplistic to assume that causal links always exist or that they are straightforward in nature.

Over the course of a number of years, it is likely that the circumstances of organisations or individuals may change in a way that means they can no longer continue in the research (attrition). Fortunately, attrition of organisations has been minimal.

There was some attrition of interviewees and organisations. This was because of staff leaving the organisation; being unable to set up interviews within the timeframe or because access was restricted by the key contact; or organisational mergers. Where possible, alternative participants were interviewed.