The Impact of Welfare Reform in Scotland - Tracking Study - Year 1 Report

3 Methodology

• The study utilises a qualitative longitudinal approach in order to best track participants' experiences over time, as the welfare changes are introduced.
• All participants interviewed are currently in receipt of working age welfare benefits, and were selected using a purposive sampling strategy which was designed to reach those in receipt of benefits from across Scotland, and cover a diverse set of household circumstances.
• In-depth, semi-structured interviews were carried out with 43 participants in Sweep 1 and 35 participants in Sweep 2.
• The data was transcribed and analysed using the qualitative analysis computer software NVivo.

3.1. The study takes a qualitative longitudinal approach, which involves interviewing participants six times over three years to 2016, with the first two sweeps of interviews being started in September 2013 and April 2014.

3.2. Longitudinal qualitative research is particularly useful for exploring the impacts of welfare reform. By returning to participants, we are able to give them the opportunity to reflect on any changes in circumstances that might have occurred since their last interview. It also allows the researcher to tailor individual interviews in order to find out more about specific sets of circumstances (Farrall, 1996). This is especially useful when interviewing people who are in receipt of benefits for a number of different reasons such as disability or ill health, and/or because they are a jobseeker and/or lone parent.

3.3. The longitudinal nature of the study can also capture the experiences of participants at different stages of welfare reform as it is rolled out, and to identify other changes, including those in their labour market status and how these have affected them. Examining change over time can also help to distinguish between long- and short-term impacts, as the effect of a change or event may vary over time.

Typical profile of participants

3.4. Participants had a variety of different reasons for claiming benefits. Although the exact combination and amount received varied according to each household's individual circumstances, there are nonetheless a number of typical benefit combinations, and most participants fell into one of these categories.

• Those unable to work due to a health condition or disability. Most claiming DLA; some claiming ESA, and also receiving HB and CTR; others reliant on income from other household members and/or occupational pensions. The sample also included participants who had a health condition or disability but were in work, or claiming JSA.
• Lone parents. Those whose youngest child was under five years old typically claiming IS, HB and CTR, plus CB and CTC for their child(ren). Those whose youngest child had turned five years old, who were not in work, were typically in receipt of JSA as their wage replacement benefit, alongside HB and CTR, and CB and CTC for their children. Those in work in receipt of WTC, CTC, and HB and CTR depending on income.
• Single people and couples (with or without children) looking for work and claiming JSA, either contribution or income-based depending on their working history and the income of any partner present in the household. Some JSA recipients also entitled to HB and CTR.
• Full-time carers in receipt of Carer's Allowance (CA), which was deducted from any other income-related support received, such as IS, HB and CTR. The household income may also include DLA for the person they care for.

Selecting and recruiting the sample

3.5. A purposive sampling strategy was used. The main criterion for inclusion in the sample was that the participant was of working age, and in receipt of at least one of the benefits subject to reform, such as JSA, ESA, DLA, IS, HB, LHA, WTC or CTC.

3.6. Within this, there is considerable scope for variation; for example, households affected by these changes may or may not have children, could be couple or single adult households, and be in or out of work. A sample framework of minimum estimated participant numbers for a range of different household characteristics guided recruitment. Data from the Labour Force Survey (Q1 2013) about the percentage of people in Scotland on JSA, ESA, IS and DLA provided guidance in the drafting of the sample framework. The sampling also sought to represent spatial (e.g. rural-urban) factors that could potentially affect the outcomes of interest (see Table 3.1 for details of the sample characteristics, both the target sample size and the actual sample size). However, it should be acknowledged that this sample is not meant to be representative. This means that there are limitations in the conclusions that this study can provide in terms of the experiences of some groups of participants (e.g. lone parents, those living in rural areas). However, the study provides valuable insights into the experiences of those in receipt of benefits and highlights some of the issues faced by specific groups which could be followed up in more depth in other research.

3.7. Recruitment of participants primarily took place through gatekeeper organisations (i.e. approaching third sector and public agencies who deal with the relevant groups); additional recruitment also took place through snowball sampling (where participants recommended potential participants for a study). This strategy was used as it not only made it easier to recruit suitable participants, but also to keep in touch with them for the follow up interviews.

3.8. After this initial recruitment the sample was re-contacted prior to the Sweep 2 of fieldwork which began in April 2014. Maintaining the interest of the sample has been treated as an on-going process, rather than a one-off event. To help minimise sample attrition there was on-going contact, via the distribution of a newsletter to let participants and gatekeepers know about the publication of study reports. In order to reduce the risk of participants refusing to take part in subsequent interviews, the same researcher has been used where possible to conduct follow up interviews in order to provide consistency for participants.

3.9. There were 43 participants in Sweep 1 and 35 participants in Sweep 2. The target sample size at the end of the study is 30 households. However, the initial sample needed to be larger than this to allow for some attrition across the three years, as participants may be unwilling or unable to participate in future sweeps, or the research team may lose contact with them. Of the 43 participants interviewed in Sweep 1, eight did not participate in Sweep 2. Of these eight: two were not followed up because they did not meet the sample inclusion criteria; two were unable to participate in Sweep 2 but it is hoped that they will return for subsequent sweeps; and the remaining four could not be contacted.

Table 3.1: Overview of sample characteristics

Household characteristic	Requirements for diverse sample	Target Minimum Sample Framework for Sweep 1	Actual Sample Sweep 1	Actual Sample Sweep 2
Children	with dependent children under the age of five years	3	5	2
	with dependent children over the age of five years	3	16	13
	with two or fewer dependent children	4	16	11
	with more than two dependent children	2	3	3
	without dependent children	15	24	21
	lone parent households	5	10	7
	where both parents/carers present	6	9	7
Employment	where members are employed full-time	2	2	3
	where members are employed part-time	4	2	4
	where some members are employed and others unemployed	4	6	6
	where all adults are unemployed	4	33	22
Protected characteristics	households with disabled adults	10	27	24
	households with disabled children	2	3	2
	household with both men and women	10	19	16
	households with working age adults of different ages	10	19	16
	households with ethnic minority adults	2	1	1
Location	rural areas	6-8	6	6
	urban areas (but not cities)	15-16	18	15
	cities	16	19	14
Gender	Male		17	15
	Female		26	20
Total sample			43	35

Note: Overlapping categories mean that totals within categories may not sum to total sample.

Data collection

3.10. In both Sweeps 1 and 2, in-depth, semi-structured interviews were carried out with participants. Interviews in Sweep 1 were longer (between 30 and 90 minutes) than those in Sweep 2 (between 25 and 45 minutes), as they were used to gather baseline information. In Sweep 2, the questions centred on the changes since the last interview and, with the background information collected in Sweep 1, could be more tailored to participants' circumstances, focussing on the areas most relevant to them.

3.11. In-depth interviews were chosen as they are particularly suited to gathering data on individuals' personal histories, perspectives, and experiences. The interviews are semi-structured, with a brief topic guide that gives a clear idea of the issues that should be covered, but with most questions focusing on open responses and the opportunity for the participant to raise other issues. This method allows participants to give rich, personal and in-depth accounts of their experiences, and also allowed the researchers to build a rapport with participants (this was especially important in helping to minimise sample attrition between sweeps). Interviews were conducted in a private setting in which participants felt comfortable, such as in their own home, or in a more neutral setting such as an advocacy organisation's offices or a café.

3.12. Participants were given an information sheet before participating in the study in Sweep 1 (see Appendix 6). Interviewers re-outlined this information prior to the Sweep 2 interviews and answered any questions that the participant had. Full consent was obtained before proceeding with both the Sweep 1 and 2 interviews (see Appendix 5 for the consent form). Interviews were audio recorded where permission was given, and partially transcribed (i.e. relevant content from interviews, such as the households' accounts of their experiences, but not incidental conversation or 'warm up' questions).

3.13. The content of the baseline interview covered a number of areas (see Appendix 3 for the interview schedule):

• Financial information - level and sources of current income (including some cross-checking against expected entitlements), household costs and expenditure, budgeting and debt;
• Current employment and training/education;
• Sources of support - the use of services and formal support (and any barriers to accessing these), and informal support networks including family and friends;
• Household members' self-reported physical, mental and emotional health, and any use of healthcare services;
• Challenges people are currently facing, how they are coping and what is helping or hindering them;
• Other areas for discussion on welfare changes identified by participants, including participants' perspectives on welfare changes as a whole; and
• Retrospective evidence on income sources and employment over the previous two years.

3.14. Sweep 2 interviews sought to establish any changes to the participant's financial or employment situation, or their wellbeing. It also contained a set of questions asking for more in-depth details of their labour market experiences than covered previously (see Appendix 4 for the interview schedule), such as:

• Had the participant been involved in any training or education, or achieved any new qualifications, since the last interview?
• Was the participant currently employed, and was this a new job since the previous interview?
• For those who were currently employed: had there been any changes in their job since the previous interview; had the participant been offered any job related training opportunities; and had the participant been offered any promotions or looked for other job opportunities?
• For those who were currently employed, did they face any constraints that affected their ability to keep working?
• For those who were currently unemployed, were they seeking work and how confident were they that they would be able to find work? Did these participants face any constraints moving into work?
• Did those who were currently unemployed think that they would be better off if they were working?

3.15. No payment for time provided by participants was given. However, participants were given a voucher to compensate for travel, and any other expenses (including childcare), at a rate of £10 per household per meeting.

Analysis

3.16. The content of the interviews was analysed (including using qualitative analysis computer software NVivo)^[12] for important and/or recurring themes.

3.17. Although the sample in this research is small and qualitative, and generalisations that can be made from the findings are limited, it is still useful to observe whether patterns emerge in participants' experiences, not least to suggest avenues for future, larger scale research. However, this study also seeks to preserve narratives rather than reduce them to constituent parts, in order to understand people's individual experiences and the impact that policies will have on them.

3.18. This study also allows for the identification of key features of a participant's life as measured by established instruments (such as those used in national surveys to classify their type of accommodation or to measure of wellbeing), and to link these elements together, and situate the participant in their own unique context.

Research Ethics

3.19. This study received research ethics approval from Edinburgh Napier Business School's Research Integrity Committee.