Job Start Payment evaluation: annex B - qualitative research

2. Methodology

2.1 Overview

An approach was designed and ordered to ensure inclusion of a range of perspectives within the evaluation, utilising secondary and primary research:

• Step 1: Identify stakeholders for involvement in our discussion groups, and begin stakeholder engagement,
• Step 2: Analysis of administrative data to identify prospective explanations for low take-up and identify potential barriers,
• Step 3: Organise and hold discussion groups with relevant stakeholder representatives/practitioners,
• Step 4: Build a sample of clients through administrative data, and top this up with further recruitment via stakeholders identified and engaged in step 1,
• Step 5: Design a survey of young people in Scotland, administered through Young Scot, to gauge awareness and perceptions of JSP and for qualitative recruitment.
• Step 6: Conduct interviews with young people including successful applicants, unsuccessful applicants, and non-applicants,
• Step 7: Analysis and reporting.

2.2 Research methods relating to objectives

Research Aim One, and its sub-aims, were primarily to be met through active engagement with clients themselves. Semi-structured qualitative interviews with young people who have received JSP were designed to help understand the impact of the payment.

Stakeholder interviews were also seen as an opportunity to gather their more holistic overview of the payments' impacts. Given their expertise and the range of clients that they are likely to have interacted with, their insights were also seen as valuable in understanding more generalised trends and impacts.

Views from stakeholders were seen as valuable in identifying possible barriers to JSP take-up. Given that low take-up of JSP is of particular interest, and will be a key consideration for policymakers, efforts were made to hear from as many young people as possible about JSP. A survey was conducted, targeted at young people, to gauge the public's general awareness and understanding of JSP. This was therefore designed as a short survey, combining closed questions and a few open questions - delivering both quantitative and qualitative data. The survey's inclusion had the dual value of providing data and as a recruitment tool for follow-up qualitative research (particularly for individuals who have not applied for JSP).

The evidence reviewed in this report is almost entirely based upon primary qualitative research, including with young people and stakeholders. Data from the survey of young people is used primarily to give a sense of awareness around JSP.

High level information on applications and payments for JSP are included, taken from data published by Scottish Government.^[14] This data includes:

• Applications received- online, telephone and paper,
• Age breakdown of applications- 16-17, 18-21, 22-24, 25 years,
• Applications processed- including authorised, denied and withdrawn,
• Value of payments made.

2.3 Young people engagement

Recruitment of young people for the qualitative research was achieved through two methods:

• 1. A survey promoted by Young Scot included an opt in box for young people interested in taking part in follow up research.
• 2. Using administration data provided by Social Security Scotland, Scottish Government researchers invited young JSP applicants to take part in the study.

The first method was particularly useful for ensuring a broad range of young people were involved in the research. These included young people across Scotland who had not applied for JSP and young people who had not heard of JSP. Through their survey questions the research team could get a sense of their circumstances before making arrangement for them to take part. As this was not an application process, or a review process, it is not possible to establish whether any young people would have been or were definitely eligible for JSP. Having said that, they did fall within the age range of eligibility, and had taken up employment since JSP began taking applications.

The second method was based upon Social Security Scotland's records for JSP applicants. Therefore, this group had both knowledge of JSP, and experience applying. Scottish Government created a sub-sample of JSP applicants and sent them an email invitation to participate in this research. For those consenting, Diffley Partnership administered a short recruitment questionnaire. This gave further information to the research team to utilise in determining who to approach such as whether they had been successful or unsuccessful in their application, and information about their circumstances (See Appendix A).

In total, 39 young people took part in interviews between 15 November 2021 and 17 March 2022. Table 2.1 illustrates the profile of the 39 young people involved (See next page). The two young people we interviewed in the 16-17 age group were both non-applicants. This reflects the very low numbers of 16-17 year olds claiming JSP. Between August 2020 and March 2022, fifty-five 16-17 year olds were successful in claiming the benefit, indeed 79% of processed applications were denied for this age group.^[15]

Interviews lasted up to 45 minutes and took part via an online platform or telephone as per the participants' preference. In advance, discussion guides were prepared for the research team to follow (see Appendix B). All young people who participated were provided with a £30 Love2Shop e-gift card as a thank you for their time.^[16]

2.4 Stakeholder engagement

A diverse range of stakeholder representatives was recruited in terms of geographical spread, field, operations, and specialisms (see Table 2.2).

A few stakeholders, including those involved in welfare rights advice, declined to take part in interviews due to pressures involved with their organisations and their work commitments during the Covid pandemic. A couple of individuals approached declined on the basis that neither themselves nor their colleagues had first-hand knowledge or experience of JSP (please see section 4.5 for more discussion of levels of awareness and knowledge of professionals). However, stakeholders from charitable organisations working with Scottish Government's poverty priority groups were included and professionals working within employability services.

All stakeholder research took place online through participants' platform of preference. Research consisted of one-to-one interviews, double interviews with two colleagues together and discussion groups. Social Security Scotland staff took part in two separate discussion groups. This period of research took place between 8 November 2021 and 3 March 2022.

Discussion Guides were agreed in advance and used as a basis for these semi-structured interviews and discussion groups (see Appendix C).

Stakeholders drew upon their current job roles, and their previous work in related roles and the wider employability sector. For example, some individuals had previously worked for DWP, others had worked in front-line roles and now in more strategic roles, one individual had been out of work for over six months themselves prior to securing their current role. Crucially, stakeholder interviews included people who currently process JSP applications (and review appeals where applications are denied) from within Social Security Scotland. Also, those from external organisations who are encouraging colleagues, and directly signposting, referring or supporting young people to apply for JSP.