Transitional employment services - phase 2: evaluation

Appendix 2: Survey Methodology & Demographics

Wave 2 telephone survey

Current and former customers of the WFS and WAS services completed a telephone interview between 26^th November 2018 and 31^st January 2019, about their experience of the service, its outcomes and their current situation. All customers initially signed up to the services between 1^st April and 31^st October 2017.

This survey was wave 2 of the research, wave 1 used the same quantitative methodology and took place in January and February 2018. Therefore, this follow up took place just under 12 months after the original wave 1 survey.

A sample of 2,788 customers (n=2,024 WFS, n=764 WAS), with valid contact details, was originally provided directly to IFF Research for wave 1 of the survey, of which 2,000 (n=1,458 WFS, n=452 WAS) were drawn to be contacted to take part in this initial stage of the research. A total of 731 interviews were achieved in wave 1, and of these 665 agreed to be contacted around 12 months later to take part in wave 2 of the research.

The starting sample for wave 2 of the research then comprised of 665 'longitudinal' customers, who took part in wave 1 and 638 pieces of 'fresh' sample drawn from the remaining original sample received for wave 1 but not drawn in the original 2,000 sample. A total of 1,303 customers were sent a letter in advance letting them know about the research and providing the opportunity to opt-out if desired and then contacted from the beginning of fieldwork. It is important to note that all of the customers contacted for both waves of the research had started the service in the same time period.

As fieldwork progressed, further sample was loaded, 781 customers who were approached in wave 1 but did not complete the survey were added to the call list as 'top-up'. For the purposes of this research, these are also classed as 'fresh' sample, as they did not take part in wave 1. A total of 2,084 customers were invited to take part in wave 2 of this research.

A total of 591 surveys were completed, split as: n=417 for WFS and n=174 for WAS. Therefore giving a response rate of 28% when including all 2,084 customers, though it's important to flag that many of the 781 'top-up' customers were only contacted towards the very end of fieldwork. The response rate was 45% when considering the original starting sample of 1,303 customers. The survey lasted around 20 minutes.

Customer case studies

Case study interviews were drawn from customer contacts from Providers who joined the services between 1 December 2017 and 31 January 2018. One group were to have left support to move into work so that these interviews could explore in more detail the support provided to customers pre and post-employment and any further issues they faced in sustaining employment in the period since they left employment and, where relevant, how they overcome these issues.

Discussions with Providers highlighted that it would also be possible to contact a smaller group of individuals who attended a first meeting with the provider and signed-up to the service (including their data sharing agreement) but subsequently did not attend any further interviews. One of the objectives of the research has been to better understand the reasons behind potential customers' decision not to participate and so these interviews were included.

The intention was to split the fieldwork with ten case studies from each group but in practice these interviews were significantly more challenging to undertake with contacts having a number of health issues that made telephone interviews difficult to sustain.

Employer case studies

A key finding in Phase 1 of the evaluation was that the vast majority of customers of both WFS and WAS do not want their Providers to contact their employers. This means that there were fewer opportunities for Providers to engage with employers and provide the type of close support that had been envisaged in designing the services in-work support specification.

Case studies with employers in Phase 2 have been undertaken with 17 employers. WFS and WAS service monitoring data does not capture full employer contact details. So contact details were sourced from WFS/WAS Providers on the basis that employers had recruited WFS or WAS customers who had not asked the provider not to contact their employer. Most Providers contacted employers on the list in advance of the fieldwork to ensure that they were happy to participate. We believe that these employers are among those who have closer working relationships with Providers and so the results of the discussions should be read with this in mind.

We have not presented the results of these discussions separately for WFS and WAS. Partly because there are too few case studies to make this credible but mainly because the comments from employers who we are aware worked with one service or another are very similar.

A key finding from discussions with employers in Phase 1 of the evaluation was that most did not recognise the names of the services nor the nature of support provided to individuals except in the most general of terms. They had, however, established relationships with Providers and more often with individual employment liaison staff within Providers.

These relationships were enduring, often spanning different generations of employment support services and different provider organisations with employers' valuing the understanding of their recruitment needs above all. Given that many are involved in regular rounds of recruitment, they value highly any support that will provide them with willing candidates with relevant skills. One employer who had been interviewed in Phase 1 of the evaluation had not continued with their relationship with a provider primarily as the lead contact at the employer had left the business.

The case study discussions with employers in Phase 2 of the evaluation re-inforce these findings. We did not find any employer who was able to name the service that was being run by their provider let alone its key features. Most knew the name of their employment liaison contact and a majority the name of the provider. They were aware that customers of the provider have a disability or health condition.

Employers were not sampled but drawn from provider contacts according to those who had a sufficiently close working relationship with Providers to be able to comment on their support. Providers were asked to remind employers that we were not aiming to discuss the performance of specific individuals whom they may have recruited from the services but on their experience in general. This tended to mean that they had recruited more than one customer. Key characteristics of the employers are:

• A range of sizes from fewer than 10 employees to national organisations employing more than 250.
• A number of sectors, although it is perhaps not surprising that these mostly represent lower-paying industries, often associated with flexible working arrangements – cleaning, hospitality, security and retail.
• Working in these sectors offers some degree of progression, but this can be limited to improving the security of employment – from zero hours to temporary/seasonal contracts before securing full-time contracts. Wage rates are at or just above minimum wage levels but some pay at or close to the Living Wage.
• It is possible to progress to supervisory positions but more often employers are fully aware that the better employees may leave in order to secure improved terms and conditions. As a result, many of these employers are involved in what appears to be a constant round of recruitment.
• Three of the case studies were with organisations who were explicitly working towards progressing recruits into other employment opportunities – a social enterprise (cleaning services), a chain of national charity shops offering volunteering opportunities and a partnership of Housing Associations, care Providers and property management companies who recruit people that are struggling to secure employment in the open market on 12 month contracts paying the Living Wage while they train and gain experience in a variety of roles across their sector.
• Five employers were previously interviewed in Phase 1 of the evaluation so that we could explore whether their experience and perspective had changed over time.

'Non-user' research

Both WFS and WAS operated as voluntary services. A key finding in phase 1 of the evaluation was that those who agreed to refer and start the services are a relatively small proportion of the total number who might benefit from such support. Actual starts on both services were around 60% of all referrals from JCP and not all those who may have been eligible agreed to be referred (for whom there are no data).

A key question for both services, therefore, has been why do people who are in apparently similar circumstances to those who have engaged with the services decide not to participate and what might be done to better engage with this group?

Undertaking research with this wider group is not straightforward. Access to the contact details of service customers is only available for those who signed the release forms during their induction interview. Therefore, there is no information on the characteristics nor reasons for deciding not to participate.

In Phase 1 of the evaluation we engaged with a range of patient groups and networks to explore the nature of their client groups and whether it would be possible to reach out to their members and explore whether they would be willing to participate in the research. Very few of these organisations were comfortable with this approach and so this approach was abandoned.

In Phase 2 we have trialled a different approach working with two specialist Providers who were part of the WFS and WAS supply chain in a number of locations but also offer a wider range of services, particularly for clients with mental health issues. This service structure meant that both organisations were able to approach their existing clients who they felt would be willing and able to participate in the fieldwork but conformed to a number of criteria:

• Be of working age 18-65
• Having not participated in either WAS or WFS services in the past but may well have been offered such support as they would be claiming ESA or Universal Credit benefits.
• The nature and purpose of the research was explained to all participants prior to the events and were told that they would receive a £25 shopping voucher in recognition of the time and effort in attending the event.
• Intention was to undertake either individual one-to-one interviews or group discussions depending on the views of participants. In the event, the overwhelming preference was for group discussions.

From the outset we agreed that the research would not seek to capture personal details and background of participants. Participants were asked which age band they fell into but otherwise no other personal information was recorded on the group. This approach meant that we were able to maintain individuals' anonymity and ensure that people were comfortable to participate

Four group discussions were undertaken at the organisations premises in Glasgow, Edinburgh, Falkirk and Kirkcaldy. Each lasted for 1¼ to 1½ hours and covered the following issues:

• To better understand where work fits in their lives and aspirations for their future employment
• Their views on their health situation and how this affects their decisions on work – the types of jobs that they believe they can do and stay healthy
• To explore their level of interest in moving into work at some point in the future, what might have to happen for them to feel more comfortable about taking this step and the type and level of support they would want to receive
• To better understand any barriers to work they have and possible routes to overcoming these
• Can work help them overcome their health problems

Two groups were attended by people who were 25 or older. The other groups were predominately younger aged 18-24 but with some older (25-55) attendees who were recently homeless but now living in accommodation in the community project.

A key part of obtaining the collaboration of the two community organisations was that we would not seek to catalogue participants' specific health conditions nor probe on their personal histories. It was clear from the general discussions and the nature of the services offered by the community organisations that participants suffer now or in the past from a range of mental health conditions with others also reporting some physical conditions. Mental health issues is the most frequently reported health condition for customers of both WFS (33%) and particularly so for WAS (68%)^[15]. These groups may have had proportionately higher prevalence of mental health conditions, there are reasons to suspect that this may simply be a reflection of the more significant needs of a group who have yet to engage with either service.

Customer Demographics

This section considers the extent to which customers of both services reflect what we know about the population of people with disabilities and long-term health conditions.

Table A.1 Demographic profile of sample WFS clients April – Oct 2017

Gender	WFS customer survey profile (417) %	MIS data
Male	57%	64%
Female	43%	36%
Age
16 to 24	11%	18%
25 to 34	19%	22%
35 to 44	21%
45 to 54	25%
55 to 64	24%
65 plus	<1%	0.4%
Ethnicity
White British	94%	93%
Other	5%	4%
Refused / Prefer not to say	1%	4%
Level of education
National 1 or 2	4%	n/a
National 3	9%	n/a
National 4 or 5	22%	n/a
Highers/SVQ3	8%	n/a
Advanced higher or equivalent	9%	n/a
Degree or above	19%	n/a
Other professional, technical or management qualification	4%	n/a
Other	1%	n/a
None of the above	17%	n/a
Don't know / Prefer not to say
Health conditions and disabilities	Total / Primary	Total
Mental health condition	39%/22%	33%
Long-term illness, disease or condition	40%/24%	17%
Physical disability	28%/13%	17%
Learning difficulty	22%/10%	10%
Deafness or partial hearing loss	11%/3%	4%
Learning disability	8%/2%	3%
Blindness or partial sight loss	9%/4%	3%
Developmental disorder	5%/2%	3%
No condition	12%/12%	-

Table A.2 Demographic profile of sample WAS clients April – Oct 2017

Gender	WAS customer survey profile (417) %	MIS data
Male	67%	56%
Female	32%	44%
Age
16 to 24	5%	11%
25 to 34	19%	23%
35 to 44	25%	22%
45 to 54	26%	27%
55 to 64	25%	16%
65 plus	1%	-
Ethnicity
White British	98%	98%
Other	1%	2%
Refused / Prefer not to say	1%	0.2%
Level of education
National 1 or 2	5%	n/a
National 3	11%	n/a
National 4 or 5	16%	n/a
Highers/SVQ3	12%	n/a
Advanced higher or equivalent	11%	n/a
Degree or above	14%	n/a
Other professional, technical or management qualification	6%	n/a
Other	1%	n/a
None of the above	18%	n/a
Don't know / Prefer not to say	6%	n/a
Health conditions and disabilities	Total / Primary	Total*
Mental health condition	69%/40%	68%
Long-term illness, disease or condition	53%/20%	36%
Physical disability	36%/15%	16%
Learning difficulty	21%/3%	12%
Deafness or partial hearing loss	11%/0%	4%
Learning disability	21%/1%	n/a
Blindness or partial sight loss	8%/2%	3%
Developmental disorder	10%/4%	8%
No condition	2%/2%	-

Source: IFF Research telephone survey of 174 WAS customers (unweighted profile) and SDS CTS data from 1 November 2018. Data on ethnicity and qualifications are not available.

* N.B Classification of conditions do not match precisely to MIS categories: Social and communication disorders are included as Developmental disorders above and Other disability, impairment or medical condition is combined with Long-standing illness and included as Long-term illness, disease or condition.