Evaluation of Scottish transitional employment services: interim report August 2018

Executive Summary

Introduction

In September 2017 Cambridge Policy Consultants and IFF Research were commissioned by the Scottish Government Fair Work Employability and Skills Directorate, along with delivery partner Skills Development Scotland ( SDS), to undertake an independent process and outcome evaluation of the Work First Scotland ( WFS) and Work Able Scotland ( WAS) transitional employment support services.

The aim of the evaluation is to provide a robust, independent evaluation of the delivery process and outcomes of both WFS and WAS. More specifically, the evaluation focuses on the following research questions:

• How well has the service delivery process worked across both services?
• What do high quality services look and feel like for customers?
• What difference does the service make to customer outcomes?
• What difference does the service make to employers?
• How are these services ( WFS and WAS) different from previous employability support?
• There is no specific economic evaluation of WFS and WAS as this was not felt to be proportionate for a transitional service model.

Findings from the evaluation will be reported in two phases. This first 'interim' report focuses on how programme design has impacted on the quality of delivery and customers' experience of services in the first 6 months after launch. Monitoring data from both WFS and WAS programmes covered the period from 1 April 2017 to 31 October 2017. This was sufficient to draw a participant sample but too early to consider the impact of services on longer term employment outcomes. A second 'final' report will explore customers' experiences and outcomes across both services in more detail, and is due to be completed by the end of March 2019.

In addition to a large telephone survey of 700 WFS and WAS customers, we have undertaken consultations with providers, delivery organisations in their supply chain, frontline staff (including Work Coaches in Jobcentre Plus), case studies with a small number of employers, and key stakeholder interviews in the Scottish Government ( SG), Jobcentre Plus ( JCP)/Department of Work and Pensions ( DWP) and Skills Development Scotland ( SDS). The fieldwork was undertaken between December 2017 and March 2018.

Background

Powers over certain employability services devolved to Scotland on 1 April 2017 under the Scotland Act 2016. The Scotland Act devolves responsibility for client groups previously served by the Department for Work and Pension's Work Programme and Work Choice schemes. The Scotland Act 2016 provides the opportunity to develop a distinctively Scottish voluntary approach to employment support for disabled people and those who are at risk of long-term unemployment due to a health condition. This includes removing the risk of benefit sanctions for non-participation, a strong focus on those who need specialist support and a flexible 'whole person' approach with a greater role for customer choice.

Scottish Government Transitional Employment Services commenced on 3 April 2017 and originally planned to accept referrals to December 2017. This was subsequently extended to 9 March 2018 and delivery will remain in place until 30 April 2019. Fair Start Scotland, the Scottish Government's fully devolved employment support service launched on April 2018. The transitional services comprise:

• The Work First Scotland ( WFS) Programme - a voluntary programme of employment support for those with a disability. WFS offers 12 months of support to disabled customers split equally between up to six months pre-employment support and six months of in-work support. for up to 3,300 clients. The programme is delivered by three contracted service providers; Momentum, Remploy and The Shaw Trust across four Contract Package Areas ( CPAs).
• The Work Able Scotland ( WAS) Programme - a voluntary programme of employment support for those with a health condition claiming Employment Support Allowance. WAS support is for 12 months and offers pre-employment support and in-work support, as required for up to 1,500 customers. The programme is delivered by three contracted service providers; Progress Scotland, The Wise Group and Remploy with a single provider in each of the four CPAs.

Both programmes required providers to offer customers at least one hour a week face-to-face contact time to help them address their barriers to employment, and to secure specialist support as required using an action planning approach. This is a relatively novel approach in contracting employability services to specify such delivery standards, with previous UK national practice tending to favour a 'black box' model, which leaves the details of delivery to providers' judgement. This evaluation therefore provides a useful first opportunity to explore the process of service delivery across both programmes in more detail.

There is a growing body of evidence in the research literature to suggest that to best support those with disability and/or health conditions, a more intensive approach, tailored to individual needs is required:

• This customer group is considerable and growing, particularly in deprived areas. Their needs are becoming more complex, especially for older age groups who more often have multiple health conditions
• Trusted, intensive support for employability linked to specialist services does make a difference to employment outcomes
• Evidence is mixed but suggests longer-term support may be required to help sustain participation in employment
• The quality of employment may also play a role in supporting improved quality of life for people living with health conditions
• Early intervention is key to supporting those who acquire health conditions while they are in work to prevent them leaving the labour force

How well has the service delivery process worked across both services?

Both programmes have engaged with a customer group that is broadly representative of their respective target client groups with the exception that:

• When considering the gender split in self-reported disabilities and health conditions, female customers are slightly underrepresented across both services
• The customer groups are on average drawn from younger age groups, when the incidence of disabilities and health conditions rises steeply with age. That said, WAS participation up to the age of 55 is close to the profile of older people and this should be considered an achievement.
• Two-thirds of WAS customers have mental health conditions, well above benchmark levels and 86% reported that their condition had an impact on their ability to carry out day-to-day activities. Both the Management Information System ( MIS) data and the survey of WAS customers suggest that on average they are likely to report having considerably higher needs than WFS customers.

The Scottish Government decision to work with existing Work Choice providers to ensure continuity of support in the transitional WFS approach, meant that it was possible to build on existing provision and provider delivery arrangements. This brought significant advantages in the time available. WAS did establish provision very quickly, but this inevitably required some time to bed-in and this happened as the programme went live.

The referral process for WFS was able to build on the then existing experience and personal connections between JCP Work Coaches ( WCs) and provider staff. This meant that providers were more often in JCP offices and were quicker to establish a "warm handover" ^[1] referral process where providers could set out and discuss the WFS 'offer' and whether it interested the customer. This was not implemented universally and the overall referral numbers reflect considerable variation at a local level.

It is important to recognise that relatively high proportions of referrals either not attending initial interviews or attending and then deciding not to participate, focused provider behaviour towards managing and improving this process.

WAS providers had to establish a presence with local JCP offices from a standing start and could not draw on previous JCP WC experience of the client group. WAS customers proved challenging to engage as potential customers were less frequent visitors to JCP offices and proportionately more felt that they were unlikely to secure work because of their health condition. JCP WCs were less familiar with the offer and often felt less able to 'sell' the programme, and the clerical referral process also added time and resources for providers.

Providers and some JCP WCs see warm handovers as an important part of ensuring prospective customers are fully aware of the programme offer and its requirements on them. This is about explaining how the support provided can help customers access employment and manage their health, giving examples of similar customers who have worked with them successfully previously .

Providers reported that day to day contact with JCP WCs was often challenging. This was a core requirement of the referral process and many providers suggested that having a single point of contact to share customer information would be beneficial.

The regular one-to-one appointments with customers were a fundamental element of the support within both programmes, and were valued by providers and highly rated by customers, with 80% of customers surveyed across both programmes reporting the frequency as right for them. Compared to other employability programmes, the compliance processes were not that different. But some providers noted that weekly meetings with clients had increased the amount of time spent on administration. Providers consistently reported that the administration of both programmes absorbed around 30% of their frontline adviser time, and therefore may at times have reduced time available for customers.

Establishing two employability programmes in such a short timeframe has been a challenge for all concerned. But The Scottish Government concluded there was an overarching need to ensure continuity of support for unemployed people, while also developing the space for a distinctive Scottish approach to employability support. The WFS and WAS programmes have enabled The Scottish Government to develop the Fair Start Scotland programme launched in April 2018, and supporting infrastructure that includes their own performance management team, IT development, data monitoring and effective strategic and operational communications with JCP and DWP. This paid dividends as the lessons learned from WAS and WFS have helped in the design and delivery of Fair Start Scotland, and will continue to do so in future iterations of employability services in Scotland.

Partnership working and communications between SG and JCP/ DWP have been transformed. Partners now have an effective platform to raise and address policy and delivery issues, such as information and data sharing and a joint approach to improving operational delivery. This was evidenced by the early and extensive preparations for the introduction of Fair Start Scotland in April 2018.

What do high quality services look and feel like for customers?

The main reason that WFS and WAS customers gave for engaging with WFS/ WAS was their strong desire to secure work, which was expressed equally by both customer groups – 44% of WFS and WAS engaged with their respective programme because they felt it could help them back to work, and 29% of WFS and 26% of WAS said that they really wanted a job.

Proportionately more WAS customers reported that they were attracted by the offer of additional support tailored to their individual needs, specialist help for people with disabilities or health conditions and that they thought the programme would build their confidence.

Voluntary participation is widely regarded by providers and some JCP WCs as an important feature of both services that helped to engage customers. However, it should be recognised that this is set within a wider DWP benefit system that still relies on compulsion in key aspects. Voluntary participation represents a significant cultural shift, particularly for customers in receipt of benefits, and it will take time for some customers to fully appreciate that participation is entirely their choice. At the time of our first survey, some 13% of WFS and 6% of WAS customers thought that their participation was mandatory.

Almost seven in ten WFS and over eight in ten WAS customers found the regular one-to-one adviser appointments useful. The vast majority of customers on both programmes report that their frequency was about right (84% WFS and 85% WAS) with only a small minority saying that they were too frequent (6% WFS and 5% WAS).

Frontline advisers were very supportive of face to face participant contact criteria and felt it provided a platform to support the customer and make sufficient progress in building trust and confidence. Some lead providers felt that this frequency did not always suit the needs of all clients. However, provider and supply chain frontline advisers were most often in favour of this approach. Regular contact based around an action planning process was considered to be very important for customers who were often not in a position to progress their confidence and 'back to work' strategies on their own.

Occupational therapy and other health and wellbeing support was offered to proportionately more WAS customers (61%) than WFS (54%) reflecting their relative needs. More than a fifth of both groups (24% of WFS and 22% of WAS customers) offered such support did not take it up, but the vast majority that did use the support found it useful. Discussions with providers suggest that they faced cost-constraints on such specialist provision, and in most cases support was relatively short courses designed to provide customers with strategies to help manage their conditions.

Extended support and those with higher level needs were referred to NHS or other local specialist services, but providers noted that access to these services was not always equally available in all parts of Scotland, and more could be done to align employability and health support. Healthy lifestyle support – short-term gym memberships and healthy living advice (often online) - were also used by frontline advisers to help customers manage their conditions.

Short course training was widely used by many providers – CSCS construction site cards, Security Industry Authority training and licence and other short entry courses (Care Routes, Customer care). The inability to access other funding such as the Individual Learning Account to help support the costs of this training was criticised by providers, who did not understand why this constitutes 'double funding' of public resources. Discussions with case study employers point to the value placed on potential candidates having the relevant licence in selecting recruits.

Group job search and work trails were not always popular with frontline advisers, as some reported that customers see this was working for nothing. Supported employment was available in one provider along with bespoke recruitment for a large retailer and bank. JCP WCs reported that this was well-known among potential WFS customers who might opt to refer to this particular provider to access these.

Bridging the gap between their last benefit payment and the first pay cheque was widely considered by frontline advisers as a challenging time for many customers. This was more often reported for WFS customers (as fewer WAS customers had reached the stage of job entry at the time of interview). Providers cited a range of methods for supporting customers through this short transition period, including payments for bus passes, supermarket vouchers, work clothes and referrals to food banks.

What difference does the service make to customer outcomes?

As a result of WFS and WAS programme participation, two-thirds of customers surveyed across both programmes felt their motivation to work had increased to some degree, with more than nine-in-ten customers exhibiting a desire to work in the future. Furthermore, two-thirds of WFS customers and two-fifths of WAS customers felt confident in their ability to take on a job without harmful consequences to their health ( WFS 63%, WAS 41%), with half feeling more comfortable disclosing their disability or condition when applying for jobs ( WFS 55%, WAS 50%).

At the time of our interview, 81% of WFS customers surveyed were already in work or felt ready to be in work (32% ^[2] , 49% respectively). Some 40% of WAS customers were the same (16% already in work, 24% who felt ready) reflecting the longer pre-employment support period required to get WAS customers ready to undertake job search. For those not yet working, 83% of WFS and 66% of WAS customers we spoke to expected or intended to be working within a year.

However, it should also be noted that a minority of customers surveyed from both WFS and WAS came to an end of their support without moving into work, or discontinued participation due to lack of individualisation and relevance. This suggests that, while the programmes work for many, they may not be long enough (an issue for some on WFS) or sufficiently tailored enough to meet all needs. WAS customers were more likely to have concerns about moving into work, especially those with with mental health conditions, greater health needs and longer durations since they last worked.

For some customers on both programmes, who had been offered jobs with more than 16 hours, the risks of leaving benefits were too high and they opted for permitted work (fewer than 16 hours),so that they could retain their benefits while they became comfortable with working. Providers did not receive outcome payments for these jobs, but this provides some evidence of customers' progression towards employment. As areas transition towards Universal Credit, the relevance of 16 hours or more employment and the fact that this requires customers to leave their current out-of-work benefits behind will have less relevance.

What differences do the services make to employers?

A large majority of customers do not want providers to speak to their employers as customers do not wish to reveal their health conditions to employers and see no benefit in doing so. This can vary from 80-95% usually. This does limit the opportunity for a more involved in-work support process as envisaged in the delivery guidance on both programmes. Providers do still engage with employers, but mostly for customers with learning disabilities and other developmental conditions, who see benefits from more specialist support from providers, e.g. Individual Placement and Support ( IPS) packages.

Evidence from a small number of case studies suggests that providers have good employer connections. Employers find the potential recruits confident, enthusiastic and well-prepared and they have come to trust the judgement of provider staff with whom they had maintained contact with through successive employability programmes. Few of these employers were aware of the detailed support provided to customers by the programmes and most were not familiar with the programme names.

In a small number of cases, providers had engaged specifically with case study employers to support them in making adjustments in job roles to adapt to the needs of the individual, or to give advice on using additional support for the new recruits, such as buddies etc. However, in most cases the employers reported that they gained confidence in recruiting people with disabilities and health conditions from the general advice gained through their working relationship with providers.

From the employers that were surveyed, the level of employer support from providers varied widely. This might include setting up a recruitment day at the provider's premises to interview a series of candidates, through to general advice on the recruitment of people with disability or health conditions. A small number had used work trials and would probably not have recruited without these. Others felt that the candidates were frequently very motivated and reported that they had good retention rates so far – so their involvement with the programmes provides them with better access to good employment candidates.

Ways to improve effectiveness and successful outcomes

There are a number of key recommendations for Scottish Government and their stakeholders arising from this first phase evaluation that would improve employability services performance:

• Warm handovers are an essential part of improving the quality of referrals to employability services. This is as much about selling the potential of the programme as voluntary participation. Referral numbers need to be managed in line with provider capacity so that resources are not under-used or overloaded.
• A deeper engagement process through community and health services would have been very challenging to establish in the time available, but providers and JCP WCs see this as being an important consideration for future programmes. Raising the possibility of moving back into work among people with disabilities and long-term health conditions, away from the context of benefit and employment services, would allow more space for potential customers to consider the offer.
• The requirement that providers undertake an hour a week face-to-face with customers was particularly successful. Adviser contact time with customers drives outcomes and was widely supported among frontline advisers and customers. This has been carried forward into the contracts for Fair Start Scotland ( FSS) and there is sufficient evidence from WFS and WAS for this to continue.
• However, we recommend that Scottish Government explore ways of enabling greater flexibility to accommodate customer preferences on frequency of intervention. A key issue is lessening the administrative burden to support this, while ensuring this is driven by customer wishes.
• A review of compliance activity should be undertaken to seek ways to better balance effective programme administration transparency and quality of delivery for participants. Investment in more technology should be considered to reduce administrative demands where possible. When customer contact drives desired outcomes, increasing the time spent on this should be prioritised.
• Scottish Government and partners should consider the Induction procedures from a customer perspective, to review how this impacts on, and can encourage, the engagement of voluntary participants.
• Specialist support services have been provided, but these are often relatively short interventions designed to help customers develop coping strategies. Where longer term and more intensive support relied on NHS services, they were often not consistently available in all parts of Scotland. Further thought needs to be given to:
  • strategic discussion with NHS Scotland and Health Boards to explore more consistent access to, and alignment with employability and mental health support services for customers who wish to return to work.
  • Consideration of how resources for specialist services can be directed more cost-effectively to a quality service, e.g. using a ring-fenced budget or providing a centrally funded service for frequently used services where the likely volume means that core funding may support higher service standards and lower unit costs ^[3] .
• Open-book accounting is now in place for FSS and should be used by Scottish Government to provide a more forensic analysis of the costs of employability service delivery, particularly specialist services, in future.
• Scottish Government should consider engaging providers more in practice development – part of the learning contract process – where good practice can be aired and discussed. These are probably best conducted in a separate forum, away from the contractual and operational issues dealt with in the programme management process.