Fair Start Scotland evaluation report 1: implementation and early delivery review

6. Process: referrals and service delivery

This chapter reflects the range of views gathered from participants, Service Providers and JCP staff on their day-to-day experiences of participating in and delivering FSS services. 

The findings are grouped together into a number of themes that emerged from both the stakeholder interviews/surveys and the focus groups with participants. These reflect the ‘process’ of referring participants into FSS services, and the broader ‘processes’ of service delivery. The final section of the chapter outlines what worked well, areas for improvement and Scottish Government actions that are planned and already taking place in response to this feedback.

6.1 Referrals

From the outset of FSS service design, it was anticipated that the majority of referrals to FSS would come from JCP. This expectation was set out in the procurement materials and was the foundation of the partnership working and governance structures that were developed in tandem with Service Providers and DWP throughout the mobilisation period. 

Now that FSS has become more established, other organisations are also signposting participants to FSS.  SG, Service Providers and JCP have an established process to allow the JCP eligibility check and formal referral to take place following identification of a potential participant by a third party organisation.    

Both Service Providers and JCP recognised that there had been an initial ‘stockpiling’ of potential FSS participants during the second half of the mobilisation period (i.e. January – March 2018), where JCP Work Coaches had identified potentially eligible FSS participants in advance of the soft launch in mid-March and the service launch on 3 April 2018. This meant that there was initially a high referral rate to Service Providers and this period has made a significant contribution to the positive start in the first year of the service. This situation created an initial spike in demand for the service, which was eased to an extent by the soft launch approach. 

While there was widespread concern about the fall in referral numbers after the initial spike, there was also a recognition that referrals are not in themselves a sign of success: lower referrals may mean a stronger client focus – in other words, JCP customers are being referred to other services that better meet their needs and situation.

The findings from the Rocket Science surveys with frontline staff suggest that, to some extent, early teething issues around referrals were resolved at ground level. JCP Work Coaches were generally positive about the referral process, with 83% of them agreeing to some extent that “so far, the referral process has worked well”, and 92% agreeing that JCP staff “were working well with Service Provider staff”. Similarly, frontline Service Provider staff were also very positive about their relationships with JCP staff, with almost three quarters (74%) agreeing that "Staff from my organisation are working well with JCP Work Coaches”.

6.1.1 Eligibility criteria

Both the senior staff of Service Providers and JCP viewed the FSS eligibility criteria as distinctly different from previous programmes – including the transitional employability services of the previous year. 

The FSS eligibility criteria were widely seen by JCP senior staff as complex and difficult to understand, and there were issues with different interpretations between Lots, and between JCP Work Coaches and Service Providers in the same contract area. 

However, this view differs from that of frontline staff. When surveyed, all responding JCP Work Coaches agreed to some extent with the statement "I am clear about FSS and what it is trying to do”, and 98% of them reported that they agreed (either strongly, agreed, or tended to agree) that "I can describe to potential participants what FSS will be like and how it could help them”.

Similarly, Service Provider frontline staff were confident that they were clear what FSS is trying to do (with 94% of those surveyed agreeing) and also that they are able to describe to potential participants what FSS is like and how it can help them (98% of those surveyed agreeing to some extent). 

Some JCP senior staff reported that issues around eligibility have improved over time, perhaps as a result of changes to criteria. There have been no changes to the FSS eligibility criteria to date, so this may more likely be the result of the developing local relationships and growing understanding of both JCP and Service Providers of the intended flexibility of the FSS service design.

6.1.2 Eligibility vs suitability 

The stakeholder research led by Rocket Science uncovered early challenges arising from different Service Provider and JCP perspectives on suitability, as distinct from the eligibility, of prospective participants. The feedback highlighted that while prospective participants may be eligible for FSS services, their personal circumstances, or other factors may in effect mean that they were not ready or suitable for FSS support at that time. Challenges arose where an individual met the eligibility criteria to be referred on to FSS Service Providers, but once referred, it became clear that they felt unable to move into employment within the 12 month pre-employment support period or attend services regularly due to fluctuating health conditions. Similarly, there were differing perspectives from Service Providers and some JCP management around the expectation that participants on the service would be able to commit to working 16 hours a week^[18]. Sometimes support services are not appropriate as the timing for the individual just isn’t right.

This is an area that requires judgement rather than tangible evidence and it was felt by some Service Providers that, for a variety of reasons, there was a low chance of some customers who were referred to them being able to work 16 hours per week by the end of the [pre-employment] support period. 

Ongoing SG discussion with JCP/DWP and FSS Service Providers has emphasised the voluntary nature of FSS services, with the individual making any final decision on their referral into FSS. Both parties also acknowledged that Service Providers are best placed to discuss with potential participants whether they would be in a position to take up 16 hours work after 12 months of personalised support. In reality, we are starting to hear of some participants being supported into working fewer than 16 hours per week. This cannot be claimed financially as an FSS outcome, but is an important step for the individual participant and is recognised as such by the Scottish Government and FSS Service Providers.

6.1.3 Knowledge of service

During the focus groups research, most participants told us they had first heard about the service through JCP, though in each focus group there tended to be one or two participants who had become aware through another source, e.g. through word of mouth from friends or family, on Facebook, through their GP, or through a job fair. Many of the participants who were referred through JCP talked about being given a leaflet about FSS by their Work Coach. The suitability of the level of information given at this first discussion was dependent on the participant – some felt that they were given the right amount of information, whereas some felt they could have done with more. For some participants, there was a feeling that Work Coaches possibly didn’t know a lot about the service.

They [JCP Work Coach] said it was a new thing and they were told to offer it to you but they didn’t know a lot about it
- Aberdeen participant

I think a lot of Work Coaches and people from the Job Centre actually don’t know about Fair Start Scotland, the programme… he [Work Coach] had only recently heard about it.
- Glasgow participant 

These findings were echoed in the Rocket Science survey of FSS Service Provider frontline staff, with only a fifth (20%) agreeing that “when referred participants arrive they are clear about the FSS service and what it will offer them”.

At the same time, 80% of JCP Work Coaches surveyed agreed to some extent that they were finding it easy to encourage people to join FSS.

Participants in the focus groups suggested that having staff from a FSS Service Provider within the Jobcentre would be helpful, or in places where those who would benefit from the service may be, such as job fairs.

It would be better if somebody was in there [JCP], sitting at a desk, where you could go up and ask.
- Inverness participant

Participants also suggested that the service was not always brought up by JCP Work Coaches. In some cases participants were told about FSS after them enquiring about what was available, and in a few cases, participants brought it up themselves after hearing about it through advertising, friends and family, or on social media.

I had been asking [about FSS] but no-one knew anything about it at the Job Centre and it wasn’t until I changed the person who I saw every second week to sign on with…she told me about it. 
- Glasgow participant 

In my case I wasn’t introduced to Fair Start but I felt ready to go back to work on a part-time basis and I knew I didn’t actually have the drive and needed a bit of a push so I actually asked my Work Coach. 
- Renfrew participant

The vast majority of participants were aware of the voluntary nature of the service. However, there was some discussion of the meaning of ‘voluntary’, with some participants worried that there may have been repercussions had they not engaged.

Although the majority of participants felt they had been told upfront by both JCP and their Service Provider about the voluntary nature of the service, there were differences in how participants interpreted this information. 

I got the impression she [Work Coach] didn’t know too much about Fair Start and she didn’t really make it sound like it was voluntary, it was a case of “you’ve been unemployed for 2 years and nothing’s worked, go and do this” so it kind of felt like if I didn’t do it…you know. 
- Hamilton participant 

I thought I had to do it, I thought I had no choice but I was quite happy doing it but I did think I had no choice, that I had to do it.
- H&I participant 

Participants highlighted that the information they were given by Service Providers in their first meeting was helpful. There was particular mention of the time that Service Providers had taken to go through any documentation or information with them at their own pace – this was especially highlighted by those who has literacy issues, learning disabilities or visual impairments.

I’m dyslexic so I can’t read and write a lot... At my first interview every single bit of paper that was there [my advisor] read through, so obviously it took a wee bit longer because he’s actually got to read everything to me and he was brilliant that way.
- Kilmarnock participant

[when asked how Providers gave information about FSS services] Verbally and with large print documentation, I don’t know whether that was scanned and then made bigger for me, but I got all the brochures in large print and stuff so it was brilliant
- H&I participant (visually impaired)

For others however, attending the first appointment with their Service Provider was a nerve wracking experience, for some due to what they felt was not enough information, and for others due to previous experiences on employability programmes. This reinforces the importance of prospective participants having access to information about services beforehand and feeling that this is something relevant to them.

It goes back to that first session where you know I’d been given the details of the service but I was still unsure exactly what I was going into, you know, so I was nervous.
- Edinburgh participant 

For a lot of folk, it’s just that, you know, the nerves you get because you don’t really know what you’re stepping into.
- Hamilton participant 

There was a feeling from some participants, especially for those who had barriers to attending the Jobcentre, that it was not made clear why the initial referral had to be done in person.

The only thing I found slightly bizarre was, when I met [Work Coach] about Fair Start, I got a note through saying I needed to go see [Work Coach] at the Job Centre. I literally sat down to have a face-to-face chat with her then [she said] away you go, so I found that slightly odd … I had to get someone to walk me, because I’m visually impaired, to walk me to the Job Centre.
- H&I participant

Many participants highlighted that the time between being referred onto the service and having their first appointment was short. For most this was a positive, as they recounted negative experiences of long waiting times for similar services in the past, and reported that a quicker transition was more likely to keep them feeling motivated.

I liked the quickness when we started because…. I was stuck in bad routines and it was good that it happened quickly and it can get you into good habits.
- Dundee participant

As well as the relatively short waiting times after referral, participants felt that the length of time that support (12-18 months)^[19] was available for was also very positive.

I think it’s a good thing for me because the type of thing I might be looking for might not be right out there just now, it might be a certain time of year it comes up, it might be next week or it might not come up. But the fact is they’re in there for the long term, 52 weeks, that’s a big help.
- Renfrew participant

6.1.4 Individualised nature of the service

Throughout the focus groups, participants were positive about the individualised nature of the service. Many felt that they were being treated as ‘a person’ rather than ‘a number’, and that they were being consulted on what they wanted from the service. Individual preference in aspects such as pace of progress and method of communication were also felt to be adjusted to the preferences of individuals. 

It’s also adapted to everybody’s different needs…if you don’t like phoning or you don’t like emailing or if you can’t emai …maybe see you twice a week or three times a week, so it’s adapted to everybody’s learning pace.
- Glasgow participant 

A few participants felt that although the service was meant to be personalised, they were still following a structure which did not always meet their personal needs. The pace of the service was also felt to be directly linked to the needs of the individual participant. 

My confidence was shattered, everything was shattered…she’s [adviser] not expecting me to jump into a job in two weeks’ time, she’s doing it slowly, building things up until…I’m capable of holding down a job.
- Dundee participant

Related to the individualised nature of the service was the one to one support from a dedicated advisor. Participants appreciated having one point of contact that they could get in touch with when they needed, but also that even if their particular advisor was not available, someone else from the team would most likely be able to help them. 

I think that’s the nicest thing, it’s a person rather than a bit of chippy chapping on the computer…this is just the person you can relate to and you build up a rapport with each other and that is real personal, it’s been good.
- Edinburgh participant 

Participants also mentioned the importance of continuity of Adviser throughout their support, and they expressed a strong preference for remaining with the same individual Adviser once a relationship was established. 

This individual and personal approach was also felt to be reflected in the type of employment that was being worked towards – participants felt they were able to have a say in what they wanted to do, and did not feel pressured into taking unsuitable positions. Participants felt that the focus of the service was the journey towards sustained employment, where getting a job was the end goal, but not the only goal. Related to this, there was a feeling that the service also concentrated on the wider benefits of being in a good job.

There’s certain things I hadn’t thought about, you know, aside from wanting a job. It’s not just about getting money to pay bills obviously, it’s the camaraderie with your workmates, it’s getting out the house, a sense of purpose, all that stuff.
- H&I participant 

Participants were especially in favour of the wide range of support they had been offered, from help with money management and debt, to counselling, as well things like occupational health. It was felt that the concentration on addressing wider or underlying barriers, and linking in with other services was a good way to approach employability, and would lead to more sustainable job outcomes. 

I mean it even goes beyond working. I think not long after I joined they were talking about healthy eating and I was like “what has that got to do with a job?” but it was all about building yourself up in a lot of different ways.
- Edinburgh participant 

Participants were also positive about the offer of in-work support, especially those with a mental health issue, who felt that continued support was crucial in being able to stay in a job. 

It’s given me hope, plus the fact they support you after you go back to work, when you get a job the support is still there, which is just invaluable because sometimes you think “yes, I’ve got a job”, you start it, and then before you know it, you sort of fall apart and start to tremble and there’s just nothing, there’s no support.
- Aberdeen participant 

In some cases, participants reported that being on the service had had a positive impact on the relationship with their JCP Work Coach. Participants felt this was because their Work Coach knew they were involved in something positive and so could provide clear evidence of job search related activities as a result. 

My experience has been since I’ve been on Fair Start my [JCP] Advisor has been far better with me, he’s not pressured me into constantly looking for work because he knows I’m actually getting something from Fair Start.
- Inverness participant

The experience and specialist knowledge of the FSS Service Provider advisors was felt by participants to be important, in terms of both being able to relate to participants’ circumstances, but also to provide accurate advice around a range of issues. 

In summary, a recurring theme for participants was the importance of a combination of both the length of support available, the personalised nature of the support and the focus on sustainable employment, creating a space for people away from the ‘pressure’ of the ‘norm’. For some participants, this meant they were able to focus on some of the more significant barriers they faced, and work with advisors to find long-term, sustainable plans. 

It does make a vast difference for people who really struggle with maybe health or other issues…this has been a long time coming and whoever has thought this up it’s a good thing because it takes the more vulnerable people out, I’m not saying it takes us out the system, but it gives us a break from the norm.
- Falkirk participant 

If I’m being honest when I started the Fair Start process I wasn’t really thinking about work…I know it’s part of it when you sign up but 12 months seemed a long way away at that point and when I was on this 6-monthly cycle with the DWP at the Work Capability Assessment Centre, there’s no incentive for you to get a wee bit better and they turn up later and say “you’re fine, away and work fulltime” whereas this programme is a breathing space where you think “actually I can just focus on my health for 3 or 6 months” .
- Hamilton participant 

6.2 Service delivery

This section provides feedback on the wider experience of service delivery and highlights both elements that worked well and those that participants, Service Providers and delivery partners felt could be improved. 

6.2.1 Governance structures

The governance of the service was felt to be well structured and effective, with a clear hierarchy and appropriate escalation of issues where needed. Senior officials from SG and DWP formed a strategic platform for oversight and escalation and an operational group to focus on identifying and resolving delivery issues, gathering and sharing good news stories, and recognising good practice.

The governance structure was seen as a key part of the effective collegiate working, involving appropriate stakeholders at each level in the structure and providing for open discussions about issues and their resolution.

6.2.2 Relationships

There was general agreement amongst the JCP and Service Provider staff interviewed by Rocket Science that FSS service delivery had noticeably improved in the 6 months since the launch of the service. Much of this had been built on enhanced communication across the JCP Scotland estate and between JCP Work Coaches and Service Provider staff at a local level, and an effort to sort issues out locally and avoid escalation. 

The positive influence of local working relationships was consistently emphasised across both the interviews and survey responses. The quality of the working relationship between Work Coaches and Service Provider staff was seen as fundamental – this was reflected in the sense that the implementation process and initial performance were better in areas where there were positive pre-existing relationships and local familiarity; where Service Providers and JCP staff had previously worked well together to deliver other commissioned services, and the need for these relationships to be actively maintained – through regular if not constant engagement about current and evolving service offers. 

In terms of accountability there were both positive and negative issues raised. On the one hand there were reports of the quality of the relationship with SG Performance Managers, and on the other, of the perceived micromanagement that this can involve. 

We have a good relationship with performance manager – I feel I can pick up the phone to ask any question – it’s a really open conversation. On the whole it does feel that the emphasis is less on contractual audit and more on true partnership working, how the programme is evolving.
- Lead Provider

It feels like we are sometimes micromanaged – but we are able to be autonomous in some elements like design etc. 
- Lead Provider

As part of the online survey work led by Rocket Science, Service Provider frontline staff were asked to identify three things that had gone well during the early implementation phase. Figure 7 shows the top three positive elements of the service were: positive outcomes for participants (33%); partnership working 26% and relationships with JCP 9%.

Figure 7: What went well during implementation – Service Provider front line staff responses

[Source: Rocket Science analysis of online survey data]

JCP Work Coaches’ responses to the same question are set out in Figure 8. The top three most common responses are again related to communication or partnership working (36%), customer outcomes (28%), and the FSS ethos (13%). 

Figure 8: What went well during implementation – JCP Work Coach responses

[Source: Rocket Science analysis of online survey data] 

Responses from the online surveys with frontline Service Provider staff and JCP Work Coaches also highlight the challenges of referrals and eligibility mentioned earlier.

Figure 9 below shows the main challenges of delivering FSS services, as identified by Service Provider frontline staff. These were initially open-text responses, which were grouped into themes by Rocket Science researchers. Almost a third (28%) of all Service Provider front line staff identified a lack of referrals or appropriate referrals, as a significant challenge in their area.

Figure 9: Implementation challenges – Responses by Service Provider front line staff

[Source: Rocket Science analysis of online survey data]

Similarly, JCP Work Coaches were asked whether there had been any initial challenges in implementing the service across Jobcentres. Just over half of respondents (53%) felt that there had been implementation challenges.

In a follow-up question asking for further detail on any challenges encountered (Figure 10), over a third (34%) of JCP Work Coaches felt that there had been challenges relating to Service Providers and eligibility criteria, with 22% feeling that ‘Provider approach’ had been a significant challenge. 

Figure 10: Implementation challenges – responses by JCP Work Coaches

[Source: Rocket Science analysis of online survey data]

6.2.3 Rural and remote areas

As part of the stakeholder interviews, Rocket Science spoke with a number of supply chain Service Provider organisations working to deliver FSS in rural and remote areas. They reported some specific challenges relating to the local geography and labour markets:

• Job opportunities were often limited and more likely to be seasonal or short-term / temporary in nature;
• There could be significant distances between Service Providers, participants and employers. Where this was coupled with the costs of travel, poor internet coverage and infrequent public transport, it could lead to limited face to face contact between Service Providers and participants and in the long term could restrict FSS participants’ access to available employment opportunities.

Service Providers in more remote areas reported concerns around the impacts that these challenges may have on their ability to deliver their expected levels of job outcomes.

6.2.4 Monitoring

The rationale for the level of monitoring and management information required by SG arose from a need to show that SG were adopting an open and transparent “test and learn” approach to delivery of the first fully devolved Scottish employability support service. This drives a higher level of public and parliamentary scrutiny of FSS performance and impacts, to ensure value for money and quality of service delivery. As a result, SG recognise the need for a broad suite of management and monitoring information in order to evidence both ongoing service improvement and long term development.

While recognising the need for effective monitoring as part of good governance of the service, Service Providers voiced concerns that the scale, range and rigidity of the monitoring requirements were getting in the way of delivering an effective service. This included regular requests for information; very detailed monitoring which some felt missed the bigger picture of identifying issues and agreeing appropriate responses; the associated range of Key Delivery and Performance Indicators; and issues about how outcomes were measured and verified. Service Providers felt that there were two specific consequences of this:

• that they had to divert front line time onto administration
• that there was a significant risk of underestimating both the conventional outcomes of FSS in terms of jobs, as well as not recording the difference made to people in terms of their health, employability and other outcomes such as FE.

What worked well?

• Frontline Service Provider staff and JCP staff felt they were clear about the purpose of FSS, and able to describe it to potential participants.
• Vast majority of participants knew it was voluntary
• Information given to participants was seen as helpful and clear
• The governance of the service was felt to be well structured and effective
• There was general agreement amongst the JCP and Provider staff that the FSS service delivery had noticeably improved in the six months since the launch of the service, with the positive influence of local working relationships consistently emphasised 
• Participants recognised the advantages of the individualised nature of the service, as well as the wide range of support offered 

What could be improved?

• There was some concern about the fall in referrals from JCP after the initial burst
• Eligibility criteria were felt not to be clear by JCP staff, which had led to issues around eligibility and suitability of participants, and the concern from Service Provider staff that some referrals were inappropriate
• A small number of participants were unaware or unsure of the voluntary nature of the service offer
• Some Service Provider staff reported that the scale, range and rigidity of the monitoring requirements were getting in the way of delivering an effective service
• There are a set of specific issues about the delivery of the service in more rural and remote areas and local Service Providers feel that many of these issues had not yet been adequately addressed

What are we doing?

• SG recognise the Service Provider resource required to capture the full suite of management and performance information requested and have already implemented changes to the MI required in response to a Service Provider feedback showing concerns over administration.
• SG are committed to the continuous improvement of FSS and are developing a Continuous Improvement Plan to drive this forward. 
• SG will review whether Service Providers are delivering the Supported Employment Framework and Individual Placement Support (IPS) appropriately as part of current contracts. 
• SG will work closely with Service Providers to make current labour market information available to local supply chain partners in the more remote and rural areas. The next stage of the evaluation will also explore employer perspectives on the delivery process, and participant and Service Provider experiences in rural and remote areas.