Health and work support pilot: interim evaluation report

3. Findings

This section presents summary data followed by an analysis of key stakeholder interviews, focus groups and online survey results as well as brief case studies derived from interviews and focus groups with clients.

3.1. Management Information Data^[16]

3.1.1 Throughput

Table 1: Total number of service users for Fife and Dundee by each stage of the pilot service (June 2018 to March 2019)

Stage of HWS	Number of service users
	Fife	Dundee	Total
Enrolments (Salus)	332	597	929
Clinical Assessments (Case Managers)	289	484	773
Discharges conducted (Case Managers)	124	128	252
Source: Scottish Government Health & Work Pilot MI data, June 2018 - March 2019.

Analysis of management information (MI) data suggests that there is a degree of drop-out at each stage of the pilot, from enrolment through to discharge across both sites. The largest proportion of drop-out occurs between assessment and discharge suggesting it is likely that clients have received clinical input. It should be noted however that as clients are eligible for up to twenty weeks of support there will be a significant time lag between enrolment and discharge.

It is interesting to note that Fife and Dundee have very similar numbers of discharges despite a much larger number of clients being enrolled into the service in Dundee which suggests potential variations in process and practice between the two sites.

3.1.2 Target Enrolments

The target set for the pilot (with regards to individual clients) is 6,000 enrolments over the two years of the pilot with an even split across the two pilot sites. To begin with this target was also split evenly across the pilot period, however following feedback in September 2018 a decision was taken to re-profile the monthly targets so that they gradually ramp up over the life of the pilot (see black dotted line in chart below).

Source: Scottish Government Health & Work Support Pilot MI data, June 2018 - March 2019.

Figure 2 demonstrates that, as a whole, the pilot has reached its monthly target only once (September 2018). The maximum number of clients seen in a given month appears to reach a plateau at around 130 clients. Given that the target increases over the two year period the gap between actual performance and the target continues to increase.

Source: Scottish Government Health & Work Support Pilot MI data, June 2018 - March 2019.

Figure 3 shows that Dundee exceeded the target on a number of occasions during the early period of the pilot whilst Fife has yet to successfully meet the target. It should be noted however that the Tayside area (which includes Dundee) has consistently tended to be high performing relative to other areas for similar pre-existing services such as Working Health Services Scotland. Additionally as noted elsewhere in this report, variations in labour market conditions, geography and marketing are likely to have impacted on differences in performance between the two pilot sites.

3.1.3 Enrolments

While there are a number of ways clients hear about the service the majority of individuals will refer themselves into the pilot instead of being referred by someone else (e.g. GP, DWP Jobcentre, employer). Figure 4 demonstrates that the most common way clients hear about the service and then self-refer is through their GP in both Dundee & Fife (58% and 54% respectively). Jobcentres are the second most common referral route in Dundee and the third most common in Fife, yet both account for approximately 13% of their total referrals. Other Health Professionals account for 14% of referrals in Fife, compared to 9% in Dundee.

Source: Scottish Government Health & Work Support Pilot MI data, June 2018 - March 2019.

Of the 929 individuals enrolled onto the service, 86% were employed and 14% unemployed (see figure 5 below). This figure was relatively similar across both Dundee and Fife. It should be noted that it had initially been anticipated that the short-term unemployed would make up approximately one third of the overall enrolments into the pilot and therefore this represents a significantly lower number than expected.

Source: Scottish Government Health & Work Support Pilot MI data, June 2018 - March 2019.

Of the 801 individuals who were employed, 56% were from small and medium businesses (<250 employees or self-employed) and 44% from larger employers (>250 employees).

Comparison of the two pilot sites reveals that Dundee has many more enrolments from individuals working for larger employers. Within Dundee 49% of those in employment came from larger employers whilst Fife only had 34% from this same group. This may be partly explained by variations in the overall labour market between the two areas or/and by differences in the marketing approach adopted by each pilot site.

It should be noted that a significant proportion of the large employers which individual clients work for are made up of public sector organisations, including local councils and NHS services. Overall this raises questions about the degree to which existing occupational health services within larger employers, including public sector employers, are adequately meeting the needs of their employees. This is an issue that requires further exploration and will be followed up in subsequent phases of the evaluation.

3.1.4 Clinical assessments

Of the 773 eligible clients who were assessed by Case Managers, most were employed and present at work (60%). It should also be noted that in addition to the above, 39 clients who are part of the Local Support stream were also assessed during this time period with the majority of these assessments taking place in Fife (69%).

Source: Scottish Government Health & Work Support Pilot MI data, June 2018 - March 2019.

3.1.5 Primary Presenting Health Condition^[17]

Historically the main client group of pre-existing services report musculoskeletal (MSK) conditions. The pilot has added a focus to target those experiencing mental health problems which are impacting on their employment.

Although the pilot is receiving a higher percentage of clients with mental health conditions (26%), the majority of continue to present with MSK as their primary condition (60%). It should be noted however that there are significant numbers of individuals who present with multiple conditions, including combinations of MSK and mental health related difficulties.

Source: Scottish Government Health & Work Support Pilot MI data, June 2018 - March 2019.

There are clear differences between the conditions reported by clients and their employment status as shown in the chart above. Namely:

Not employed – more people with mental health conditions (43%) accessed the service than those with MSK conditions (29%).

Employed – a higher percentage of people with MSK conditions (65%) than mental health conditions (23%) accessed the service.

3.1.6 Discharge and Outcomes

As previously noted lower numbers of clients than expected have successfully been discharged to date. This may be due to difficulties faced by Case Managers in engaging clients until the final discharge appointment. At present, data suggests that 33% of all clients who have been assessed to date have received a discharge whilst 16% have dropped out of the service before receiving a full service. Analysis by employment status suggests that those not in employment are more likely to drop out (20% of all assessed) than those in employment (15%) and are therefore less likely to see the service through to discharge.

Source: Scottish Government Health & Work Support Pilot MI data, June 2018 - March 2019.

Partly due to the issues noted above there is currently a very limited amount of data on outcomes available for clients. Full analysis of outcomes for clients will form a central part of the analysis which is due to be undertaken as part of the second phase of the evaluation process.

3.1.7 Employer Engagement

Whilst the service has a target to provide support to an additional 200 employers, work is still ongoing to reach agreement between service delivery partners on a definition for this target.

Data that is currently available suggests that across both pilot areas, 53 employers have engaged with the service to date (June 2018 to March 2019), although the majority of this engagement has been in Fife as opposed to Dundee. Reasons for this will be explored in more detail in subsequent stages of the evaluation.

In addition to formal engagements with employers (in order to provide a particular service) local Healthy Working Lives staff situated within the pilot have also been engaged in marketing activity, primarily directed at SME’s. It is hoped that building relationships and raising awareness of the service will eventually lead to increased uptake of the service, both by employers and their employees.

3.2 Design and Development

3.2.1 Pilot Design

There was a six month lead-in between funding being confirmed and the service going live in June of 2018, with the bulk of service delivery, process and data collection design taking place during this period. This was seen by many of the strategic staff who were involved as being very pressurised and has had knock-on effects through to the implementation stage, including impacts on staffing, securing premises for the delivery teams, defining roles and on data collection processes.

Despite the challenges experienced, the general view from staff and stakeholders is that there was significant buy-in from all service delivery partners.

“It (design phase) has been good, a lot of buy in from different stakeholders – there’s also been academics involved and generally people with a real wealth of experience…. without that kind of early relationship building the project would not have got off the ground.”
Stakeholder interview

3.2.2 Staffing and Premises

There were a number of recruitment challenges that had an impact on the design and early implementation period of the pilot. Such recruitment issues were noted to have affected both local delivery teams as well as the Scottish Government’s national pilot team.

“There was a lead in time, but, because of budgets and recruiting issues, it was difficult to recruit staff because money hadn’t come down or the financial plan was not signed (off) by the appropriate people.”
Stakeholder interview

With regards to the logistics surrounding set up of front-line delivery, Fife initially struggled to obtain suitable premises for their team. The team in Dundee were, on the other hand, able to secure use of facilities at a local hospital with relative ease. This was attributed by some stakeholders to the structures and processes in place within local NHS services and variations between local health boards. This should be noted for any potential future service provision which uses the pilots existing structure as there may be significant variation in both the availability of premises and buy-in from senior stakeholders within local health services.

3.2.3 Data Collection

Staff noted concerns with the data collection system and processes in place for the pilot^[19]. In particular there were issues identified with the design of questionnaires used during contact points with the client (assessment, review, discharge). This has resulted in unnecessary duplication (and therefore increased burden of use) throughout the system as well as inconsistencies in the amount and type of data collected for individuals. This has potential negative consequences both for effective service delivery as well as for future evaluation of the service^[20].

3.3 Early Implementation

3.3.1 Changes since the design stage and knowledge of the pilot

There have been a number of changes to the pilot since the launch in June 2018, including changes to eligibility criteria, processes and data collection^[21]. Whilst the changes were generally perceived as positive and as adding value to the pilot, they have also caused some issues and frustrations. According to the staff and stakeholders who were consulted these frustrations have mainly been related to issues pertaining to communication of changes, rather than with the changes themselves.

One of the potential consequences of this is that amongst local delivery staff (Case Managers) there appeared to be a perception that the call handlers (HWL and Salus) lacked knowledge of the pilot with regards to its aims and eligibility criteria. Both Salus and HWL were however open in talking about the challenges that they encountered in adapting to changes required by the pilot. Representatives from both organisations felt that the changes contributed to a lack of clear understanding in the initial weeks after implementation.

“With two extra weeks we would have had time to look at what the different streams were doing and be more confident.”
Call handling focus group

It should also be noted that unlike the Case Managers call handling staff work across a number of different services with variant aims and eligibility criteria.

3.3.3 Liaison Between Service Delivery Partners

Stakeholders noted that experience developed working together on pre-existing services (such as Fit for Work and Working Health Services Scotland) was beneficial with regards to the development and implementation of this service.

However throughout the consultations, there was some concern that the pilot may not have been a high priority for all delivery partners. Some stakeholders reported that there may be issues with organisational priorities and agendas being given precedence over the pilot. It was also reported that there were some challenges associated with running localised pilots within existing nationwide services.

Surveys conducted with front-line staff indicated that 65% of Case Managers ‘agreed’ or ‘strongly agreed’ that they coordinated well with HWL, with 73% indicating the same for Salus.

Although the online survey indicated that the majority of Case Managers felt that the service coordinated well with HWL and Salus, the findings from the focus groups and interviews did not fully support this. It was also noted that call handling staff were somewhat disconnected from the clients total journey through the service. As clients are passed from HWL to Salus and from Salus to case management, staff involved at the various stages of the pilot reported not being fully aware of what happens to clients throughout their involvement with the service. The overall feeling from delivery staff was that the pilot coordinates reasonably well from the clients’ perspective, but behind the scenes there is less coherence.

3.3.4 Governance

Some stakeholders noted that there were still governance related issues that needed to be resolved. For example, all three delivery partners have their own governance and reporting standards outwith the pilot, which can result in decisions made outwith the pilot’s governance structure that have consequences for the running of the service.

Concerns were also raised regarding the effectiveness of the pilot’s governance groups with regards to their capacity or willingness to provide sufficient challenge and to hold the different delivery partners accountable for the progress of the pilot. This issue may also result from having multiple service delivery partners involved in the delivery of the project, which may result in a lack of coherence with regards to governance.

3.4 Service Delivery

3.4.1 Referrals and Client Support Needs

Key to the design of the pilot is that the clinical interventions provided to participants are not markedly different to what can be accessed via mainstream routes or through pre-existing services. Rather what has changed is the access routes into these interventions.

Feedback from client focus groups and interviews suggests that the pilot’s capacity to circumvent longer waiting times for mainstream NHS services is seen as one of its main selling points and key strengths.

Client Case Study 1

Client ‘A’ works at a SME and had been suffering from a flare up of a longstanding back problem. She heard about the pilot through her employer and decided to get in touch, hoping that she would receive quick access to physiotherapy.

The client self-referred and was assigned a Case Manager who she engaged with via phone appointments. The Case Manager in turn assigned her to a physiotherapist from whom she received clinical support.

The client reported that the input she received prevented her from needing to take time off work and equipped her with the knowledge she needed for ongoing self-management.

“I absolutely cannot fault it, it was a great service, it was so quick, anyone that I spoke with was helpful...”

Client A

“I would have gone privately if this service wasn’t available, I was seen within 10 days which is brilliant.... before things become even more troublesome...”
Client interview

However one of the unintended consequences of providing faster access routes into clinical interventions has been the number of clients entering the service with significantly more complex care needs than was initially anticipated. As noted in the data section, the majority of clients self-refer but are made aware of the service from their GP. However Case Managers have noted that they receive referrals from GPs which are not necessarily appropriate for the service, for example, clients with terminal cancer or long term mental health conditions that will require years of ongoing support. While such individuals may benefit to some degree from the support provided, the pilots focus on work may not be appropriate, and as such they may be better served by mainstream NHS services.

“GPs want a permanent service they know they can go to…but it has to be more work focused, it has to be clearer that we are trying to keep these people in work. We have to justify the people that are trying to come through the service. GPs will use us for anything in order to not put clients into long waiting lists.”
Case Management focus group

Across all service delivery partners it was felt that clients who were being referred or signposted by GPs were less likely to be aware of the specifics of the pilot’s service delivery offer. These clients were often under the impression they were simply calling up to book an appointment for either physiotherapy or counselling.

Client Case Study 2

Client ‘B’ works at a SME in Dundee and had been struggling with mental health concerns when she contacted her GP. Her doctor recommended that she self-refer to the pilot in order to avoid having to wait for potentially over a year to be seen by mainstream NHS services.

She had managed to continue to attend work during this time and was looking for some preventative help before her condition got worse.

Although she found accessing the service relatively straightforward she had difficulties in obtaining appointments with the counsellor due to the fact that the service only operates during normal office hours. She stated that she had waited up to six weeks between counselling appointments which were carried out on the phone. During this time she had several crisis episodes and felt unclear about who to turn to for help.

“They think they’re calling to book in an appointment (with a physio) – if the GP says, ’Phone that number and you’ll get physio’ they think that’s all they need to do.”
Call handling focus group

“...when the GP gives them the number they don’t explain, they phone up and think they will get an appointment immediately and are disappointed.”
Case Management focus group

This appears to be corroborated to some extent by the feedback received from clients during interviews and focus groups. Several of those interviewed who were directed towards the pilot by their GP were simply told that the HWS pilot would provide them with fast track access to clinical interventions without necessarily explaining the service in detail.

“They never really told me much about it...I found out more once I actually contacted the...service”
Client interview

The additional level of complexity of patients has resulted in more time being needed for interventions, and additional training requirements for staff being identified (i.e. training for suicide prevention), all of which impacts on capacity.

Case Managers reported that in some cases, this was the first support that some clients had received, despite having serious health concerns (mainly to do with mental health issues). This highlights the value of the pilot in attracting people who may not realise how serious their condition is, or who have slipped through the gaps in the current service landscape.

“You hear relief in people’s voices when they realise we can help.”
Case Management focus group

Client Case Study 3

Client ‘C’ was unemployed at the time she engaged with the pilot. She heard about the service from a Work Coach at the local JCP.

She self-referred into the pilot, looking for a service that could help her communicate her health needs to a prospective employer. The Client’s Case Manager put together a letter which detailed her health condition (fibromyalgia), how it would impact on her work and any adjustments she might need.

The client included these in applications and was successful in gaining employment.

Client ‘C’ felt that the support she received from the Case Manager, including the report on her health condition, helped her get back into work and that she would definitely recommend the service to others.

Case Managers felt that it was their clinical backgrounds (as occupational therapists, nurses, mental health nurses etc) that enabled them to address the wide range of client needs, even those that have proved more complex.

“I provide a general and holistic assessment, but because I’m an occupational therapist I feel as if I do interventions at that point as well. I’m able to support people who have a physical or mental health problem that requires urgent intervention, and this can lead to an action plan or a longer intervention occurring at the point of assessment.”
Case Management focus group

In addition to the issues outlined above there have also been some challenges associated with referrals from Jobcentre Plus (JCP). Namely, the number of referrals from JCP for unemployed individuals has been significantly lower than originally anticipated. JCP staff have highlighted the issue of drop outs in reference to this (i.e. the difference between the number of clients who agree to self-refer into the pilot at the point of discussion with their Work Coach versus the number who actually do make contact). This may be due in part to the fact that the pilot’s primary mode of access for individual clients (i.e. self-referral) assumes a level of health literacy, capacity and willingness to engage which may be problematic for more vulnerable members of the population.

As a result of such issues being identified, a formal web-based referral route from JCP through to Salus is being developed. This will allow JCP Work Coaches to refer the client directly, or the client can self-refer using a computer in the Jobcentre.

3.4.2 Client Experience and Quality of Engagement

Staff reported that irrespective of the various challenges the service has faced, they are working hard to ensure that these do not have an adverse impact on the client’s experience.

One of the areas that has been identified for improved efficiency is the access route into the service. The multi-stage process to ultimately refer clients to a clinician such as a physiotherapist or counsellor has been reported as being clunky with too much repetition. It remains the case that by the time clients have had their first physiotherapy or counselling session, they could have spoken to five different people in the service^[22]. It is felt by many that there is a risk that clients might feel unsatisfied with this process. In response to such concerns both Salus and HWL suggest that the service could be provided with only one call handling service.

“When a client is calling in, they’re told, ‘Call here then call here then call here’
Case Management focus group

Case Managers raised concerns that the current system of having two call handling services increased the risk of client disengagement. This is supported by HWL and Salus who, as previously noted, both suggested that only one service delivery partner is actually required to deliver the initial call handling element of the pilot.

3.4.3 Staff Roles

It was clear from the fieldwork carried out that there is a need for clearer definitions, guidance and expectations about staff roles. Case Managers in particular reported a lack of clarity with regards to their role. This is due to the fact that the service is designed with the expectation that the Case Manager role is there to provide assessment, review, referral and discharge functions in addition to liaising with wider affiliated services and employers where appropriate. However, Case Manager’s emphasised that their background and training as clinicians meant that they are also capable of providing a range of clinical interventions to clients as opposed to simply referring clients on to others for intervention (e.g. physio or counselling).

“I provide a general and holistic assessment, but because I’m an occupational therapist I feel as if I (can) do interventions at that point as well.”
Case Management focus group

“There was...confusion around who was doing what. The roles were a bit unclear.”
Call handling focus group

Concerns around ensuring clarity with regards to Case Managers’ roles is more widespread than this service alone. A review of the literature around Case Management led services highlights the importance placed on ensuring clarity of roles and remits (see Goodman et al, 2010^[23], Chapman et al, 2009^[24], Ross et al, 2011^[25]). A review of Case Management led services undertaken by the King’s Fund in 2011 stated that:

“Case management programmes have often been characterised by confusion over roles, which can lead to tension.... These problems are mostly due to a lack of clarity regarding role boundaries and/or a lack of communication between the different care providers”
(Ross et al, 2011).

Given existing concerns with regards to capacity within local Case Management teams (further discussion of which is found below) this is an issue with requires further exploration. If Case Management staff are struggling with existing workloads, as has been suggested, then clarity around staff roles is vital to ensure that staff are not engaging in additional work that is not required nor expected of them. However it should also be noted that as qualified clinicians Case Managers may feel that their skill are being under-utilised if expected to simply provide a basic case management function.

3.5 Pilot performance

An initial look at comparative data between pre-existing services (Working Health Services Scotland and Fit for Work) and the pilot demonstrates improvements with regards to numbers of clients accessing help. However this increase appears to be largely supported by widening of eligibility criteria to include individuals not qualified for access to pre-existing services (e.g. the unemployed, those employed by large organisations and off work sick for less than four week). Actual growth in core target client groups, such as those employed by SMEs, has been limited to date (less than 5%).

Additionally it should be noted that growth in client numbers varies between the two pilot areas with Fife demonstrating stronger improvements in the numbers of SME clients accessing support over baseline figures as compared to Dundee. This is due to the fact that a significant proportion of Dundee’s increase in activity has been supported by the inclusion of clients from large, often public sector employers whereas the service in Fife has continued to receive the majority of its clients from SMEs.

3.5.1 Targets

Targets are one of the biggest challenges associated with this pilot. The focus on enrolment numbers (targets) is seen as a concern by many stakeholders who feel that there should be greater focus on other outcomes from the pilot. In addition, there is confusion as to how the target numbers were derived.

“It’s about what each individual client needs and what quality we can provide…it would be sad to give that up because we can’t get the targets. Do we want to be a very unique service or just meet numbers?”
Case Management focus group

“The numbers are very unrealistic; they didn’t get them right.”
Call handling focus group

The equal split of targets between Fife and Dundee (1,500 each per annum) is seen to be problematic in so far as that it is not reflective of the local populations (Fife has a 16 plus population of 307,437 and Dundee 124,734^[26]) . Additionally the underlying geography of each of the pilot sites is likely to have an impact given that the Dundee pilot site serves a discrete city based population whereas in Fife the population is dispersed over a much larger and largely rural area.

From the fieldwork undertaken to date, it is clear that although the pressure of meeting target numbers appears to fall largely on Case Managers, they feel that they have very little opportunity to actually influence the number of people calling the service and are primarily there to provide assessment and support to individuals within the service.

HWL and Salus filter eligible clients through to local delivery teams yet are not aware of any targets they need to meet, or if local teams are meeting their targets. They also have limited opportunity to influence the total referral numbers.

It should be noted that during the focus groups, Case Managers stated that in their opinion they were already at full-capacity based on the current number of people accessing the service. Although strategic level interviewees felt that it was a well-funded pilot, especially in relation to its size, findings from the implementation period suggest a potential mismatch between targets and resources, with the potential need for more staff in all organisations as client numbers increase. Case Managers in Dundee noted that the time between physiotherapy and counselling sessions is already increasing, with some clients still in the service, past the expected 20 weeks. This suggests that the level of staffing resource is not well aligned to targets and expectations of numbers of clients coming through the service.

3.5.2 Marketing

Marketing activity has been viewed by those within the pilot as a key mechanism which can influence whether targets are met, but it is not without its own challenges.

Key considerations for marketing from consultations were that:

• Awareness of the programme was still low among the public and employers in both pilot areas.
• It was felt that the NHS branding should be more apparent as people reported to Case Managers that they thought it was a private service which they would have to pay for.
• Case Managers stated that marketing material should be adapted to more clearly indicate both eligibility criteria and what the service offers.
• Staff stated the use of the word “disability” might be off-putting to some individuals as people with mental health problems or common MSK issues may not view themselves as having a disability.
• Some stakeholders felt that the national 0800 number may be having an impact on the target numbers as people don’t feel comfortable calling/receiving calls from a non-local number.
• Referral numbers are expected to increase as a result of word of mouth from both employers and clients who have been through the service.
• As the pilot is set out, there are three clear and distinct target groups (employed, unemployed and employers). These three groups have different needs and this suggests that targeted marketing approaches are required.
• Marketing should not be a responsibility of existing staff and a specialist role should be created for this purpose.

3.6 Employer engagement

The pilot includes a pathway for employers to access support or advice for their employees who may need additional support. This can include information and advice, work-place visits by trained professionals, or a referral into the pilot for the employees that they are concerned about. However, engagement with employers – and learning about how to effectively engage employers in helping their staff make use of the service - is still at an early stage.

Stakeholders did not offer many views on the employer stream. This could be down to the low numbers coming through the service which could in turn be related to the marketing of the service. Marketing to employers is still at an early stage and there is scope to draw on HWL and Salus’s experience to develop effective local approaches to raising awareness with employers and encouraging them to help staff come forward for appropriate support.

More work is required to further develop this stream and to develop an effective marketing approach that engages employers in the local areas. This is particularly important for the pilot as engagement with employers provides opportunities to engage with individuals upstream.