Fair Start Scotland - individual placement and support review: findings

Annex A. Fidelity reviews: Process and outcomes

Review Methodology

Introduction to Fidelity Reviews

Individual Placement and Support (IPS) is a specific type of employment service. Research has demonstrated that this method of supported employment is the most effective approach for helping people with serious mental illness who want to work in regular jobs. Because research has consistently shown that IPS is more effective than other types of employment programs, it is called an evidence-based practice.

A fidelity scale is a tool to measure the level of implementation of an evidence-based practice. The IPS Supported Employment 25-item Fidelity Scale defines the critical elements of IPS in order to differentiate between programs that have fully implemented the model and those that have not. As demonstrated through research, high-fidelity programs are expected to have greater effectiveness than low-fidelity programs^[83]. The IPS Supported Employment Fidelity Scale is a guide for commissioners and delivery teams to plan for and achieve better employment outcomes.

Understanding the scores: The IPS Supported Employment 25-item Fidelity Scale is divided into three sections: Staffing, Organisation, and Services. Each of the 25 items is rated on a 5-point Likert scale, ranging from 1 (no implementation) to 5 (full implementation), with intermediate numbers representing progressively greater degrees of implementation. The scale has a maximum score of 125 (25 items x 5 maximum point each = 125). Programs that fully implement IPS according to the scale criteria have shown to have higher competitive employment rates than those that do not.

There are 4 main categories to define service practice based on a fidelity review and scores are shown in the table below:

Using the scores: Reviewers encourage the service to consider the review and report to identify areas of strengths and area for improvement. The service needs to then draft an IPS fidelity action plan based on recommendations in the report. Ideally action plans will be discussed in local IPS steering committee meetings (where such forums exist) so that committee members can suggest strategies to improve IPS implementation.

Again, we must stress that good employment outcomes are correlated to a rating of at least good fidelity.

Fair Start Scotland Considerations

In Fair Start Scotland LOTS with low or no IPS uptake, we used our guided self-assessment methodology for the Prime Organisation. This draws on the Dartmouth "Agency Readiness for IPS Checklist" and will:

• Engage IPS delivery staff, clinical teams and clients
• Map current supported employment delivery against the evidence-based 25-point IPS Fidelity Scale
• Highlight practical steps the service and clinical team can take to become "IPS-ready"

The rationale for providing a readiness review rather than full fidelity review in these circumstances is that it can focus on practical steps that the Prime Organisation could take to start to deliver IPS, rather than potentially demoralising and confusing multiple supported employment service providers at every lot who are not currently delivering IPS with a low IPS fidelity score.

Our initial conversations with providers and review of the data have highlighted that there are multiple lots that are not currently delivering IPS services. We have mapped below the various providers, their current delivery of IPS and noted which Prime Organisations gained a desk based guided self-assessment and which lots received a formal IPS compliance fidelity review.

Covid-19 impact

During the period of COVID-19 in-person IPS fidelity reviews are not recommended. A virtual IPS review can be completed to identify what services are doing well and areas they can improve to help people gain successful employment. The intent is to mimic a fidelity review, to learn about current services and provide a report aimed at continuous service improvement.

The IPS reviewer will complete a fidelity report with program data, interview responses, and client file information. Usually the data is from the previous 6 months of delivery. The reviewer will work with the IPS supervisor to have a complete schedule prior to conducting virtual interviews in which they will meet key stakeholders, including service users and virtually observe employer engagement. The reviewer will summarise the IPS program strengths, provide recommendations and offer a teleconference with the IPS supervisor and other key staff to provide quality improvement consultation.

Considerations:

Due to Covid-19 impact there are 8 items from the 25-item fidelity scale which will be impacted and unable to be delivered on for the last 6 months (see detail below). The total impact of Covid-19 is a loss of potentially 36 points from 125 points on the fidelity scale.

We can identify and recognise a service attempts to commit to the items with other virtual approaches. We can also review pre Covid-19 workings and evidence. However, we cannot state the "virtual fidelity review" is a true and accurate representation of the service IPS fidelity given the impact of Covid-19.

There are 7 items with full 5 points impact due to covid-19:

Item 5: integration of Employment Specialists with Mental Health teams (teams were unable to meet face-to-face)

Item 6: collaboration with JCP/DWP (this is unlikely as staff were drawn to other work)

Item 15: face-to-face contact with employer within 30 days (face-to-face contact not possible)

Item 17: frequent employer contact of 6 face-to-face meetings a week (face-to-face contact not possible)

Item 18: quality of employer face-to-face contact (face-to-face contact not possible)

Item 23: time unlimited follow along support with specific client face-to-face prep and review once in work (face-to-face contact not possible)

Item 24: community-based services with Employment Specialists spending 65% of their time in the community (community-based working not possible)

There is 1 item with 1-point impact due to covid-19:

Item 8: IPS supervisor cannot undertake community observation and field mentoring of staff

Summary:

The Fidelity Review Report will provide a score that is indicative of what could have been possible if we can find suitable pre Covid-19 evidence as well as a final score based on available evidence. We recognise that the final score could be affected with an immediate loss of 36 points due to the impact of Covid-19.

The desk based guided assessment will provide a provisional score based on conversational review and offer real time coaching for the Prime Organisation on what each IPS fidelity item entails.