Moving Forward Making Changes: evaluation of a group-based treatment for sex offenders

3. Participation in MF:MC

Key findings

Referral and assessment process

• Interviews with MF:MC staff indicate that the referral and assessment process appears largely to be followed as outlined in the manuals. However, there is some variation between sites with respect to: the precise stage at which the pre-programme SA07 is conducted; who conducts the pre-programme SA07; the process for determining suitability (as distinct from eligibility); and the assessment of deniers (an issue on which it was suggested further guidance or training might be required).
• For the most part, MF:MC is successfully targeting men rated as 'high' or 'moderate' risk – just 2% of participants were rated 'low risk' based on their pre-programme Stable 2007 score.
• However, some concerns were raised about whether the current process always identifies those most suitable for the programme, including:
  • whether SA07 is always completed accurately and based on sufficient information to make a realistic assessment of risk
  • whether SA07 omits a number of key risk factors, specifically around thinking styles and attitudinal risk factors, and
  • the perceived inadequacy of the current process for identifying whether or not internet offenders are suitable for MF:MC.

Participation, attrition and completion

• The proportion of men who refuse to participate in MF:MC was generally believed by staff to be low, particularly in the community where it is usually a condition of men's CPO. Participants cited concern about confidentiality as one reason for refusing to participate in MF:MC when in custody.
• Limitations in the monitoring data for MF:MC mean it can only provide an imperfect estimate of the number and profile of men starting, completing and leaving MF:MC. However, taking these limitations into account an estimated:
  • 911 men had started MF:MC, most (81%) in a community site
  • between 230 and 266 were recorded as having fully completed the programme, and
  • 97 men were recorded as having left the programme before completing all their prescribed modules.
• Where a reason for leaving early was recorded, the most common were: insufficient time on the man's order or time left in custody to complete; being recalled or having breached their order; and personal reasons such as poor health or change in employment circumstances.

Attendance and engagement

• Attendance at MF:MC sessions was reported (by staff) to be very high. In terms of active engagement, participants' comments suggested that starting group work and new members 'rolling on' to the programme are both points where, for different reasons, engagement may sometimes be lower. However, overall, participants and staff both felt that engagement was primarily influenced by an individual's own motivation, rather than programme stage, offence type, age or cognitive ability.

3.1 Introduction

This chapter discusses the level and nature of participation in MF:MC, including:

• the referral and assessment process and the extent to which this is believed to identify the men who are both eligible and suitable for the programme
• the level of participation, completion and attrition on MF:MC to date, and the reasons for non-completion
• perceived engagement with MF:MC.

The information in this section is based on a combination of feedback from Treatment Managers, Practitioners, participants and stakeholders, and analysis of the monitoring data recorded by each delivery site.

3.2 Referral and assessment

3.2.1 Outline of the intended process

The process of referral to and assessment for MF:MC outlined in the manuals differs between prison sites and community sites. The different pathways are summarised in Figure 3.1, below. In prison, referral to the programme is linked to the Integrated Case Management process. Men are assessed for suitability for a range of programmes available through SPS through the generic programmes assessment process, which involves:

• a file review to examine offending history, risk, need and responsivity factors
• an interview (covering their offending behaviour, previous interventions, motivation and responsivity issues),
• discussion at the Programmes Case Management Board to determine suitability for the programme (including discussion of risk level, denial, time remaining in custody, motivation level, and what needs to be put in place to ensure the programme is responsive to their needs).

Once identified as suitable for MF:MC, an appointment is arranged for the man to be interviewed by MF:MC practitioners, where they carry out a more detailed assessment, using the Stable and Acute 2007 tool ( SA07, see below), upon which eligibility for entry onto the programme is predicated. In custody, a completed SA07 should be available for the Programmes Case Management Board – if this is not available, an assessment would be requested from the prison-based social work team.

In the community, a potential MF:MC participant is identified following a conviction in court for a sexual offence. A member of the Criminal Justice Area Team will complete a Criminal Justice Report on the man in question, which may recommend that MF:MC is imposed. The report writer may contact the MF:MC team to discuss the man's risk assessment and suitability for the programme prior to completing the report. If the court has already imposed attendance on a sex offender treatment programme prior to assessing eligibility, the Area Team manager will allocate a Case Manager for the man, who will carry out the relevant assessments. In either case, the referral will be reviewed by the MF:MC Treatment Manager to ensure it is appropriate.

Figure 3.1 Standard referral and assessment process (taken from the MF:MC Assessment and Evaluation manual)

Scottish Government/Scottish Prison Service - Crown Copyright

The assessment process for internet offenders as outlined in the manual is different to that for contact offenders. The Assessment and Evaluation manual provides a 'decision tree' for determining suitability of internet offenders for MF:MC, taking into consideration both their SA07 and the Risk Matrix 2000 ( RM2K) scores, plus consideration of aggravating factors (including criminal history, access to children, and types of images viewed). Following this process may result in a recommendation for MF:MC, or for treatment through case management alone. The decision tree is shown in Annex B, Figure B.1.

3.2.2 Extent to which the assessment process follows the manuals

The referral and assessment process appears largely to be followed as outlined in the manuals. However, there is some variation between sites with respect to: the precise stage at which the pre-programme SA07 is conducted; who conducts the pre-programme SA07; the process for determining suitability; and the assessment of deniers.

First, the stage at which the pre-programme SA07 assessment is carried out varies between the community sites. Some carry it out at the court report stage, while other sites appear to wait until the man is further into his treatment order (in some cases having completed some of the pre-group work), on the basis that it is more accurate and more useful in formulating treatment needs at that stage:

We don't do the SA07 at the court report stage…. Here it isn't done until the man is three months into the order. At that point, there is more information to populate the SA07 and at that point you can look better at whether the man is suitable or not.

(Treatment Manager 14)

Second, who carries out the pre-programme SA07 appears to vary between community sites – while most commonly this assessment was conducted by general Criminal Justice Social workers (who carry out court reports), there were also cases where the SA07 was completed by the MF:MC team or the police, either independently or jointly with Case Managers or with each other (i.e. the MF:MC team and Police Scotland).

The process for determining suitability for the programme also varies between community sites. In some sites, this is assessed through a discussion between the Case Manager and the Treatment Manager, while in others it involves discussions among a wider team (in some cases operating as a "screening panel"). Community sites generally reported that the process of assessment worked well in their own area: there was no feedback to suggest that one of these approaches was more effective than the other.

Some sex offender interventions explicitly exclude those who deny their offence ("deniers") from participation. However, the MF:MC Theory Manual states that "no offender should be refused entry to MF:MC due to denial". Practitioners from both custody and community settings nonetheless felt that deniers can have a negative impact on group dynamic. In response to this concern, MF:MC teams within SPS have developed a specific assessment process for deniers, which seeks to establish whether these issues can be managed within a group setting, alongside a "readiness process" which helps prepare these participants for group interaction. Community sites do not appear to have such an approach in place, and feedback from practitioners highlighted a need for further training and support on how to assess whether or not "deniers" are suitable for MF:MC.

3.2.3 Views on the assessment process: is it reaching the right men?

MF:MC is intended to target men who are rated as posing a 'high' or 'moderate' risk of re-offending. In both prison and community, eligibility for MF:MC in terms of risk-level is primarily assessed using the Stable and Acute 2007 ( SA07) tool. SA07 aims to measure two sets of dynamic factors – 'acute' factors, which are characterised as environmental and intrapersonal stresses known to be related to imminent sexual reoffending and subject to potentially rapid change (e.g. intoxication, disengagement from supervision), and 'stable' factors, which are personal skill deficits or behaviours that may be addressed through treatment and supervision (e.g. substance misuse, negative peer influences, antisocial attitudes). While the SA07 tool measures both stable and acute risk factors, the MF:MC dataset only records Stable 2007 scores, since these are the factors that are considered capable of being changed through a process of "effortful intervention" ( MF:MC Assessment and Evaluation manual).

Stable 2007 produces a score for each of 13 risk factors, which are then combined to classify men as high, moderate or low risk based on their Stable dynamic risk factors. The table below shows the Stable 2007 risk profile of men recorded in the monitoring data as having started MF:MC. Before commencing the programme, 68% of men were classed as moderate risk, 30% were high and only 2% were low (see Annex C, Table C.1 for breakdown by site). This suggests that, for the most part, MF:MC is successfully targeting men rated as 'high' or 'moderate' risk.

However, among Treatment Managers, there were mixed views on the effectiveness of the assessment process in identifying the 'right' men for MF:MC – in other words those who MF:MC staff felt were most likely to engage with and benefit from the programme. One view was that the assessment process works well:

We have a screening panel, and I think that process works well. There is full discussion about the men, their circumstances, responsivity, their offending…it's a kind of fluid process that everyone who is there takes part in.

(Treatment Manager 12)

However, a number of issues were also raised, including:

• Whether SA07 is always completed accurately ('inter-rater reliability'). Treatment Managers and Practitioners indicated that the results of SA07 assessments sometimes varied depending on who conducted them. They cited examples of cases where MF:MC teams received referrals for men rated as 'moderate' or 'high risk' by Case Managers, but on reviewing the evidence the team considered them to be low risk, or vice versa. In part, this was attributed to the level of information the Case Manager had to make their assessment at the time.

By the time we formulate a case we will have a good understanding of how he has come to commit a sexual offence, and can pick up on things that the SA07 hasn't. That's where the discrepancy lies. The assessment asks them quite intrusive questions, but it could be the first time the social worker has met the man, so he may not open up as much as he would with us.

(Treatment Manager 3)

However, it was also suggested that, on occasion, Case Managers may classify a man higher on SA07 than the MF:MC practitioners would, because the Case Manager is keen for the man to get onto the programme. In these cases, there would typically be a discussion between the practitioners and the Case Manger to agree whether or not the man is suitable for the programme. It is worth noting in this context that refresher training on SA07 was rolled out across Scotland over the last year, which may help to improve consistency between assessors.

• Whether the content of SA07 is adequate to identify men who are suitable for MF:MC. While overall SA07 was perceived to be a useful tool, practitioners and Treatment Managers questioned whether it fully captures all elements of risk and treatment need for this offender group. In particular, it was suggested that it does not adequately cover thinking styles and attitudinal factors that are relevant to risk, such as problematic attitudes towards child abuse or rape, or feelings of entitlement. These attitudes were considered by practitioners to be critical risk factors and key elements of the MF:MC treatment plan. Reflecting these perceived gaps, in a small number of cases, staff reported they had allowed men on the programme who were rated as 'low risk' based on their SA07 score (as discussed above, overall 2% of men recorded as starting MF:MC were classed as Low risk).

Somebody might be scoring quite low, but we know that attitudinal stuff might be the key factor, and that doesn't get captured by Stable...sometimes the only way we will find out [if there are attitudinal problems] is by bringing them into the programme.

(Treatment Manager 9)

The Risk for Sexual Violence Protocol ( RSVP) was suggested as a possible alternative or additional tool for identifying risk factors and supporting treatment formulation.

• The adequacy of the process for identifying whether or not internet offenders are suitable for MF:MC. There was a broad consensus among Practitioners and Treatment Managers that the "decision tree" included in the manual was not an effective approach to assessing internet-only offenders' suitability for MF:MC. Although it was also acknowledged that there is general uncertainty about the best approach to assessing and treating internet offenders, there was a desire for greater clarity on this issue (and on alternate treatment options for those deemed unsuitable). There was concern that internet offenders who are in reality 'low risk' might be inappropriately assessed as eligible for MF:MC via the current process.

There remains a lack of clarity around which internet offenders are the right offenders for MFMC and which ones aren't. If they're not suitable for MFMC, there is probably a lack of guidance or understanding around what to do with them then.

(Treatment Manager 11)

3.3 Programme refusal

Data on refusals is not recorded in the MF:MC IT System, since refusal would usually occur at the sentencing stage, before men are entered onto the MF:MC system. However, feedback from community sites suggests that it is rare for men to refuse MF:MC, as to do so would mean going against the requirements of their sentence.

In prison, it was estimated (by a Treatment Manager) that approximately one third of men refuse to participate in any treatment programmes, including MF:MC. SPS staff linked refusal with either denying the offence altogether, or with men approaching parole and worrying that participation might extend their time in prison. Men interviewed for the evaluation also indicated that they had previously refused to take part in MF:MC or similar programmes in custody due to concerns about confidentiality:

The first time I heard about it was in [prison site] and I wouldn't do it, because everybody was talking about it in the exercise yard and pointing the finger at folk [who were on it] so I turned it down.

( MF:MC Participant 13)

Particular prisons also cited difficulties filling spaces on their groups from the national waiting list, because of a perceived reluctance among men to transfer from their current prison to a different prison. However, there was a degree of scepticism among the small number of Police Scotland stakeholders interviewed about this issue – they speculated that some men may use a reluctance to transfer prisons as an excuse to avoid participating in MF:MC.

3.4 Participation and attrition

This section uses data from the MF:MC IT system to examine the number and profile of men participating in MF:MC and the numbers leaving the programme without completing. When interpreting this data, a number of limitations should be borne in mind:

• The data is known to underestimate the total number of men who have been on the programme and the numbers who have completed MF:MC since its inception in 2014. There are three main reasons for this:
  • A number of delivery sites began running MF:MC before the IT system was in place, and have indicated that men who started before this point are not captured in the monitoring data. ^[8]
  • The IT system's structure precludes cases from being recorded as complete until all post-programme data is available, including the post-programme SA07 , psychometrics and significant other questionnaires. Treatment Managers reported that in a number of cases men have completed all of their assigned modules, but they cannot generate a 'completion date' in their record because some of this data is missing. ^[9] In some cases, these men have been coded as 'removed' from the programme, but the verbatim reason given for removal is that they completed the programme but some element of the post-programme data was missing. In other cases, they may appear still to be on the programme (i.e. their cases is 'open' because of missing post-programme data, even though they have completed all the required modules).
  • The IT system's structure means that in the event that a man re-enters MF:MC (for example, because his treatment needs were deemed not to have been fully met), his previous data is over-written. The system is a record of where they are at on MF:MC at that point in time, but will not show whether they have fully completed a previous run of MF:MC. There may therefore be men who have completed a full round of MF:MC group-work, but are not recorded in the data as such because they have subsequently begun a new 'run' of the programme.
• Data on the numbers leaving MF:MC is also unclear – a number of cases are coded as having been 'removed', but the reason recorded for removal indicates they are unlikely to have started MF:MC group work in the first place (for example, MF:MC was not a condition of their sentence).

Given these limitations, the monitoring data can only provide an imperfect estimate of the number and profile of men starting, completing and leaving MF:MC.

3.4.1 Numbers participating in MF:MC

The data provided to the evaluation team includes records for 1,949 men, of whom:

• 911 men had started MF:MC ^[10]
• 619 did not start the programme because they were deemed unsuitable ^[11]
• the status of the remaining 419 is not clear from the data, but they are assumed not to have started. ^[12]

Of the 911 men recorded in the data as being suitable and having a start date for MF:MC, the majority were recorded as participating in MF:MC in a community site (81%, n = 736), with 19% (n = 175) participating in one of the four custodial sites (see Annex C, Table C.2 for a breakdown of numbers of participants by site).

The age profile of men participating in MF:MC was fairly broadly spread for both community and prison sites (see Annex C, Figure C.1). Younger men participating in MF:MC were more likely to be classed as High risk (38% of 16-24 year-olds) compared with older participants (20% of over 55 year-olds were assessed as High Risk based on their pre-programme Stable 2007 score) ( Annex C, Figure C.2).

3.4.2 Numbers completing MF:MC

Establishing a definitive number of participants who have completed MF:MC to date is not straightforward. Leaving aside the issue of participants who are not recorded in the MF:MC IT System at all, there is no single field in the IT System which definitively records completion. Applying various possible definitions indicates that, of the 911 men recorded as participating in MF:MC, between 230 and 266 appear to have fully completed the programme, ^[13] while 97 men appear to have left the programme before completing it. ^[14] Calculating an accurate completion rate for the programme does not appear to be feasible, both because establishing a definitive number who have completed it is difficult, and because the programme is not a fixed length (so it is not clear how many of the 911 men who started the programme should have completed it by now).

The pre-programme risk profile of men completing the programme broadly matched the pre-programme risk profile of all participants – high risk men did not appear particularly more or less likely to complete than moderate risk men.

3.4.3 Reasons for leaving MF:MC before completing

The reason for leaving the programme was not recorded for all those who appear to have left early (which again indicates potential issues with entry of the monitoring data). However, where reasons are available, the most common were: insufficient time on the man's order or time left in custody to complete; being recalled or having breached their order; and personal reasons such as poor health or change in employment circumstances (Table 3.1). In cases where men do not complete the programme in the community, the order is returned to court for resentencing. The outcome/disposal made would depend on the nature of the non-compliance and on whether the order was clearly made as a direct alternative to custody.

Table 3.1: Reason for leaving programme

Stage of removal	Number of men	% of men who left
Insufficient time on order/in custody to complete	16	16%
Recalled or breached CPO	16	16%
Personal reason (health, employment, etc.)	12	12%
Chose to leave group ('deselected')	10	10%
Other reason/unclear reason	10	10%
Removed from group	6	6%
Doing 1-1 or 2-1 work instead	1	1%
No reason recorded	26	27%
Total	97	100%

Notes to table: based on verbatim 'reasons for removal', coded into broad categories by the report authors

The stage at which 'non-completers' left the programme was not recorded for around half (48%) of the 97 who left early (see Annex C, Table C.3). Of the remainder:

• 5% were classed as leaving at the point of pre-programme psychometrics, and therefore had not actually commenced any group work
• 38% left during group work (28% during Essential and 10% during Optional modules), and
• 8% were recorded as leaving at the post-programme stage (either psychometrics, risk assessment of exit interview).

3.5 Engagement

Engagement with an intervention can be assessed on a number of levels: the level of attendance at sessions; the extent to which men actively take part in and contribute to sessions; and the extent to which they are deemed to be "genuinely" engaging with treatment in terms of their motivation to change (which is, of course, more difficult to accurately gauge).

Practitioners reported that in general, attendance on MF:MC was very high and that it was rare for men to miss any meetings. This finding was supported by interviews with men who, for the most part, said that they had not missed any MF:MC group sessions unless there were exceptional circumstances that they could not avoid.

In terms of specific stages of MF:MC where men might be less engaged, some participants indicated that they had felt less motivated to engage at the beginning, because they felt uncertain about what to expect or worried about discussing their offence and their lives in a group setting. In general, these fears appeared to have been allayed relatively early in the programme after meeting with staff and attending the initial group meetings. After any initial concerns had been addressed, men generally reported feeling strongly engaged with MF:MC throughout the duration of their time on it (although, as discussed above, the repetition of concepts when new participants 'roll on' to MF:MC was associated by both staff and participants with some waning attention among other group members). Among men who had taken part in custodial settings, there were also examples of some continued reluctance to engage openly in group discussions due to a perceived lack of confidentiality.

[In prison], as soon as you disclose anything that someone didn't know that was it. By the time you get back to the hall, the whole hall would know what you'd been saying at the groups.

( MF:MC Participant 7)

The monitoring data also includes MF:MC Practitioners' assessment of how engaged participants were in group sessions for each module they have completed. As shown in Table 3.2, across most modules, most participants' attitudes were rated as 'supportive', with around 20-25% 'neutral' (presumably indicating that their level of genuine engagement was difficult to gauge). Very few participants (3% or less for each module) were recorded as displaying 'non-supportive' attitudes within group sessions. The only module where a higher proportion were recorded as 'neutral' rather than supportive was the 'social support' optional module. However, as this is only based on data for 78 men, some caution should be applied in extrapolating from this to the conclusion that participants are less engaged with this module.

Table 3.2: Perceived engagement by module

		Neutral	Non-supportive	Supportive	Base (number of men with record of engagement for that module)
Essential modules
Entry module	%	25	3	72	540
Re-entry module	%	24	2	74	42
Introduction to Thinking Styles and Self-Management	%	20	3	78	476
Essential HSF module	%	22	2	76	456
Discovering Needs	%	18	3	79	404
Moving to the Future	%	17	3	81	352
Optional modules
Motivation	%	30	0	70	23
Social support	%	39	3	59	78
Relationship skills	%	20	2	78	130
Self-management	%	21	2	77	145
Thinking Styles	%	24	3	74	144
HSF	%	21	2	77	343
Empathy & Perspective Taking	%	23	0	76	56

Overall, programme participants and staff both felt that engagement was primarily influenced by an individual's own motivation, rather than programme stage, offence type, age or cognitive ability.

3.6 Suggestions for improvement

Feedback from interviewees and analysis of monitoring data outlined above indicates a number of potential areas for improvement. First, in terms of the assessment process, there is a need to:

• Review the process and guidance for assessment of internet offenders. Work that may inform this is already underway – the Risk Management Authority have been commissioned to carry out a review of international research literature on internet offending, and to look at the trajectories of online sexual offenders in Scotland. This research is due to report in Summer 2018. In light of the lack of current consensus over the most appropriate approach to treatment of internet-only sex offenders, there is also a need to ensure that any guidance on this issue is kept up to date in light of new evidence.
• Consider whether further guidance or training on completing SA07 is required, to address the issues around inter-rater reliability and accuracy identified by MF:MC staff and treatment managers.
• Consider whether or not additional tools (e.g. RSVP) are required to supplement the SA07 as a tool for identifying and formulating treatment plans for offenders who may be suitable for MF:MC, in light of perceived gaps in the information captured by this tool.
• Provide further guidance on assessing deniers for suitability for MF:MC, perhaps drawing on the system already established in SPS.

In terms of suggestions for increasing engagement in the programme, potential changes (some of which have already been identified in Chapter 2) include:

• Considering ways of reducing repetition, particularly when new men 'roll on' to groups
• Increasing the frequency of sessions in the community, to reduce the length of time men have to wait before they are 'in focus' discussing their assignments.

• Providing further guidance on managing deniers within MF:MC, to avoiding the risk of disruption to the rest of the group (which could undermine other participants' engagement).

In general, the difficulties accurately identifying numbers starting, leaving early, and completing the programme indicate a need to review how these outcomes are captured within the monitoring data. One possibility would be to incorporate a single 'Final programme status' field, to be completed for every case but which is not dependent on completion of other post-programme data. This field should (provided data is being entered regularly) then be up to date for those who either did not start, left early, or have completed all required modules at any given point in time.

There is also a need to gather clearer data on programme attrition (given that the 'reasons for removal' field currently appears to include a number of cases that either did not start the programme or completed successfully), and to monitor this as the programme develops, to ensure that mitigating actions can be put in place.