Moving Forward Making Changes: evaluation of a group-based treatment for sex offenders

5. Conclusions

This concluding chapter revisits the three primary research questions for this evaluation, and reflects on the key findings with respect to each. It ends with a brief discussion of the wider debate around 'what works' with sex offenders, in order to inform future discussions about the development of MF:MC.

5.1 Is programme integrity maintained?

Manualised programmes are intended to ensure that interventions are delivered consistently across sites and that the integrity of programme design is retained. As discussed in the introduction, MF:MC is supported by a set of manuals, outlining how the programme is intended to operate in terms of: its broad theoretical approach; practicalities, such as frequency of delivery, management and resourcing; content; and evaluation and assessment.

Overall, the evaluation indicates that MF:MC is being delivered with fidelity to the Good Lives Model – that sessions are focused on helping participants re-evaluate their life goals and develop skills to achieve those goals without harming others. It also appears to be being delivered in an engaging and responsive manner, with staff tailoring both the optional modules and more specific content where necessary to address individual needs.

In terms of the practicalities of delivery, while the programme is being delivered on a group basis, in 2-3 hour long sessions, by staff who have received mandatory MF:MC training, most of the community sites were diverging from the frequency of sessions recommended in the manual, running sessions once a week rather than twice. The main reason given for this was staff resourcing, which was also perceived to be impacting to different degrees on the teams' abilities to provide consistent supervision at the frequency recommended in the manual, and to engage in CPD to support delivery. Specific challenges around access to dedicated psychological input to support delivery in community sites means it is unclear whether or not the HSF module is being delivered as intended by the programme designers.

Assessing whether or not programme integrity is being maintained with respect to content is complicated by the fact that a degree of flexibility and responsivity is built into MF:MC. In practice, sites reported spending considerable time identifying their own materials to supplement the manual and support delivery, creating inevitable variations in the precise content being delivered. There was a particular debate about the acceptability or desirability of discussing the offence within MF:MC, which may require further consideration. While some sites actively avoided discussing the offence and saw this as a strength of the programme, others felt it created "an elephant in the room" and had incorporated material from other programmes around this.

The referral and assessment process appears largely to be followed as outlined in the manual, and is largely successful in identifying men who are 'medium' or 'high risk' as rated by Stable 2007 (only 2% of entrants were 'low risk'). However, some concerns were raised about whether the current process is always identifying those most suitable (as distinct from eligible) for the programme, particularly with respect to internet offenders and men who deny their offence. In terms of evaluation, as discussed below, there appear to be a number of ongoing issues with the MF:MC IT system which are limiting the usefulness of the data it contains, both for evaluation purposes and for informing and supporting programme delivery.

5.2 What are the main outcome measures and to what extent have outcomes been realised?

Ultimately, MF:MC aims to reduce reoffending. However, as a programme based in the Good Lives Model, it aims to do so by helping men who have committed sexual offences to develop life goals and the skills they need to achieve these in a pro-social manner. More specifically, it aims to address a number of issues identified as 'Stable Dynamic' risk factors for re-offending (which are also factors that influence the ability to live a 'good life'), including: significant social interests; intimacy deficits; general self-regulation; sexual self-regulation; and co-operation with supervision. In addition, it aims to identify problematic attitudes, motivation to change, and victim empathy, which are all viewed as potential risk factors (although there is doubt in the research literature about the significance of victim empathy in this respect).

Examining recidivism among sexual offenders is very challenging – as Przybylski (2014) notes, "The surreptitious nature of sex crimes, the fact that few sexual offenses are reported to authorities, and variation in the ways researchers calculate recidivism rates all contribute to the problem". Rates of reoffending also tend to be lower for sexual crimes than for other kinds of crime, to differ across different types of sexual offender, and to take a relatively long period of time to occur (Przybylski finds estimates ranging from 5% after 3 years to 24% after 15 years). MF:MC has only been in operation in Scotland since 2014. Even without the challenges noted above, this is an insufficient timescale for any impact on reoffending rates to be observable. As such, Chapter 4 of this report focuses on evidence of potential impact on the risk factors the programme aims to influence, rather than on reoffending itself.

As discussed, there are some significant limitations to the data available to measure impact on these risk factors. In particular, the lack of a control group means it is not possible to attribute any observed changes conclusively to MF:MC, while as discussed below there is room for improvement to the outcome measures included in the programme monitoring data. However, the data that is available is generally quite positive in terms of the perceived impact of MF:MC. Overall, level of risk as scored by Stable 2007 decreases for a majority of men over the course of the programme, while a significant proportion of men also see positive changes in scores measuring psychological traits potentially associated with risk of reoffending. Men expressed positive views about the programme's impact, both within the exit survey and in the qualitative interviews conducted for this evaluation. And staff and Case Managers reported that feedback from others and their own observations indicated it was having a positive impact across a range of risk factors.

Comments from staff and men indicate that the Good Lives approach was viewed as having a " better chance" of engaging men and having an impact than previous programmes, which focused more exclusively on risk and on the offence itself. The views of the three police stakeholders interviewed for this evaluation, however, struck a more sceptical note, and reinforce the need to reconsider what data is collected on outcomes going forward (discussed below).

5.3 Are there difficulties with delivering any aspects of the programme and improvements that could be made?

The final section of each of the previous chapters has included detailed suggestions for improvement to MF:MC. Rather than simply repeat these, this final section reflects on three over-arching areas the evaluation suggests may need further consideration and refinement in order to maximise the potential impact – and to assess this more effectively – going forward.

5.3.1 Programme scope and resourcing

Chapter 2 in particular identified a number of areas for improvement relating to programme resourcing. Ultimately, the resourcing requirements – in terms of staffing, training, expert input, and materials provided – depend on exactly what the programme is trying to deliver. If MF:MC is intended to be suitable for most medium-high risk sex offenders in Scotland, regardless of nature of offence, whether they deny their offence, their psychological profile or cognitive capacity, then the implications of this need to be considered for:

• The expertise required to deliver MF:MC – for example, what psychological approaches teams need to be trained in; what level and type of ongoing psychologist input is required in community settings; and what expert input might be required to ensure the programme is (and stays) at the cutting edge of understanding around treating a wide range of presenting needs.
• The overall level of resourcing – in the light of increasing numbers of RSOs in Scotland, the level of resourcing required to deliver MF:MC needs to be kept under continual review, particularly if there is an aspiration for community sites to deliver sessions twice a week.
• Additions/alternatives to MF:MC groups – comments about groups of men whose needs are not currently perceived to be fully met by MF:MC raise questions about whether there is a need to supplement MF:MC with alternative or additional input for these groups. While it may be possible to meet some presenting needs in a slightly different group programme (such as the specific programme for men with low cognitive function, discussed in Chapter 2, or within the Caledonian System for men who commit sexual offences within relationships), comments by staff and Case Managers suggest that some men (e.g. those with personality disorder, or with Autism) might require both alternative content and delivery (e.g. 1-1 or 2-1) to more fully meet their presenting needs.

5.3.2 Programme manuals

Discussion with Treatment Managers and Practitioners identified a number of areas where the Programme Manual in particular appears to require further development to ensure that it provides sufficient depth to meet diverse presenting needs. The Thinking Styles and Self-Management modules were particularly singled out in this respect. There is also a need to consider strengthening the supporting materials provided to enable staff to deliver MF:MC in an engaging manner, both to reduce the burden on staff who are currently spending time identifying their own materials, and to reduce the potential for unintended variations between sites. This might be supported by the development of an MF:MC Knowledge Hub, where additional materials can be accessed and shared.

At the same time, interviews with MF:MC staff identified an overarching issues around what level of deviation from the manuals is actually acceptable. This requires clarification in general, and in particular with respect to discussing the offence - how far (if at all) is it acceptable to go with introducing additional material around this?

5.3.3 Monitoring and evaluation

This report has identified a number of issues around the monitoring data collected for MF:MC. In order to improve the validity, quality and relevance of the data going forward, there is a need to review:

• What is collected – to ensure that all data items (including all psychometric scales) are clearly mapped to intended programme outcomes and that all measures are valid and robust. This should include considering whether there are any more objective measures that could be incorporated, for example around the presence of deviant sexual attitudes.
• When and how it is collected (and how data input is monitored within and across sites) – to minimise inconsistencies between sites where possible in terms of, for example: the stage at which pre-programme Stable 2007 assessments are conducted; how psychometric tests are introduced and supervised; and who completes the significant other questionnaire.
• How to increase the usefulness of the data collected in ongoing monitoring and evaluation – in order to support the delivery and development of the programme by staff and, as a consequence, make it more likely that data collection will be prioritised (and will therefore be complete and accurate).
• Whether/how longer-term outcomes can be monitored – at present, the data collected only relates to short-term changes in men's reported attitudes and behaviours, immediately following completion of MF:MC. Consideration should be given to the scope for incorporating longer-term follow-up of MF:MC participants – for example, revisiting Stable 2007 and/or other measures with men in the years after completion of MF:MC (Dennis et al, 2012, suggest that offenders should be followed-up for a minimum of five years in the community).
• Whether/how a control group can be established for MF:MC – a recent systematic review of research on sex offender interventions came to the "inescapable conclusion" that there was a need for further randomised controlled trials (Dennis et al, 2012). Conducting such trials is costly and challenging. However, given the impossibility of concluding that any observed positive impacts from MF:MC are the result of the intervention itself, and the fact that recent evaluations of other sex offender interventions have found either inconclusive or negative evidence for any impact on reoffending, there is a clear argument for assessing the feasibility of a longer-term experimental or quasi-experimental evaluation of MF:MC.

5.4 MF:MC and 'what works'?

A detailed summary of the international literature on 'what works' in interventions with sex offenders generally, or an assessment or the extent to which MF:MC conforms to current thinking on 'best practice' (in itself a contested issue) is beyond the scope of this evaluation. However, some brief reflections on emerging evidence and key debates may be useful in informing future discussions about the development of MF:MC. First, it is important to note that internationally, the quantity and quality of evidence in this area has repeatedly been judged to be limited (for example, a 2012 Cochrane review concluded that there were "far fewer than the number [of studies] that would give one any confidence in the findings" – Dennis et al, 2012). Cognitive behavioural approaches have been most frequently evaluated. The evidence is not always conclusive – some recent systematic reviews have found no difference in outcomes between sex offenders who participated in CBT-based or other behavioural programmes and those who did not (e.g. Dennis et al, 2012, Langstrom et al, 2013). However, Kemshall (2013) reports that overall there is evidence of a positive impact from CBT-based approaches. Similarly, Schmucker and Losel's meta-analysis (2015) found a small but significant impact on recidivism rates, and report that Cognitive Behavioural and multi-systemic treatment revealed better effects. Kemshall also reports some support for the Good Lives Model (e.g. Hanson and Yates, 2013) as an example of more positive, future-oriented approaches which are strongly supported in the wider desistance literature (though to date there have been few robust outcome evaluations of GLM-based interventions with sex offenders).

In her review of the evidence, Mann (2014) argues that "a constant theme from the systematic reviews is that we need to get our treatment content more firmly fixed on what we know to be criminogenic needs for sexual offences". The treatment targets she identifies as having a strong link with reconviction closely mirror those included in MF:MC:

• Sexual preoccupation and deviant sexual interests
• Offences supportive attitudes and hostile schemas
• Relationships – lack of intimacy with adults, emotional congruence with children
• Self-regulation – impulsivity, poor problem solving, non-compliance with rules.

Conversely, Mann notes that evidence indicates that 'taking responsibility for offending' appears not to be related to reoffending, which supports the potential for including deniers on such programmes, as in MF:MC. Victim empathy is also not shown to be associated with reoffending, although as discussed earlier, this has been included in MF:MC on the basis of it "being an appropriate way to achieve attitude change" ( MF:MC Theory Manual, 2014: 65).

While there is an emergent consensus on the intermediate outcomes which programmes should target to reduce sexual offending, the most appropriate treatment structures and precise techniques for effecting change in each of these areas remains the subject of considerable debate, however. For example, while many sex offender interventions are based around group work, various authors have highlighted the need for care to avoid unintended negative effects, such as 'deviance modelling' by higher risk offenders to lower risk offenders (e.g. van der Put et al, 2013). While this would support MF:MC's exclusion of low risk offenders, the recent evaluation of the prison-based Core Sex Offender Treatment Programme in England and Wales (Mews et al, 2017) has stimulated further debate about group-work programmes. The evaluation authors, who found that recidivism was higher among those who had participated in the programme than in a matched comparison group, argue that treatment approaches may need to include more individual, as well as group, sessions. However, they also acknowledge that "The study does not reveal the extent to which Core SOTP reoffending outcomes are due to treatment design or poor implementation", highlighting again the contested nature of evidence in this area.

In terms of the efficacy of specific techniques, again the evidence is often tentative at best. Although cognitive behavioural methods are generally regarded as effective, Kemshall notes that not enough is known about "exactly what works, how and for whom" (2014: 6). Mann's 2014 summary of the evidence indicates that:

• Cognitive restructuring therapeutic approaches can be effective in addressing attitudinal risk factors (but the approach can often be misapplied or misunderstood as focusing on 'taking responsibility for the offence')
• Schema therapy may be useful, for example in reducing grievance thinking, entitlement and suspiciousness, though the research is limited
• Empathy training (rather than 'victim empathy') can change attitudes, and
• Mindfulness may have an impact on emotional regulation (though evidence is early).

In summary, the research base on sex offender interventions is still developing – definitive conclusions about 'what works' are difficult. If interventions like MF:MC are to remain 'evidence informed', this will require ongoing engagement with emerging evidence on good practice.