Moving Forward Making Changes: evaluation of a group-based treatment for sex offenders

Key findings and policy messages

Programme outcomes and improving impact

• There are significant limitations to the data available to quantify the impact of MF:MC. In particular, the lack of a control group means it is not possible to conclusively attribute any changes observed to MF:MC, while staff raised concerns about inter-rater reliability and the validity of some outcome measures.
• However, the monitoring data does indicate a number of positive changes among participants (though these cannot be conclusively attributed to MF:MC), including:
  • A reduction in risk scores over the course of the programme, and
  • Positive changes in scores measuring a range of psychological traits potentially associated with risk of re-offending.
• Overall, staff who contributed to this research felt that, in comparison with previous sex offender interventions, MF:MC had a 'better chance' of being effective as they viewed its overall design as being more evidence-based. While this evaluation cannot robustly assess whether every element of MF:MC's design is best practice, in general, the programme design appears to be informed by current best evidence on 'what works' in terms of appropriate treatment targets and approaches to working with sex offenders.
• However, while staff and stakeholders gave examples where they believed MF:MC had impacted positively on participants, there was also a strong feeling among staff that it was very difficult for the current programme to fully meet the diverse presenting needs of all participants.
• Interviews for this evaluation (particularly with Treatment Managers) identified a range of suggestions for improvements aimed at enhancing MF:MC's impact. In addition to the general resourcing issues, discussed above, these included:
  • providing more external expertise to support delivery
  • assigning national leads on specific approaches and issues to ensure the programme is kept continually up to date in terms of best practice
  • providing guidance on 'over-treatment' (where offenders have attended programmes numerous times with limited evidence of progress), and
  • considering the feasibility of establishing separate groups for men with lower cognitive functioning.

Improving programme delivery

• Overall, MF:MC is being delivered broadly in line with programme design with respect to: length of group sessions; group size; staff roles; staff participation in mandatory MF:MC training; and Practitioner supervision and support.
• However, the evaluation identifies a number of concerns around staff resourcing. Most community teams are currently unable to deliver MF:MC at the recommended intensity due to staffing constraints. Both Community and Prison-based Treatment Managers identified significant challenges covering leave or finding time for staff development within current resourcing. The role of psychological support for delivery within the community also needs to be clarified, and any inconsistencies in access to such resource addressed.
• While overall staff who contributed to the research felt the programme included the key elements of an effective intervention, the Programme Manual was viewed as 'a little vague'. There is also a lack of clarity about the level of deviation from the manuals that is permissible: in practice, sites were adapting content and delivery in a range of ways. While these adaptations were aimed at improving delivery, they introduce scope for inadvertent deviation from the programme design. Findings suggest that the MF:MC manuals should be reviewed, taking account of the various issues and suggestions for improvement identified in this report.
• Alongside this review, consideration should be given to developing an MF:MC 'knowledge hub', where additional (approved) materials to support delivery can be shared and added to in the light of emerging evidence on 'what works' in treating sex offenders.
• While MF:MC appears, for the most part, to be successfully targeting men rated as 'high' or 'moderate' risk, some concerns were raised over whether the current assessment process is always identifying those most suited to the programme. Guidance on assessing internet offenders and deniers for MF:MC in particular should be reviewed and updated in the light of new and emerging evidence on managing and treating these groups.
• Sites varied in whether they offered ongoing support to participants after the group stage of the programme. This suggests that guidance on ending the programme should be clear, and resourcing sufficient to ensure a consistent and appropriate approach to post-programme support.

Improving monitoring and future evaluation

• The monitoring data currently available for MF:MC contains some significant weaknesses in terms of the robustness of the outcome measures included, the perceived usability of the IT system for recording key data, and the timeliness of data collection and entry. Suggestions for improvement include:
  • A systematic review of the outcome measures included within MF:MC, taking account of concerns raised in this evaluation about: the utility and appropriateness of the current psychometric battery; the reliance on self-reported data; and completion of the 'significant others' questionnaire.
  • Considering whether any further guidance and training is required around the completion of SA07 for MF:MC specifically, given the concerns about inter-rater reliability and accuracy raised by interviewees.
  • Reviewing the structure and content of the MF:MC IT system with a view to simplifying data entry and clarifying data outputs, and enhancing data usability for both evaluators and MF:MC teams.
• Any changes resulting from this review process should be supported by refreshed guidance and training on how monitoring data should be entered and used by MF:MC teams and by the Scottish Government, SPS and any external evaluators.
• In addition to reviewing and improving the current monitoring dataset, consideration needs to be given to how longer-term outcomes can be monitored, and to whether/how a control group for MF:MC can be established. Given the relative dearth of robust evidence on the effectiveness of sex offender interventions in general, there is a clear argument for assessing the feasibility of a longer-term experimental or quasi-experimental evaluation of MF:MC. However, an experimental approach would require sufficently large sample sizes to be robust. Men would have to be randomly assigned into the programme or into a control group. Alternatively, a large matched sample of offenders who were not on the programme would have to be identified. Both approaches would be challenging to implement in practice.