Moving Forward Making Changes: evaluation of a group-based treatment for sex offenders

4. Programme outcomes

Key findings

Limitations to outcomes data

• There are significant limitations to the data available to assess outcomes from MF:MC:
  • The lack of a control group means it is not possible to conclusively attribute any changes observed to the impact of MF:MC
  • There are perceived issues around inter-rater reliability, validity of selected outcome measures, and high levels of missing data
  • Qualitative interviews with participants were limited in terms of the number of case study sites (5 of 15) it was possible to visit, and were arguably unlikely to reveal potential negative impacts from the programme.

Quantitative outcome measures

• Although changes cannot be conclusively attributed to MF:MC, the quantitative data does indicate a number of positive changes in participants:
  • A reduction in risk scores over the course of the programme. 69% saw some level of reduction in their risk score (although in 10% of cases, risk scores increased, albeit typically by only 1 or 2 points).
  • Positive changes in scores measuring a range of psychological traits potentially associated with risk factors for re-offending.
  • Men expressed positive views about its impact. 85% of men who completed an exit survey on leaving MF:MC said they thought it would stop them re-offending in a similar manner.

Staff, participant and stakeholder perceptions

• MF:MC staff and Case Managers found it difficult to assess what impact MF:MC is having. However, overall they felt that feedback from men, significant others, other professionals, and their own observations indicated that it was having a positive impact. It was also suggested that the design of the programme meant it had a "better chance" of being effective than previous programmes. The small number of Police stakeholders that we interviewed, however, were more sceptical about its likely impact, questioning whether participants might project an overly positive picture of their progress within MF:MC groups.
• Participants and staff gave examples of where they believed MF:MC had impacted on men's ability to develop and sustain healthy social relationships, to regulate their emotions, to co-operate with supervision, and to understand and change problematic attitudes. The programme's impact on sexual self-regulation was more difficult to explore and the evidence on perceived impacts more mixed, however.
• Key features participants perceived may be contributing to perceived positive impacts were the fact that it was group based and the overall theoretical approach. Views on whether specific modules and approaches were particularly helpful varied, but the Thinking Styles, Self-management and Relationship modules were singled out by participants as particularly useful.
• Overall the ability to tailor MF:MC to different needs was praised. However, Treatment Managers and Practitioners identified a number of groups of men whose needs they felt were not fully met by MF:MC at present, including men with: Personality Disorder and/or acute emotional difficulties; early maladaptive schemas; Autistic Spectrum Disorder, and men who are Internet offenders; paedophiles who view their sexual pre-occupation with children as an acceptable orientation; commit sexual offences in a relationship; or have already completed multiple interventions.
• Various suggestions were made as to how the programme might be strengthened or adapted to meet some of these needs. However, there was also a perception that at present MF:MC is expecting delivery staff to have an unrealistically wide range of knowledge and skill across diverse psychological approaches and presenting needs, and that this needs to be re-examined going forward.

4.1 Introduction

This chapter explores MF:MC programme outcomes. It summarises the key outcomes the programme is intended to influence and discusses what data is available to assess whether or not these have been met. Quantitative monitoring data are used to look at distance travelled with respect to indicators of dynamic risk factors and psychological traits, while qualitative data from participants, staff and stakeholders illustrate perceived impacts.

In addition, the chapter:

• examines views on the key factors influencing programme impact, including aspects of the programme design and delivery, and factors external to the programme
• considers whether or not the programme is believed to be effective in meeting the needs of diverse participants, and
• outlines a number of suggestions for improvement aimed at enhancing programme impact and improving the measurement of impact.

4.2 What outcomes is MF:MC intended to influence?

The MF:MC Theory Manual describes the programme as aiming "to assist participants to lead a satisfying life that does not involve harming others". In other words, it attempts to develop participants' skills and capacities to achieve 'primary goods' (including happiness, community and relationships) in positive, pro-social ways, which should, in turn, reduce their risk levels. As already noted, the precise goals of the programme will vary depending on the assessed needs of each participant (determined during the formulation stage). However, the overall treatment targets have been mapped onto the static dynamic risk factors identified in the Stable 2007 risk assessment protocol, and include:

• Significant social influences – the Social Support module aims to help participants develop a social network of individuals likely to "promote pro-social values, encourage self-control strategies, and provide material and emotional support" (Theory Manual, 2014: 58)
• Intimacy deficits – the Relationship Skills and/or Thinking Styles modules address a range of intimacy deficits which are characteristic of sex offenders, including capacity for relationship stability, emotional identification with children, hostility towards women, general social isolation/loneliness, and lack of concern for others
• General self-regulation – the Self-Management and/or Thinking Styles modules address issues around general self-regulation (which has been shown to be associated with re-offending), including impulsive behaviour, poor problem-solving, and negative emotionality and hostility (feeling hostile, victimised and resentful)
• Sexual self-regulation – sexual deviancy and sexual pre-occupation have been found to be strong predictors of sexual recidivism. Both are addressed in the Healthy Sexual Functioning and/or Self-Management modules.
• Co-operation with Supervision – non-compliance with supervision is one of the best predictors for sexual and non-sexual recidivism. The Managing the Future module aims to help participants engage meaningfully with those involved in their case management.

In addition, MF:MC also aims to influence three other key risk factors, which are not covered by Stable 2007:

• Attitudes – the Thinking Styles and Victim Empathy/Perspective Taking modules challenge 'cognitive distortions' believed to play a role in sex offending, such as beliefs of entitlement, rape supportive attitudes, and child abuse supportive attitudes
• Victim empathy and perspective taking – although the link between victim empathy and reoffending is disputed in the literature (Hanson and Yates, 2013), the MF:MC Theory Manual indicates that most programmes include some work on this issue on the basis of it "being an appropriate way to achieve attitude change" (2014: 65).
• Motivation – evaluations of other sex offender interventions indicate that developing skills and strategies to avoid reoffending is not enough if participants are not fully motivated to change. The Motivation module of MF:MC aims to increase participants' motivation to develop pro-social behaviours.

4.3 What data is available to assess MF:MC outcomes?

This evaluation makes use of two main sources of data to explore MF:MC outcomes: quantitative data taken from the MF:MC IT system, which records data for each man referred to the programme, and qualitative data derived from interviews and workshops conducted by the evaluation team with Treatment Managers, Practitioners, men who have participated in MF:MC and stakeholders. In discussing and interpreting these two sources of data, it is important to be clear about their limitations.

4.3.1 Quantitative data

There are a number of measures included in the MF:MC monitoring data that could be used to examine outcomes:

• Pre- and post-programme Stable 2007 scores can be used to assess changes in the assessed risk level of men participating in the programme
• Pre- and post-programme scores across 8 psychometric scales can be used to assess any changes in specific psychological traits believed to be associated with risk of sex offending (details of the scales included are provided in section 4.4)
• Data from an 'exit interview' with participants, which asks them to comment on the perceived impact they believe MF:MC has had in a number of areas, including whether or not they feel it will stop them re-offending in a similar manner.
• Data from a 'significant others' questionnaire, administered at the end of the programme, provides an indication of whether other people have observed any changes in participants.

As discussed in Chapter 1, the extent to which any conclusions about impact can be drawn from the quantitative data collected for MF:MC is severely limited by the lack of a control group: without a sufficiently large comparator group of men who have not been through MF:MC, it is not possible to establish that any observed changes are a result of the programme.

In addition, interviews with Treatment Managers and Practitioners also identified a number of concerns about the validity and/or reliability of each of the quantitative measures identified above.

• In principle, Stable 2007 was believed to be a valid tool for establishing 'stable' dynamic risk factors (although, as discussed in Chapter 3, it was suggested that it did not capture all relevant risk factors). However, as noted, Treatment Managers and Practitioners expressed concerns about inter-rater reliability and accuracy of completion. It was suggested by Prison Treatment Managers that social workers completing SA07 were often basing their assessments on a fairly brief interview with the man, rather than drawing on all the available evidence (for example, feedback from other staff) that might enable them to assess dynamic risk.
• With respect to the eight psychometric scales administered before and after the programme, MF:MC teams questioned the validity of some of the scales selected for use with sex offenders, particularly the 'Hostility towards women' and 'Sex with children is justifiable' scales. While one view was that social desirability bias is an inherent problem with psychometric scales, these scales were considered particularly poor in this respect – it was suggested it was " transparent" what answers men were " meant" to give. These views are echoed in academic debate about the use of psychometrics – for example, Russell and Darjee (2013) argue that psychometric tests for use with sex offenders are:

"transparent and prone to self-report bias. There is conflicting evidence on the relationship between self-report psychometrics and recidivism … Changes in scores may or may not indicate a reduction in risk, but there is little reliable evidence that such measures indicate whether there has been positive change with treatment or whether risk has reduced " (2013: 59).

Discussion at the workshop also revealed apparent differences in the way in which psychometric tests were administered between sites – whether men were simply given the tests to complete themselves, or whether they were supervised by MF:MC staff, or completed them with MF:MC staff. These differences in administration may have implications for the consistency of this data between sites.

• There is a relatively high level of missing data for the 'exit interview' with participants – 161 men completed it (compared with 230 men for whom there is a post-programme Stable 2007 score). It is also obviously reliant on participants' self-report, which cannot on its own be regarded as a reliable outcome indicator, particularly with respect to perceived risk of re-offending.
• There is also a high level of missing data and 'unknown' responses for the significant others questionnaire. Moreover, the questionnaire was considered problematic as an outcome measure because in a significant proportion of cases, particularly for men in prison, there was no non-professional who felt able to complete it. Staff reported that in some sites, this questionnaire was completed by MF:MC practitioners based on their own observations instead. However, this means that the data is inconsistent (professional judgements may be very different from those of friends or family members), and there appears to be no way of establishing from the dataset who completed individual questionnaires.

Given these concerns, significant caution must be applied in interpreting any findings about 'distance travelled' by participants on the programme based on this data. The end of this chapter includes a number of suggestions about potential improvements to the outcome measures collected for MF:MC going forward.

4.3.2 Qualitative data

The aim of qualitative sampling is to identify the nature and range of views on and experiences of a particular programme or issue. However, as noted in Chapter 1, while the evaluation included staff from across all 15 MF:MC delivery sites, it was not possible to include participants from every site, and we were only able to interview a limited number of stakeholders within the evaluation timescale and resources. As such, there may be other views on programme impact that are not captured here.

In addition, as discussed in Chapter 1, participants who were motivated to agree to an evaluation interview are also likely to be those relatively more motivated to participate in MF:MC. Moreover, although researchers emphasised to MF:MC participants that the purpose of the interviews was to assess the programme not its participants, it is arguably unlikely that offenders would reveal anything in a research interview that might suggest they felt their risk had increased in particular areas. As such, while interviews with participants can help identify perceived positive impacts and suggestions for improvement, they are less likely to be able to identify any areas where the programme is having a negative impact on key treatment outcomes.

4.4 Quantitative outcome measures: distance travelled

4.4.1 Overall risk profiles

As noted in Chapter 3, the 'Stable' scale produces a score for 13 risk factors, with a score of either 0 (suggesting no need for intervention), 1 (possible need for intervention), or 2 (definite need for intervention) on each one, resulting in an overall maximum score of 26. Those scores are then banded into "High", "Moderate" or "Low" risk categories. As shown in Figure 4.1, across the 230 men for whom there was both a pre- and post-programme Stable 2007 score in the monitoring data, there was a significant shift in overall risk profiles from the beginning to the end of the programme: the proportion of men classified as high risk decreased from 29% to 17%, while the proportion of men classified as low risk increased from 1% to 22%.

Figure 4.1: Stable 2007 Bands at beginning and end of programme

This change in risk level can also be illustrated by looking at the difference between individual Stable 2007 scores at the beginning and end of the programme, expressed as either a positive or negative figure (a negative figure representing a reduction in the risk score, a positive figure representing an increase). The majority of men (69%) saw a reduction in their overall Stable 2007 risk score (although in some cases this reduction was not sufficient to move them from 'high' to 'moderate' or from 'moderate' to 'low'), while 20% saw no change in their scores and 10% saw their risk scores increase. Among the 10% who exhibited an increase in their Stable 2007 score, in most cases this was only by 1 or 2 points (within a maximum score of 26).

Table 4.1 shows movement between risk bands pre to post-programme in more detail – among the 66 men rated as high risk based on their Stable 2007 scores at the start of MF:MC, 53% remained high risk at the end of the programme, while 38% had moved into the 'moderate' category and 9% were rated low risk. Among the bigger group (161 men) initially rated 'moderate' risk, most (72%) remained in this category, but 26% moved into the 'low risk' category, while a small number (3%, n=4) were rated 'high risk' at the end of the programme.

Table 4.1: Post-programme Stable 2007 scores by pre-programme Stable 2007 score

Pre-programme Stable 2007 score	Post programme Stable 2007 score			Base
	High (12-26)	Moderate (4-11)	Low (0-3)
High	35	25	6	66
	53%	38%	9%
Moderate	4	116	41	161
	3%	72%	26%
Low	-	-	3	3
	-	-	100%

Base: All those with a Stable 2007 score at beginning and end of programme (230)

The average reduction in mean SA07 scores from pre- to post-programme was -2.7 (from 9.5 to 6.8 out of a maximum of 26). However, those starting with a High Stable 2007 score reduced more than those starting with a Moderate score (by 4 points compared with 2.2 - see Annex C, Table C.4). These findings suggest that men who are 'high risk' at the start of MF:MC are more likely to see their risk levels reduce, and by a bigger margin, over the course of the programme.

The reduction in mean Stable 2007 risk scores was also bigger for men who participated in MF:MC in community sites compared with those who took part in prison, across those for whom a pre- and post-programme score was available. There was no significant variation by age group (see Annex C, Table C.4).

4.4.2 Changes in specific risk factors

As discussed above, Stable 2007 produces scores for 13 specific risk factors, around which much of the content of MF:MC has been designed. Table 4.2 shows the mean scores pre- and post- completion of MF:MC for each individual factor (from a minimum of 0 to a maximum of 2). While on average scores reduced pre- to post-programme on each individual risk factor, the mean reduction was greater for some than others:

• ' Sex pre-occupation', 'General social rejection/loneliness', 'Sex as coping' and 'Poor cognitive problem solving skills' were the categories that saw the biggest reductions in mean scores
• 'Co-operation with supervision' and 'Capacity for relationship stability' were the categories that reduced the least.

The mean score for cooperation with supervision was, however, relatively low at both the start and end of the programme. In contrast, 'Capacity for relationship stability' was one of the higher scoring risk factors at both the start and end, suggesting both that this is relatively more likely to be rated a risk factor for men participating in MF:MC, and that the average level of risk men present on this factor is less likely to change over the course of participation. Two attitudinal risk factors – emotional identification with children and hostility towards women – also reduced less on average from pre- to post-programme, although this largely reflects the fact that most men scored '0' both pre- and post-programme on these measures, so the mean score at both points was relatively lower than for other risk factors.

Table 4.2: Mean score for each Stable category, pre and post-programme

	Mean score pre programme (Between 0-2)	Mean score post programme (Between 0-2)	Mean Change in score	Base
Social influences	0.79	0.63	-0.16	230
Capacity for relationship stability	1.31	1.24	-0.07	231
Emotional ID with children (1)	0.27	0.16	-0.11	183
Hostility towards women	0.31	0.21	-0.10	231
General social rejection/loneliness	0.95	0.61	-0.34	230
Lack of concern for others	0.52	0.31	-0.21	230
Impulsive acts	0.50	0.30	-0.20	230
Poor cognitive problem solving skills	0.92	0.61	-0.31	230
Negative emotionality/hostility	0.65	0.43	-0.22	230
Sex preoccupation	0.80	0.45	-0.35	230
Sex as coping	0.67	0.36	-0.31	230
Deviant sexual interests	1.40	1.19	-0.21	230
Co-operation with supervision	0.42	0.36	-0.06	230

Base: All those with a score at beginning and end of programme

Notes to table:

1 – the difference in the base for this measure is largely accounted for by men who were either rated 0 at the start of the programme and 'not applicable' at the end, or for whom the measure was deemed 'not applicable' both pre- and post-programme

Although most men saw a reduction in their risk scores, it is worth noting that there are question-marks over the reliability of the risk data.

4.4.3 Changes in psychological traits

The psychometric tests administered before and after participation in MF:MC are:

• The Self-Compassion Scale, designed as a technique to assist men in regulating emotional responses to triggering situations, as negative emotionality is a risk factor included in the SA07. The higher the score on the Self-Compassion scale, the more likely that someone is self-compassionate. The lower the score, the more likely someone needs to work on developing compassion for self. MF:MC is intended to increase men's self-compassion, as part of addressing intimacy deficits and self-regulation.
• The Social and Emotional Loneliness Scale for Adults ( SELSA) is measured on three levels: romantic, family and social loneliness. MF:MC attempts to encourage participants to build healthy social networks and to develop their relationship skills, which should in turn reduce social and emotional loneliness.
• Social Problem Solving Inventory ( SPSI). As discussed above, problem solving skills are a key element of general self-regulation, targeted by the Thinking Skills and Self-Management modules in particular.
• Levenson Locus of Control ( LOC), measures the extent to which individuals feel in control of their lives. It is made up of three sub-scales relating to: internality (the extent to which people believe they have control over their lives); powerful others (the extent to which people believe that powerful others control their lives); and chance (the extent to which they believe chance affects their lives). Encouraging participants to take control of their lives is a general theme within MF:MC and the Good Lives Model, which encourages participants to identify appropriate life goals and develop skills and strategies to achieve these.
• Barratt Impulsiveness Scale assesses nine factors associated with impulsivity, with high scores being used to classify someone as highly impulsive. Impulsivity is another risk factor addressed as part of attempting to improve participants' general self-regulation.
• Hostility Towards Women Scale, included to measure problematic attitudes towards women.
• Sex with Children is Justifiable Scale, included to measure the presence of child abuse supportive attitudes.
• Paulhus Deception Scale which measures: impression management (the tendency to purposefully describe oneself in overly positive terms) and self-deception (the tendency to attempt to be honest but still exaggerate positive virtues).

The potential limitations of psychometric scales in general and of a number of these scale in particular have already been discussed. However, insofar as any limitations are likely to apply equally to both the pre- and post-programme measures, examining any change in scores over the course of the programme may provide some indication of whether participants have experienced change in these psychological traits.

On each psychometric scale, the majority of men had either shown improvement or stayed at the same level over the course of the programme (Table 4.3). Across most scales, 5% or less recorded a worse score at the end of the programme than at the start. However, this figure was slightly higher for the 'self-compassion scale' and 'social problem solving inventory' – 9% and 12% respectively recorded lower scores post-programme on these scales.

Table 4.3 shows what proportion of men have scores that are higher or lower by any amount post-programme compared with pre-programme. In some cases, these changes may have been too small to indicate meaningful change. However, it is worth noting that the change in scores pre to post programme tended to be smaller, on average, among those recording a worse score post-programme than among those recording an improvement in scores. For example, the mean improvement in scores (within a possible range of 12-60) on the 'self-compassion' scale was 34 points, while the mean change among those whose score got worse was 12 points (see Annex C, Table C.5). So the scale of 'decline' among those men who recorded a worse score post-programme tended to be smaller than the scale of improvement among those men who recorded a better score.

Table 4.3: Change in scores on psychometric scales pre and post-programme

	Improved	Stayed the same	Worse	Base
Self-Compassion Scale	36%	54%	9%	228
Social Problem Solving Inventory	35%	54%	12%	205
Social and Emotional Problem Solving Scale for Adults ( SELSA)
SELSA 1 – Romantic (Loneliness/struggling with romantic relationships)	38%	58%	4%	205
SELSA 2 – Family (Loneliness/struggling with family relationships)	42%	56%	2%	205
SELSA 3 – Social (Loneliness/struggling with social relationships)	42%	55%	2%	205
Locus of Control ( LOC)
LOC 1 – Internality (Belief that I control my life)	39%	54%	7%	205
LOC 2 – Powerful others (Belief that powerful others control my life)	41%	55%	3%	205
LOC 3 – Chance (Belief that my life is controlled by chance)	21%	74%	5%	205
Barratt Impulsiveness Scale	42%	55%	3%	205
Hostility Towards Women Scale	22%	75%	2%	205
Sex with Children is Justifiable Scale	18%	81%	1%	205
Paulhus Deception Scale	34%	60%	5%	205
Self deception	27%	66%	7%	205
Impression management	35%	57%	7%	205

The scales on which the highest proportion of men recorded an improvement were SELSA 'Family' (42%) and 'Social' (42%), the Barratt Impulsiveness scale (42%), and LOC 'Powerful Other' (41%). Improvements in 'family' and 'social' isolation as measured by SELSA broadly tally with the finding above that 'general social rejection and loneliness' is one of the Stable 2007 risk factors that reduces most, on average, for MF:MC participants over the course of the programme. The other findings suggest that participants may also be relatively more likely to experience reductions in impulsiveness and to become less likely to view their lives as controlled by powerful others by the end of the programme.

In terms of both the Hostility Towards Women and Sex with Children is Justifiable scales, more than three quarters of men stayed at the same level between the beginning and end of the programme. However, this largely reflects the fact that scores on each of these scales tended to be low to begin with – so there was less scope for improvement. This may reflect the fact that these scales are not likely to be equally relevant to all categories of sex offender (for example, men who are not paedophiles would be expected to score low on the 'Sex with children is justifiable' scale at both pre- and post-programme stage). There were very few statistically significant differences between prison and community sites or between men with different pre-programme risk levels. The few differences that did exist did not point in a clear direction, so are not reported here.

4.4.4 Perceived outcomes based on participant 'exit survey'

A majority of the 161 men who completed an exit survey expressed positive views on the impact of MF:MC across a range of measures (Table 4.4). Over 80% in each case said that MF:MC had: helped them learn skills that would help them to deal with other problems (89%); that the programme met their needs (87%); and that it had help them to understand better why they committed their offence (83%). Participants were slightly less likely to report positive changes in their family relationships, though a majority (72%) reported that these had improved since taking part in MF:MC. In terms of perceived impact on offending behaviour, although 85% said MF:MC would stop them re-offending in a similar manner, 10% said it would not (although this does not necessarily imply they think they will re-offend – just that they do not think MF:MC has influenced this), and 5% were not sure.

Table 4.4: MF:MC participant exit interview responses

	Yes	No	Unsure
Do you think MF:MC will stop you reoffending in a similar manner?	85	10	5
Do you think that MF:MC met your needs?	87	7	6
Do you think that the programme helped you to understand better why you committed the offence?	83	12	6
Have there been any positive changes in your family relationships since taking part in MF:MC?	72	19	9
Do you think that the skills you learned in MF:MC will also help deal with other problems and difficulties in life that you have?	89	7	4
Would you recommend other people who have committed sexual offences take part in MF:MC?	90	5	6

Base: All who completed an Exit Interview (161)

4.4.5 Perceived impact based on the 'significant others' survey

Across all the measures included in the 'significant others' survey (which, as noted above, was completed by either a family member/friend or by an MF:MC practitioner), very few of the 119 respondents (under 5% in each case) reported observing negative changes in participants since they took part in MF:MC. However, in each case a high proportion of responses (48-53%) were coded "unknown". It is unclear if this is because the respondent felt unable to comment on whether the participant had changed with respect to the behaviour asked about, or whether no significant other was available to complete the survey.

The areas with the highest proportion of respondents reporting positive changes were:

• men's considerations for the feelings of others (38% saw positive changes)
• their motivations to develop pro-social behaviours (37%)
• their relationship with the significant other (36%), and
• their ability to cope with significant life stress (34%).

Lower proportions reported observing positive changes in:

• the extent to which men felt comfortable with adults (27% reported a positive change but 23% saw no change)
• the extent to which they have the skills to establish and maintain an intimate, live-in relationship (23% reported a positive change but 21% reported no change), and
• their capacity to make friends and be with others (28% reported a positive change, 20% no change).

However, where 'significant others' observe 'no change', it is unclear whether or not they believed this was an area where the man had issues at the beginning of the programme that needed to be addressed – in other words, 'no change' could imply a continuation of a problem, or that this was not an area the significant other considered problematic in the first place.

Table 4.5: Results of exit interviews with significant others

What changes, if any, have you observed in the participant since they were on MF:MC?	Positive	No change	Negative	Unknown
Relationships
Relationship with me	36%	15%	1%	48%
The capacity to make friends and be with others	28%	20%	3%	49%
The skills to establish and maintain an intimate, live-in relationship without any obvious problems	23%	21%	4%	53%
Interaction
Interaction with people in the community	30%	20%	2%	48%
Feels comfortable with adults	27%	23%	2%	48%
Behaviours
The motivations to develop pro-social behaviours	37%	11%	3%	50%
Consideration for the feelings of others and behaves in a caring manner	38%	13%	1%	48%
The ability to cope with significant life stress	34%	13%	3%	50%

Base: All with data for a significant other interview (119)

4.5 Staff, participant and stakeholder perceptions of impact

4.5.3 Participants' expectations of MF:MC

The MF:MC participants we spoke to did not always have a clear idea of what MF:MC would involve prior to starting on the programme, reporting that they were either told very little in advance, or had struggled to take in what was involved because of their unsettled state of mind at the time. A degree of initial scepticism about the programme was apparent from men who said they had not felt they needed to attend and had only agreed because it was a condition of their license or parole. However, men also described positive hopes for participation in MF:MC, including hoping it would help them change in general, gain a better understanding of themselves and/or the reasons they offended, and help to prevent them reoffending by developing strategies to cope better with the factors that led up to the offence. Overall, Men's expectations of the MF:MC appeared to reflect programme aims.

4.5.4 Overall perceptions of impact

The dominant position among Treatment Managers and Case Managers interviewed for this study was that it is extremely difficult to judge exactly how effective MF:MC actually is in reducing the likelihood of reoffending. Professional comments on the challenges of making a robust judgement on impact reflected many of the general evaluation challenges already discussed in this report: men might not come back to custody or re-enter the community programme, but short-term re-offending rates are low; Stable 2007 scores might reduce, but you would expect this to happen even if they were just on ordinary supervision; and it is difficult to separate out the impact of MF:MC from the MAPPA system as a whole.

However, in spite of this lack of certainty, the overall perception among Treatment Managers, Practitioners and Case Managers was that MF:MC is having a positive impact. They based this on feedback from men, significant others, other professionals (Case Managers and Supervising Officers) and on their own observations of behavioural change occurring (to varying degrees) among participants. A Case Manager described seeing someone on a three-year order " develop into a whole different person … even the way they hold themselves and their body language" when on MF:MC. Treatment Managers and Case Managers also said they felt it was a more evidence-based programme than previous sex offender interventions. As such, even when they felt unable to comment definitively on impact, they felt it had a "better chance" of being effective:

There isn't any data to show its effectiveness. But my view, having worked on various sex offender programmes, in various settings, to me this is by far the most effective in relation to getting the engagement with the men, enabling the men to understand their offending behaviour and why they're doing what they have been doing, and planning for the future … We've certainly got positive feedback from Case Managers as well, who are seeing the difference in their guys.

(Treatment Manager 8)

The three stakeholders from Police Scotland, in contrast, were less convinced MF:MC was having a positive impact – they felt that men may present themselves differently in a group setting and know how to say the "right things" even when they actually go on to reoffend.

Professionals again reiterated the view that the overall impact of the programme will in large part depend on participants' own motivation – " it can be effective if the individual wants to change" (Treatment Manager 10).

Perceptions of MF:MC among the participants who had completed MF:MC were extremely positive. They reported that it had not only helped them address their behaviour, but also to understand the reasons for it and to develop a better understanding of themselves:

It helps you discover things you never knew about yourself; look at yourself in a different way; all aspects of your life, not just your offences. If you give it a chance, you will get the benefits from it.

( MF:MC Participant 12)

4.5.5 Perceived impacts on treatment targets

Participants and staff also gave examples of the ways in which they thought MF:MC had impacted on the specific treatment targets described above (section 4.2).

• Social influences and intimacy deficits – MF:MC staff and Case Managers reported observing men displaying better social skills (within and outwith groups) and overcoming social anxieties, while participants described having better, more open relationships with family and friends and feeling less socially isolated. Participants attributed these perceived changes both to the Relationships module specifically, and to a belief that MF:MC had made them more confident (via the group work process), compassionate, and/or helped them redress their priorities to focus more on family and friends in general.

At first I was a bit shy, I fell back, I didn't know if I could trust them (the group) with what I was saying. … (Now) I feel more relaxed because I have known them, now they are always up front with me. … So, I'm more open, I like to talk to them and be up front.

(INTERVIEWER) ...does that go to you like outside the group as well?

Yes, it does, yes, … since I've been in this one (relationship), I feel more relaxed and honest, we're doing a lot more things together, we have not actually had any arguments in this relationship at all so far, we are really like a family

( MF:MC Participant 16)

• General self-regulation – Participants reported that MF:MC had helped them to approach problems in a better way – to step back and assess situations rather than acting on impulse – and to gain better control of their emotions (particularly with respect to managing their anger). For example, one participant who had recently lost a job felt he had responded to this very differently in comparison with the past:

I don't feel that I'm as hung up on things any more. I don't constantly ruminate on it, I'm moving on, I'm saying to myself, "okay, it's happened how do we move forward and what can I do to get into work?" So, being more practical, being more task focused about things rather than being emotionally focused and getting caught up and hung up about things that's went on in the past.

( MF:MC Participant 9)

Participants also reported changes in their outlook indicating a reduction in 'negative emotionality' – that they no longer saw the world as a " threat" or a "bad place".

• Co-operation with supervision – Participants gave examples of where they had used techniques learned in MF:MC to manage meetings with social workers in a calmer fashion, indicating that the programme can help support better engagement with supervision:

Before I used to always have problems communicating with them (social worker) and actually getting my point across. Since I've been down here it makes it a lot easier because I can now see things from a slightly different perspective and it makes it easier just to get the point across to them. (…) Less heated, less heated, because quite a lot of times normally ended up in an argument, so it definitely has been a big change in the way I actually deal with them now.

( MF:MC Participant 14)

• Attitudes and empathy – Participants described MF:MC as challenging and increasing their understanding of problematic attitudes – for example, a participant with learning disabilities described gaining a better understanding of what 'entitlement' is and why it is wrong from the programme, while another said it had taught him to see women as equals when he had previously viewed them as existing for his benefit. Empathy tended to be discussed in general terms (for example, developing more compassion for other people and themselves, or learning empathy from being in a group) rather than specifically with respect to victims. However, participants reported that MF:MC had helped them understand how many people were affected by their actions (and specifically by their offence).

The perceived impact of the programme on sexual self-regulation was more difficult to ascertain. It was not appropriate to delve into this area within a one-off interview with participants, although there were examples where participants indicated feeling they had gained a better understanding of issues around sex and healthy/unhealthy sexual thoughts. Among Treatment Managers and Practitioners, while one view was that MF:MC tackled sexual function more effectively than previous programmes, there was also some concern about whether aspects of the programme might, in some cases, have unintended negative consequences for sexual self-regulation. In particular, Practitioners speculated that the use of the 'sexual thoughts' diary as part of the HSF module with men whose offence was committed a long time ago could potentially increase their level of sexual preoccupation. Participant interviews also revealed some concerns about completing this diary, particularly in custodial settings – one participant felt he had been left with the diary for too long a period before being able to discuss it with Practitioners, and was also concerned about the diary being read by Prison Officers and " misinterpreted".

There was also less discussion within staff or participant interviews of MF:MC's impact on motivation to change – discussion around this issue tended to focus on men's motivation as a factor influencing the impact of the programme, rather than something the programme influences. However, both staff and participants discussed the fact that the programme gives men 'goals' to aim towards, which can in turn help bolster their motivation to change.

Having goals , it's not something I had really considered at that point. (…) I think the group in the way the programme is set up is just to try and encourage you to have a goal, and take steps towards it and the goals, well hopefully, setting these goals will hopefully stop you offending or reoffending in the future.

( MF:MC Participant 10)

4.5.6 Wider impacts

In addition to impacts on specific treatment outcomes, participants also discussed the perceived impact of MF:MC on:

• Self-esteem – Perceived positive impacts on self-esteem ranged from helping participants to feel more confident, to reducing self-harm and suicidal thoughts.

• Insight into offending - The way in which MF:MC helps participants link their past experiences and thoughts with their offending behaviour was viewed as particularly useful:

I've kind of been able to link in my mind things that I've been thinking about in my past and things I've done, so it's kind of helped me at this point anyway to link a few things together, giving me some ideas about possibly why I did what I did.

( MF:MC Participant 11)

• Feelings about the future – Participants felt MF:MC had helped them to develop plans and ambitions for the future (which were, in many cases, previously lacking), and to feel more optimistic about what the future holds:

There are different possibilities about what I can do in the future; I can find alternatives to the things that are restricted to me, but have a meaningful life at the same time.

( MF:MC Participant 14)

4.6 Perceived influences on impact

Interviews with MF:MC staff and participants also identified the specific features of the programme they felt contributed to the impacts observed above.

4.6.1 Group structure and dynamic

MF:MC participants and staff expressed very similar views on the scope for group work to support change via:

• Enabling participants to learn from the experience of others – for example, understanding how other people came to commit their offence could help with understanding of their own offence. The rolling structure also enabled participants to learn from, and by motivated by, others at different stages of the programme.
• Peer questioning – which both participants and staff reported could be more effective than professionals questioning participants. Participants reported that the questions from other group members were sometimes challenging, but that they encouraged them to engage honestly with their behaviours
• Helping to build confidence, emotional regulation and social skills by allowing participants to practice talking about challenging topics and listening to others.

They have got different coping strategies and that's how I'm learning all the time in this programme … and I can empathise with the guys because a lot of the guys are in the same boat as me. (…) Some of the questions can be a bit challenging, but I've been told that I don't hold anything back (…) I kept quiet the first couple of sessions until I got my first assignment and that's when I started to come out of my shell.

( MF:MC Participant 15)

4.6.2 The overall theoretical approach

Treatment Managers and Practitioners felt that the Good Lives Model meant MF:MC was more engaging, holistic and future-oriented than previous interventions, while the flexibility of the programme and the fact it can be tailored to individual needs was mentioned by both professionals and participants as a positive feature.

Professionals and participants both contrasted MF:MC favourably with previous sex offender interventions, particularly CSOGP (Community Sex Offender Groupwork Programme), which was viewed as both unhelpfully confessional and offence-focused and too strictly structured (it was based on fixed rather than rolling groups), meaning men lost interest. A Case Manager reported that CSOGP had felt like " punishment" – in focusing on the future and tailoring delivery MF:MC was " everything that CSOGP wasn't."

However, a number of interviewees (including the small number of Police Scotland Stakeholders and MF:MC Treatment Managers) suggested that MF:MC could be a bit " fluffy" in comparison with other programmes, and that some men would benefit from more direct challenge about their behaviour. This view was closely linked with debate about whether the programme should address the offence itself more directly, discussed in Chapter 2.

4.6.3 Specific topics and modules

Staff and participants both expressed mixed views on whether any modules were particularly more or less effective. One view was that all topics were equally helpful, although as discussed in Chapter 2 staff expressed concerns about the level of depth in the programme manual guidance and supporting materials associated with some modules. A number of modules and topics were singled out by participants as particularly useful, however.

• Thinking Styles – The perception among professionals that some of the content of the Thinking Styles modules was somewhat " shallow" did not appear to be reflected in participants' views: a recurrent theme among participants was that work around Thinking Styles was the most challenging element of MF:MC, but also the most beneficial in terms of identifying, challenging and changing unhelpful thinking patterns, and helping participants understand the thinking styles that may have contributed to their offending.

This is what's important for me this bit (Thinking Styles), this is the bit I really want to try and get my head into it and kind of get a comprehensive understanding of why I have ended up where I am today. How my thinking styles linked to my offending. I'm not quite there yet because I'm still trying to un-write the rewritten things in the back of my mind. But, even if I have to spend a bit longer on the programme tackling them, then I'm quite willing to do that, because it will be very beneficial, because then I would be able to unravel it and get a better understanding of it, I can then start to work on how to change it and fix it … have a new thinking style.

( MF:MC Participant 4)

• Self-management – Again, although some MF:MC Treatment Managers and Practitioners criticised this module as lacking depth, participants felt that the topic and associated exercises (e.g. the 'problem solving log') had helped them develop better problem solving skills by helping them consider wider options and analyse the possible outcomes.
• Relationships – As discussed above, where participants reported improvements in their relationships with friends and family, this was in part attributed to the topics covered in the Relationships module, which had helped them to be more aware of other perspectives and of their own behaviour and its impact on others.

As discussed above, more mixed views were expressed on the HSF module. Some participants said this module had helped them understand themselves and their emotions, encouraged different ways of thinking, and helped them open up about sensitive topics. However, others said that although they had found elements of it useful, they were very uncomfortable with aspects of its delivery (such as the sexual thoughts diary and the fact that some sessions were with a female Practitioner which "didn't sit right" for one participant). Among MF:MC staff, while the module as a whole was viewed as valuable, it was also criticised for being too limited in the range of topics and techniques included, which were perceived to limit its impact. It was also suggested by Treatment Managers that the HSF module was less likely to impact on paedophilic sexual attitudes, in part because it was perceived to go into insufficient depth for this group of men, and because of challenges around using some of the suggested techniques (such as fantasy replacement) with men with men whose sexual interest are not age appropriate.

4.6.4 Specific approaches

As discussed in the introduction, MF:MC draws on a number of different approaches, including Mindfulness, CBT and Schema therapy. Where specific approaches were discussed by MF:MC staff, this tended to be in the context of debating whether their use as described in the manual is effective for all participants and/or whether there is enough guidance and support for their use, rather than whether or not a particular technique is effective per se.

There was a particular debate around how Mindfulness is included in MF:MC. Both participants and staff reported that Mindfulness can be a very useful technique in helping men to self-manage – men described using it to manage depression and anxiety, or to identify and focus on more positive emotions. However, both also reported that some men found it frustrating to have a mindfulness exercise at the start of every group – for men who did not find it useful, this could become a barrier to engagement. It was suggested it should be used more flexibly – for example, towards the end of a session that has been particularly emotionally charged.

4.6.5 External factors

In addition to discussing the ways in which different features of MF:MC design and delivery did or did not contribute to programme outcomes, participants and professionals also discussed a number of factors external to the programme. In particular, the wider social context participants find themselves in could be both a barrier and a facilitator to change. For example, a participant who had found a volunteer role reported that this had given him confidence and helped him to feel trusted again. In contrast, other participants felt that while MF:MC had expanded their horizons, they were limited in the opportunities they had to put their learning into practice by restrictions on who they can have contact with or (for those in custody) a general lack of opportunity to try new things.

I think it should be in tandem with other things. Like you've learnt all this and it's helped you be a better person and all that, but for me it's like, aye, but you've done that work, you've learnt all this stuff, but what are you going to do with it?

( MF:MC Participant 2)

4.7 Meeting diverse presenting needs?

Overall, Treatment Managers and Practitioners indicated that MF:MC is able to address a range of needs and work effectively with a range of offender types. The ability to choose which modules each participant takes and the flexibility to adapt discussion to individual needs were considered key in this respect. However, although staff felt the programme could work for a range of men, they also identified a number of specific groups whose needs they felt were not fully met by the programme at present including:

• Men with Personality Disorder and/or emotional difficulties – it was suggested that some of these men have been on several sex offender programmes with no sign of progress and are ultimately not suited to group programmes.
• Men with early maladaptive schemas ^[15] – a psychologist who supported Community MF:MC teams suggested that MF:MC Practitioners are not always equipped to respond to this particular issue. In their view, these men ought to have access to specialist treatment, but this was not always readily available.
• Men with Autistic Spectrum Disorder – whose needs were perceived to be more difficult to meet in a group setting.
• Internet offenders – as discussed in Chapter 3, there was a debate about the process for identifying the "right" internet offenders to benefit from MF:MC and the risk of 'over-treating' some men in this group.
• Paedophiles who view their sexual pre-occupation with children as an acceptable orientation - is was suggested that the manual is particularly limited with respect to addressing the attitudes of this group. One view (expressed by a Case Manager) was that this group of men need external controls and " no arena for debate" and should be excluded from group programmes.
• Men who commit sexual offences in a relationship – as discussed above, a Case Manager reported that they sometimes referred these men to the Caledonian Domestic Abuse programme rather than MF:MC, on the basis that it was more effective in addressing the specific attitudes they associated with this kind of offence.
• Men who have already completed multiple interventions – while staff felt there might be good reasons for repeating sex offender programmes where treatment needs have not been fully met, they questioned the appropriateness of allowing men to attend multiple treatment programmes with no observable impact. It was suggested that issues around potential 'over-treatment' were more directly addressed within CSOGP.

Another view was that while diverse needs can be met by MF:MC, attempting to meet very diverse needs in the same group may limit its effectiveness for individual group members – for example, including men with lower cognitive abilities and those with high IQs in the same group can lead to frustration among the latter that the group is not operating at a sufficiently high level.

Discussion around the effectiveness of MF:MC for these diverse groups touches on issues already raised around:

• the assessment process and whether it effectively assesses programme suitability (particularly with respect to internet offenders and "deniers")
• the content of the manuals and whether the topics and techniques are sufficient to enable effective delivery to diverse participants
• the training and support staff receive to develop the skills needed to address very diverse needs, and
• the external resources available to support delivery.

With respect to this latter point, MF:MC staff and stakeholders argued that it is not reasonable to expect every MF:MC Practitioner to develop expertise in every delivery technique that may be needed to meet every set of presenting needs likely to be encountered within MF:MC. It was felt that this was not sufficiently recognised within the programme at present – the manuals were seen as assuming staff would simply supplement materials with their own expertise to ensure the programme could meet identified needs. This issue is discussed further in the next section (suggestions for improvement).

4.8 Suggestions for improvement

4.8.1 Potential improvements to enhance impact

Chapters 2 and 3 have already outlined various suggested improvements to the delivery of MF:MC and to the assessment process that are ultimately aimed at enhancing its impact. Additional recommendations stemming from the discussion in this chapter relate primarily to the challenges of meeting extremely diverse needs. Suggestions (primarily from Treatment Managers) include:

• Provide more external expertise to support delivery – in addition to reviewing the availability of external psychological support for community sites, discussed in Chapter 2, it was suggested that given the scope of MF:MC there was a need to engage experts in specific areas (such as internet offenders, working with people with offenders with learning disabilities, working with paedophiles who view their sexual pre-occupation with children as an acceptable orientation, etc.) and approaches (e.g. Schema Therapy, sex therapy) to review current content and practice (which could include, for example, reviewing guidance on whether/when/how the 'sexual thoughts' diary should be completed), and to help upskill staff. A participant who had learning disabilities suggested that while MF:MC staff were very " patient" with him, he felt they ought to involve someone with expertise in learning disabilities (similar to a Special Educational Needs teacher in school), who could both help him progress faster and advise staff on how to make the content more accessible to him.
• Assign national leads on specific approaches and issues – as an alternative or addition to involving external experts on a regular basis, it was suggested that there should be national leads on specific approaches and issues within MF:MC, with a remit to keep up to date on best practice and research and to ensure the programme reflects this. Given the resourcing issues identified in Chapter 2, this would almost certainly require additional staff resources.
• Consider the impact of/guidance on 'over-treatment', where men have attended several group interventions without apparent progress. Alongside this, it was suggested that options for supplementary treatment and/or structured alternatives to MF:MC should be considered for those who may not be well-suited to group-based programmes (e.g. men with ASD, some men with Personality Disorders).
• Consider the feasibility of establishing separate groups for men with lower cognitive functioning – while this may not be feasible across all community sites, it was suggested it ought to be possible to establish such a group within SPS.

4.8.2 Potential improvements to impact measurement

The first part of this chapter identified substantial perceived limitations to the data currently collected within MF:MC which could be used to assess impact. Overall, there is a need to systematically review what outcome measures are being collected to:

• Ensure they include valid and reliable measures of all intermediate treatment targets
• Improve practitioner trust, understanding and use of these measures, so that they are more likely to be collected and entered systematically and on time, and practitioners are able to use them to inform both individual treatment and their own understanding of how the programme is working locally.

Specific issues this review of outcomes should consider include include:

• The utility of the psychometrics included in the programme – given concerns raised by MF:MC teams, and wider academic debate about the utility of psychometrics, the utility and appropriateness of the current battery for use in measuring treatment outcomes with sex offenders needs to be formally reviewed. Are they the best measures available for the treatment outcomes MF:MC is attempting to effect? If any of the existing scales are to be retained, or if they are replaced with alternative measures, there will also be a need to provide clearer evidence to support their inclusion and clearer guidance to support their interpretation at individual, site and programme level. This should include providing clear guidance on:
  • When a particular score indicates a clinical issue?
  • What would be a significant pre to post-programme shift (at individual level)
  • What normative data (from appropriate comparative groups for the population) should scores be compared against?
• Whether additional measures of success are needed, linked to specific treatment needs. Although Stable 2007 is closely aligned with treatment objectives, it was suggested that this could be supplemented with, for example, scales rating the level of insight participants are believed to display alongside evidence of specific behaviours linked to each outcome.
• Ways of reducing the reliance on self-report data to measure outcomes. Treatment Managers in custodial settings suggested that there are various more objective measures of deviant sexual interest/sexual pre-occupation that could be incorporated, for example. Suggestions included using software developed to measure view time of images, in order to identify deviant sexual interests pre and post-programme.

• The wording of the significant others' questionnaire, and clarifying who should (and does) complete it. The current wording of this questionnaire makes interpreting the findings difficult – in particular, it is unclear what 'no change' means, since we do not know if the respondent considered this to be a problem area in the first place. There is also a need to clarify who is able to complete this, and to record who actually completes this (i.e. their relationship to the respondent), to enable accurate analysis and interpretation of this data. Finally, the value of both the significant other and participant exit questionnaires would be enhanced if the questions were more closely aligned – enabling both practitioners and evaluators to compare men's views of their progress with the assessments of their significant others more directly.