Evaluation of Community Payback Orders, Criminal Justice Social Work Reports and the Presumption Against Short Sentences – Research Findings

As part of the Scottish Government's commitment to 'a coherent penal policy that uses prison for serious and dangerous offenders but deals with lower-risk offenders in the community' (Scottish Government, 2007), three related reforms were introduced in early 2011: Community Payback Orders (CPOs), the Presumption Against Short Sentences (PASS) and Criminal Justice Social Work Reports (CJSWRs). This evaluation aimed to examine whether the reforms were implemented as intended; consider the extent to which the short and medium term outcomes were realised; and identify potential obstacles to realising those and longer-term goals.

Background

As part of the Scottish Government's commitment to 'a coherent penal policy that uses prison for serious and dangerous offenders but deals with lower-risk offenders in the community' (Scottish Government, 2007, p.1), three related reforms were introduced in early 2011: Community Payback Orders (CPOs), the Presumption Against Short Sentences (PASS) and Criminal Justice Social Work Reports (CJSWRs). Together, these were intended to contribute to a more effective and publicly acceptable system of community disposals, to provide a credible alternative to short sentences and, in turn, to contribute to a reduction in 'churn' in the prison population.

CPOs effectively replaced many of the existing community penalties (including Community Service Orders, Supervised Attendance Orders and Probation Orders) with a new single disposal with a range of requirements, capable of being imposed either singly or in a variety of combinations. Specifically, CPOs were aimed at ensuring that community penalties were high quality, effective, immediate, visible, flexible and relevant and, as such, acceptable to members of the public and sentencers alike. PASS placed a legislative onus on sentencers to avoid the use of short prison sentences (of less than three months) in all but exceptional cases and, in the latter, to record formally the reasons for the use of such a disposal. The introduction of CJSWRs complemented these other legislative changes by creating a national template for social work court reports, with the aim of ensuring that sentencers have access to consistent and high quality information and analysis in order to inform decision-making about the appropriateness of particular disposals.

The Evaluation

Following a competitive tendering exercise, a team based around ScotCen Social Research was commissioned to evaluate the reforms. The evaluation was loosely structured around a logic model showing the relationship between the three main elements and highlighting the 'theory of change' through which specific inputs and activities were expected to lead to particular short, medium and long-term outcomes. The overall aim of the evaluation was to examine whether the reforms were implemented as intended; consider the extent to which the short and medium term outcomes were realised; and to identify potential obstacles to realising those and longer-term goals.

The evaluation drew on a variety of quantitative and qualitative methods including:

• Analysis of national monitoring data relating to the use and outcomes of CPOs
• National surveys of the judiciary and criminal justice social work managers
• Qualitative interviews in each of four case study areas
• A 'participative audit' of Criminal Justice Social Work reports in the four case study areas, which collated both factual information and subjective assessments of the quality of analysis and recommendations contained within the reports.

Criminal Justice Social Work Reports

In a number of key respects, the new template appears to be delivering on its original objectives. Despite the fact that many criminal justice social work (CJSW) staff believe that the new template is more time consuming than previous court reports to complete, the vast majority of CJSWRs are being delivered on time. The standardisation of reports, within and across local authority areas, has been welcomed by Sheriffs and CJSW staff. And it is widely felt that the template has improved navigability and increased the focus on offending.

There is, however, a widespread concern among Sheriffs that the new format has contributed to long and overly detailed reports. CJSW staff and managers also express concern about 'writing to the template' and highlight the risk that the quality of analysis suffers as a result.

Sheriffs appear to find much of what they receive in CJSWRs helpful, but are especially positive about the information about individuals' current and previous offending, personal and social circumstances and suitability for a community disposal. They are more ambivalent about the analysis of risk provided by report writers and markedly less positive about social workers' reviews of relevant sentencing options and conclusions about preferred sentencing options.

The participative audit suggests that reports are drawing on a broad and appropriate range of sources but also raises questions about the quality of analysis contained within reports, especially in relation to risk of reoffending and risk of harm. Overall, there is room for improvement in the quality of reports, with around one in six being given a negative rating by auditors.

There are issues here relating to the availability of training for new starters, and the extent to which training and support around CJSWRs in general moves beyond the transactional to focus on report writers' analytical skills. It also appears that the new template is more time consuming to complete than the previous court reports. While this has been partly offset by a reduction in the number of reports requested - and is seen by many CJSW staff as worthwhile if it leads to better quality - there are potential resource implications here.

Procedures for quality assurance and retrospective audit are inconsistent across and within areas. Although there is a solid base of peer support and line management, formal or systematic processes of assurance and review are often absent. This makes it harder to identify the minority of below-standard reports and also means that a potentially 'virtuous loop' of skills improvement is not completed.

Given the central role that offenders' accounts play in the preparation of CJSWRs, it is important that they understand the process and feel able and willing to contribute to it openly and honestly. For the most part, this seems to be the case, although some offenders are still unprepared for the depth of questioning in the preparation of the report or do not fully grasp its purpose. The skills and experience of report writers - and the nature of their relationship with the offender - are clearly critical here. Most of the offenders interviewed for the evaluation acknowledged the role that the report had played in securing a community disposal; a few felt that it had played a major role in tailoring the sentence to their needs; and others were simply grateful for the way that it helped prepare them for the sentencing process.

An overview of CPOs imposed

By 2012-13, CPOs accounted for more than 80% of all social work orders imposed, increasing by roughly 5,500 in absolute terms over the previous year. There is no sign, however, that their introduction has led to a marked or immediate upturn in the total number of social work orders imposed.

Two of the nine requirements, Unpaid Work or Other Activity (UPWOA) and Supervision, were used with far greater frequency than any other - not surprisingly as all orders must contain one or both such elements. The use of UPWOA is especially common and became even more pronounced in 2012-13.

None of the remaining requirements was imposed in more than one in ten orders and, with the exception of Compensation, all were used relatively less frequently in 2012-13 than in the year before. While the sharp reduction in the use of the Conduct requirement can be directly attributed to an appeal court ruling, the contraction in the use of some of the 'treatment' requirements - especially Drug Treatment Requirements (DTRs) and Alcohol Treatment Requirements (ATRs) - needs to be more fully understood.

Multiple requirements are not the norm. Roughly half of all orders (55%) in 2012-13 involved a single requirement - usually UPWOA - up from 43% the year before. The fall in the average number of requirements over the same period (from 1.82 to 1.59) should also be noted. This raises questions about the willingness of CJSW staff to recommend, and of Sheriffs to impose, multiple requirements and the implications for the expected tailored and flexible character of the order.

There is wide variation in the use of different requirements across local authority areas. While this will reflect, to some extent, differences in the characteristics of these areas and the nature of the cases coming before the courts in each, it is also likely to signal a degree of local variation in interpretation and/or practice.

This variation is especially clear in relation to the use of Drug Treatment and Testing Orders (DTTOs) and DTRs. Although the legislation and practice guidance signal a relatively clear distinction between these two options, across local authorities, it is clear that higher use of one tends to be associated with lower use of the other, suggesting a more blurred interpretation on the ground.

Gender is strongly related to the type of CPO requirements imposed. Women - and young women in particular - were relatively less likely to receive an UPWOA requirement, but more likely to receive a CPO with a DTR, Mental Health Treatment Requirement (MHTR) or Residence requirement.

Use of the Unpaid Work or Other Activity requirement (UPWOA)

There appears to be a reasonably wide range of Unpaid Work (UPW) placement types available. All four case study areas were making similar use of group placements (in the form of both workshops and project groups). There was greater variation, however, in the availability and use of female only and individual placements.

Although CJSW staff in most areas indicated that they had struggled to cope with the initial increase in UPW placements following the introduction of the reforms, the situation now seems to have eased somewhat - in part, because of an expansion in the availability or use of individual placements. However, the 'supply' of such placements remains somewhat patchy and ad hoc.

Key challenges in terms of overall provision include competition for individual placement opportunities from other sources (such as large employers offering their staff the opportunity to volunteer); variation in the number and range of local organisations potentially able to offer placements; and funding constraints limiting the use of workshops or other resources.

Flexibility of provision is both a desirable and a necessary characteristic of UPW provision - desirable because of the aspiration to tailor disposals to individual needs and circumstances; necessary because of the often highly unstructured character of individual offenders' lives. Consequently, across the case study areas, UPW provision allowed offenders a degree of flexibility about when and how to complete their hours.

CJSW staff face a number of challenges in trying to assess suitability and match offenders to particular UPW opportunities. For example, it can be difficult to match the skills of an offender to the placements available; to find appropriate placements for those for whom English is not the first language or who have complex needs. Other challenges include the difficulty of finding suitable community placements for sex offenders.

There are also significant issues around the imposition of Level 1 orders in the absence of a CJSWR, in terms of the potential for both needs and risks to be missed and the potential for offenders to be given unsuitable placements. There was some evidence of the emergence of informal local arrangements to address these.

There is a widespread perception that UPW placements are commencing faster than in the past and that a large majority are certainly meeting the target of commencement within seven days of the CPO being imposed. Around one in five, however, are not meeting this target. There is also evidence that the emphasis on speed of commencement may be shaping the type of initial placements available, and that matching to a more suitable placement to complete the rest of the order may sometimes be delayed.

In terms of speed of completion, the timescale for Level 1 orders (three months for up to 100 hours of UPW) is seen as unrealistic by many CJSW and UPW staff, although some feel that this relatively compressed time frame could actually be useful in keeping momentum going from the start to the finish and motivating staff to stay vigilant to ensure offenders got through their order.

The 'Other Activity' (OA) element was slow to develop following the introduction of the reforms. CJSW and UPW staff needed time to adjust to the new reforms as a whole, and to manage increased numbers of offenders with UPW orders, leaving little scope for a clear focus on OA. A lack of local guidance also means that some CJSW and UPW staff still lack a clear understanding about what OA consists of or how it might be used.

While there are increasing signs of OA being used creatively and effectively, the flexibility that allows it to be closely tailored to individual need also leads to variations in use. In combination with differing views and interpretations of how and when it should be used, this has meant that OA is used inconsistently within and across local authorities.

Use of the remaining CPO requirements

The three 'treatment' requirements - Alcohol Treatment Requirements (ATRs), Drug Treatment Requirements (DTRs) and Mental Health Treatment Requirements (MHTRs) - are being used relatively rarely but also inconsistently across different areas. There are issues here around differences in understanding of the exact target group for each and of what constitutes treatment.

From a CJSW point of view, however, the most significant obstacle to proposing an ATR or MHTR is the apparent necessity to obtain a medical assessment in advance of sentencing, although, in a few areas, it appears that some of these requirements are being imposed without a formal medical assessment. While there are some indications that this latter approach may lead to a higher level of use, some participants feel it may also run the risk of such orders being returned to court.

Even if the relevant assessments can be obtained, an ATR or MHTR (or indeed a Programme requirement) all need the treatment to be specified in detail at the point of sentence. This can again be difficult within the timeframes for preparation of the CJSWR.

For all three treatment requirements, waiting times for certain types of treatment (for example, residential detox and psychological interventions) are often lengthy - indeed, waiting times to begin such treatment are often longer than the length of the CPO itself. This also has the effect of limiting their use.

Although most CJSW staff are more content with the scope and wording of the DTR than they are with that of the other 'treatment' requirements, some remain uncertain about exactly when a DTR should be used instead of a DTTO. CJSW interviewees also gave examples of Sheriffs imposing a DTR when they felt a DTTO would have been more appropriate, and vice versa.

It is clear that many CJSW staff have a concern about the 'layering on' of multiple requirements and this, too, may be a factor in limiting the use of the treatment requirements.

In the context of the various issues described above, it is perhaps not surprising that alcohol, drug or mental health issues are commonly addressed under the Supervision requirement. While this may often be entirely appropriate, there was also some evidence of concern about its use as a potential 'catch all' and about the effectiveness and transparency of work conducted under Supervision by comparison with that which might occur under one of the more structured requirements.

While interviewees suggest that the needs of offenders with drug, alcohol or mental health issues are generally being met, there is considerable variation in exactly how this is happening. Moreover, the lack of transparency around these issues has potential implications for effective service planning and provision.

Engagement, compliance and breach

CPOs with a single requirement are more likely to be completed than those with more than one, though an increase in the number of requirements beyond two does not raise the risk of non-completion further. The combination of requirements does appear to influence chances of successful completion.

More than half of CPOs completed in 2012-13 had UPWOA as their only requirement - and it was these orders that had the highest rate of successful completion and the lowest rate of being revoked due to breach.

Those with just a Supervision requirement had the next highest rate of successful completion and next lowest rate of revocation due to breach. CPOs containing a Programme requirement (alongside a Supervision requirement) had the lowest levels of successful completion.

CPOs containing an ATR had high levels of successful completion when there was no additional UPWOA requirement. The same pattern can be seen for CPOs containing a Compensation requirement.

Although progress reviews were previously available for DTTOs and probation orders there was a perception that their use has increased under CPOs. The flexibility with which reviews could be used was welcomed by CJSW staff and Sheriffs, though inevitably leads to variation in use.

There were still some issues around who has responsibility for issuing warnings/breaches in joint orders with supervision and UPWOA - specifically around first and final warnings.

Sheriffs appear to have broad confidence in CPOs in terms of monitoring of progress and appropriate use of breach. However there are still some areas in which it is felt the processes could be improved - especially in relation to the length of time it takes to declare a breach and the number of warnings given. Some Sheriffs were also unhappy that further offences do not constitute a breach of a CPO. CJSW interviewees, by contrast, were more likely to appreciate the flexibility that this provides in allowing them to continue to work with offenders following a subsequent offence.

Most offenders felt their CPO had been clearly explained to them; that they knew what was expected of them and that they understood what would happen if they breached the order. Signing the order, or an agreement, seemed to help this understanding.

Offenders were usually very positive about the relationship they have (or had) with their case manager, citing this relationship as being of key importance for engagement and compliance.

Engagement and compliance was most likely when a CPO was tailored to an offender's needs and interests. Other factors that offenders responded positively to included the fact they were paying back to the community, and the sociable element of UPW.

Judicial decision-making in relation to community penalties

The two years following the introduction of the reforms saw both an increase in the use of community penalties and a fall in the use of short prison sentences in Scotland - two key long-term objectives for the reforms. Caution is warranted in the interpretation of these figures, however, and it would certainly be unwise to draw a direct line to the reforms. The increase in the use of community penalties is consistent with a much longer-term trend, while the fall in the use of sentences of three months or less has been accompanied by an increase in the number of sentences of three to less than six months and six months to less than two years - a trend which also predates the introduction of the reforms.

Nevertheless, on balance, there is evidence that CPOs are seen by Sheriffs as an improvement on previous community penalties, have contributed to a greater willingness to use such disposals, and are viewed with a reasonable degree of confidence by most Sheriffs. While many Sheriffs see little change in the balance of their use of community and custodial disposals, there is a minority who say they are using community penalties more since the introduction of CPOs and there are almost none who say they are using them less.

Those Sheriffs who were strongly positive about the reforms typically welcomed the range and flexibility of the options on offer to them, the simplification of the overall framework and evidence of swifter implementation and more rigorous enforcement.

A minority of Sheriffs indicated that they would like to make still greater use of the UPWOA requirement, reflecting concern about the resourcing of UPW and provision for offenders with particular needs. There was also evidence that Sheriffs would like to be able to make greater use of the treatment requirements and other, more highly structured, programmes and interventions.

One of the aspects of the reforms that Sheriffs are most positive about is the introduction of Level 1 orders, especially for lower tariff, young male offenders and those on benefits, for whom it is seen as a more constructive option than a fine.

There was little sign from the qualitative interviews of PASS figuring prominently or explicitly in decision-making in relation to specific cases - in part because Sheriffs indicated that they already used short prison sentences very rarely, but also because sometimes such a disposal was the only option left to them. That said, the survey found that some Sheriffs felt that PASS had contributed to an increase in their use of community penalties or slightly longer sentences.

Apart from the seriousness of the offence, the clearest marker of the 'inevitability' of a short sentence is not repeat offending, but serial non-compliance - particularly in the light of clear and repeated warnings about the consequences of flouting existing orders. In the case of wilful non-compliance, this can have an explicitly punitive aspect; for others, whose lives are simply too complex and chaotic to break the cycle of offending, a short prison sentence is sometimes viewed as offering the chance to dry out or to 'wipe clean the slate' of accumulated penalties.

There is considerable variation in the attitude of Sheriffs towards the preferred sentencing options (or recommendations) contained in CJSWRs. Most Sheriffs say they are happy to receive 'a steer' on the most suitable disposal. While some like to see these couched in relatively cautious terms - avoiding any sense of encroachment on the independence of the bench - others called for social workers to be more creative and directive. Differences in Sheriffs' attitudes towards this issue are likely to be a reasonable proxy for their wider attitudes towards the respective roles of social workers and the judiciary.

There was some scepticism among Sheriffs about the seriousness with which CPOs would be viewed by offenders and, especially, the acceptability of such disposals to the general public - although some interviewees were at pains to emphasise that they had little access to reliable evidence about either issue.

There was a sense that elements of CPOs such as unpaid work were still insufficiently visible to communities, and that it would be helpful for members of the public to understand more about the reparative aspects of such programmes.

Overview: Were the reforms implemented as intended?

The introduction of CPOs and the new CJSWR represented a major practical challenge for CJSW and the Scottish criminal justice system more generally. This initially slowed down implementation of some elements and meant that some of the more visible and familiar elements of the reforms tended to be foregrounded. As a result, some of the processes and requirements were established relatively quickly - for example, UPW and Supervision, which directly mirrored work previously undertaken under CSOs and Probation. Other less familiar and more complex elements (such as OA), by contrast, received less in the way of immediate focus and attention. Other requirements - such as DTRs, ATRs and MHTRs - presented similar challenges.

In relation to CJSWRs, the scope for staff engagement was again limited by the range of other simultaneous developments and, in particular, the introduction of Level of Service/Case Management Inventory (LS/CMI). However, the basic template was up and running in time and there was no evidence of any major disruption to the availability of reports when needed by the courts. The introduction of PASS had little in the way of immediate practical consequences for Sheriffs or court staff, though it did introduce a requirement for reasons to be formally recorded for any short sentences passed by the court.

Overview: What evidence is there that the short and medium term outcomes have been realised?

The introduction of the new CJSWR template does not appear to have had an adverse impact on the timeliness of reports, and it is generally recognised that it has led to greater consistency in the form and content of reports. This has been welcomed by some Sheriffs, who find the reports easier to navigate, and by some CJSW staff, who feel that the process is more structured and focused on offending than previously. The concerns about length and duplication, however, suggest that the aspiration of more concise reports is not yet being met. And there is evidence (from both Sheriffs and CJSW staff) of room for further improvements in the overall quality of analysis and recommendations contained within the reports.

There are signs that, on balance, the introduction of CPOs has improved (and certainly not harmed) judicial confidence in community penalties, but that overall attitudes have not been radically transformed.

There was also widespread evidence of a commitment to use community penalties wherever possible - and even in the face of serial offending or non-compliance - if there was any indication from the CJSWR of scope for constructive engagement. Whether this approach can be directly attributed to the introduction of CPOs (or to other aspects of the reforms) is less clear.

Sheriffs remain sceptical about the seriousness with which CPOs will be regarded by offenders and their acceptability to the general public. Both factors are potentially important background influences on their own confidence in the appropriateness of the disposal.

There is relatively little sign of PASS figuring prominently or explicitly in judicial decision-making; although some Sheriffs did suggest it had been a background factor in avoiding a short prison sentence in a small number of cases. Most, however, considered the presumption to be of little practical consequence both because of an existing commitment to use community penalties wherever possible and because of the 'inevitability' of a short custodial sentence in a small number of cases.

A full set of CPO requirements is technically available and all are being used to some extent. However, the evaluation raises some important questions about whether some specific requirements are being used appropriately and with sufficient frequency. There are particular issues around the level of use of the three 'treatment' requirements (ATRs, DTRs and MHTRs). Differences in interpretation of the National Outcomes and Standards (NOS) Practice Guidance and other factors (for instance, relating to service availability) may be creating inappropriate diversity in the extent to which these are being deployed across Scotland.

Most offenders interviewed for the evaluation seemed to have a good understanding of what was expected of them as part of their CPO, and of what might happen should they fail to comply. There was positive feedback about the nature and extent of social work support received under Supervision and evidence of the importance of relationships with individual CJSW staff.

Around two-thirds of CPOs in 2012-13 were completed - a figure broadly consistent with that achieved under the previous framework for community penalties. Despite this, there is a widespread perception among both CJSW staff and Sheriffs that the arrangements for the monitoring of compliance and enforcement of breach are more robust as a result of the reforms.

Where now?: Potential next steps suggested by the results of the evaluation

There would be benefit in incorporating feedback from Sheriffs about the length and relevance of reports into training for CJSWR writers. Previous exercises involving the piloting of summary reports should be revisited, along with the scope for making greater use of oral reports in certain types of less complex cases.

Opportunities for additional joint work involving Sheriffs and CJSWR report writers should be identified in order to develop shared language and understanding around risk, and agreement about the types of cases in which a more or less thorough assessment is warranted. This would also allow for greater mutual understanding of roles and priorities, sharing of concerns and improvement of the fit between what Sheriffs feel they require and what CJSWR writers provide The scope to increase CJSW access to prosecution summaries should also be revisited and the results of previous pilot exercises re-examined.

The Scottish Government should provide regular national oversight of the training provided to CJSWR report writers within individual local authority areas; and that training should explicitly aim to improve the quality of analysis and recommendations, as well as covering the technical requirements of the template.

Estimates of the time required per report need to be reviewed and cannot be assumed to be consistent over time. The balance of staff time spent on report-writing and other aspects of casework should also be monitored.

Local authorities should have access to clear best practice guidance about the use of quality assurance as part of routine working practices. Consideration should also be given to regular intra- and inter-local authority audit of samples of reports. The findings of such exercises should be fed back into training and practice.

The Scottish Government should seek to improve understanding of the current use of OA - perhaps through a review of its use in each area and the identification of examples of good practice. Insights from such work should be incorporated into any further training on CPOs and taken into account in consideration of funding requirements.

The wording of the NOS Guidance - and, if necessary, the legislation - should be reviewed in order to identify, on a requirement by requirement basis, issues that seem to be creating uncertainty or misunderstanding.

There is scope for greater clarity around the role and responsibility of partner agencies (especially within the NHS) to provide rapid access to assessment and services in relation to the 'treatment requirements'. If such involvement cannot be provided or resourced, the implications of this need to be reflected in the wording and guidance for the relevant CPO requirements.

The Scottish Government should seek to develop a fuller understanding of how the treatment requirements are being used - for example, by undertaking a review of cases in which a DTR, ATR or MHTR has been imposed, and an audit of such cases returned to court. Local authorities should perhaps be asked - as part of the annual CPO returns or a bespoke exercise - to provide numbers of offenders subject to a CPO who are in receipt of support or treatment for drug, alcohol or mental health issues via another requirement.

Published guidance could provide greater clarity about the circumstances in which the use of Supervision, in particular, is appropriate; and there could be improved monitoring of the extent and outcomes of work conducted under these more flexible provisions.

There is a need to build on and extend emerging good practice in relation to the use of Level 1 orders - for example, encouraging Sheriffs to make enquiries at sentencing about individuals' suitability for UPW and making greater use of oral reports where appropriate.

Sheriffs might benefit from regular briefings about local and national use of the various requirements, and easy access to research and statistical evidence about outcomes and effectiveness. There needs to be discussion about how such information might be made available, and by whom.

Court user groups and other cross-professional fora might also be encouraged to look specifically at expectations and experiences of the various aspects of the reforms.

Local authorities should be required to provide clear plans for the prospective management of community consultation, rather than just retrospective examples. Opportunities should be explored for providing Sheriffs with summaries of research evidence about public attitudes in this area.

In conclusion, the reforms have provided a framework that has helped to move community penalties in the right direction. However, the key long-term outcomes - around reducing reoffending, increasing reintegration and reducing 'churn' in the prison population - will only be realised if all three elements are subjected to ongoing scrutiny and analysis and there is a system-wide commitment to partnership working and continuous improvement.

This document, along with full research report of the project, and further information about social and policy research commissioned and published on behalf of the Scottish Government, can be viewed on the Internet at: http://www.scotland.gov.uk/socialresearch. If you have any further queries about social research, please contact us at socialresearch@scotland.gsi.gov.uk or on 0131-244 2111.