Multi-Agency Public Protection Arrangements (MAPPA): national guidance

19. Performance Management and Quality Assurance

19.1 For the MAPPA framework to work effectively each of the Responsible Authorities must fulfil their statutory duties (as detailed in Sections 10 and 11 of The Management of Offenders etc. (Scotland) Act 2005) and show that by working collaboratively they meet the requirements of comprehensive and defensible risk assessment and management of individuals subject to MAPPA.

19.2 The Responsible Authorities individually, and through the Strategic Oversight Group (SOG) need to be satisfied that the multi-agency arrangements in their area are working well to maintain effective public protection, monitoring and evaluating performance.

Objectives

19.3 The two key objectives that should be at the forefront of monitoring and evaluating MAPPA performance are:

• To provide evidence that the defensibility test is being met, i.e. was everything done that could reasonably have been done to prevent MAPPA individuals from re-offending?
• To provide evidence that the Responsible Authorities are fulfilling their statutory duties and MAPPA is being delivered effectively.

19.4 In order to meet these objectives, it will be necessary to collect, monitor and analyse a range of quantitative and qualitative data.

Collection and Analysis

19.5 Monitoring and evaluating the activities of the SOG will drive the business planning, contribute to the MAPPA Annual Report and provide the means for reviewing the effectiveness of MAPPA in each area against the national MAPPA Performance Indicators.

19.6 The collection and analysis of both quantitative and qualitative data will assist in measuring and identifying issues (such as good practice and areas for improvement). The MAPPA Co-ordinator will collate and provide this information to the SOG.

19.7 It is for the MAPPA Co-ordinator to analyse the data in the first instance and present to the SOG. This will assist the SOG in their review and validation of the effectiveness of MAPPA in their area.

19.8 Any issues that require urgent action should be raised with the SOG Chair and relevant agencies without delay.

19.9 As a minimum, the SOG should review quantitative and qualitative data every quarter. This enables the SOG to assess performance, provide an opportunity for bench marking and for timely interventions to be put in place where issues are identified.

Quantitative Data

19.10 The Responsible Authorities and the SOG should ensure that quantitative data is used to inform and improve performance.

19.11 Data that is required for the MAPPA Annual report can be reviewed on a regular basis to support the SOG in monitoring and improving the performance of MAPPA. Further information on the data required for the Annual Report can be found at Annex 3 of this guidance.

19.12 The data, and changes in data over time, will indicate whether further questions need to be asked to identify and address any performance issues. For example, if the numbers of cases managed at a certain level (e.g. Level 2) decreases significantly, individual agencies may want to check that their own assessment of cases is still effective. Similarly, a marked increase in Level 2 cases might be because agencies are referring cases into MAPPA due to a lack of confidence in ability to manage a case, rather than through a genuine need for active inter-agency management.

19.13 ViSOR can produce management reports on a monthly, quarterly and annual basis in order to inform local performance measurement. These reports are not intended for the public domain with the exception of the data that is used for the MAPPA Annual Reports. If the data on ViSOR is accurate, it can provide a fast and efficient way of providing much of the data that is required for the MAPPA Annual Report. Further detail on the system can be found at Chapter 17.

19.14 Advanced searches can be run on ViSOR to gather both numerical and individual-related information regarding relevant MAPPA cases. It can also report via pre-set functionality on a range of relevant measures. Each pre-set ViSOR statistical report has a number of variables that users can select including age, gender, ethnicity, occupation, MAPPA category, management level, and risk level. Reports which can be run include:

• The total number of individuals subject to MAPPA by category and at what Level (1, 2 or 3) for a specific time period; including those being managed in the community. This can be further broken down into:
  • The total number of individuals subject to the Sex Offender Notification Requirements (SONR) being managed at all levels in the community;
  • The total number of Other Risk of Serious Harm individuals being managed at Levels 2 or 3 in the community;
• New cases by category and Level being managed in the community for a specific time period;
• Cases which are archived by category and Level for a specific time period;
• The number of wanted or missing individuals (see Chapter 8 for more detail and definitions of wanted and missing individuals) within a specific time period - including the length of time they have been wanted or missing and the number of wanted or missing individuals who have been located within a specific time period;
• The number of individuals subject to MAPPA by category and level who commit a further sexual or violent offence;
• The number of civil and criminal preventative orders which have been applied for, the number granted, the number refused and the number made by the court at point of criminal conviction;
• The number of breaches of preventative orders as required; and
• The number of breaches of licence or orders by category and level where the individual was recalled to prison.

Qualitative Data

19.15 The Responsible Authorities and the SOG should ensure that qualitative data is used to inform and improve performance.

19.16 Analysis of qualitative data gives a more detailed breakdown of how activities contribute to outcomes. It enables a judgement to be made about the quality of actions undertaken individually or collectively and an assessment of how this contributed to an outcome. Learning points and plans for improvement can then be identified.

19.17 Qualitative information can be obtained from:

• Audit of MAPPA Level 2 and Level 3 cases;
• Audit of MAPPA Level 2 and Level 3 meetings;
• MAPPA Significant Case Reviews; and
• Peer review.

Audit of MAPPA Level 2 and Level 3 Cases

19.18 The MAPPA Co-ordinator, on behalf of the SOG, should undertake or co-ordinate a random case audit of a selection of Level 2 and Level 3 cases. The purpose of the audit is not to inspect the work of individual agencies, although how they work will affect the overall quality, but to evaluate how MAPPA operated. The frequency of this should be agreed locally but must be at least once every reporting year. To assist with this, there are suggested case audit forms for Level 1 case audits, Level 2 and 3 case audits and completion notes for Level 2 and 3 (where required) found within MAPPA document set of this guidance.

19.19 As a minimum the Level 2 and Level 3 case audit should:

• Review the referral and minutes of the MAPPA meetings;
• Identify whether all agencies were present at all relevant meetings to allow the work of the MAPPA meeting to progress effectively;
• Review the quality of MAPPA Risk Management Plan(s);
• Identify if agencies completed the actions assigned to them in a timely manner;
• Identify areas of effective practice;
• Identify areas where practice, co-operation and quality of information between agencies could be improved; and
• Identify how feedback will be provided to the SOG and relevant agencies.

Audit of MAPPA Level 2 and 3 Meetings

19.20 To ensure that MAPPA meetings are effectively managed, the MAPPA Co-ordinator should put in place a process to support Level 2 and Level 3 MAPPA meeting Chairs through feedback and observation of their performance and the effective conduct of the MAPPA meetings. Ideally this should take place on a quarterly basis. To assist with this, there is a suggested MAPPA meeting audit form found within the MAPPA document set of this guidance.

Level 1 meetings

19.21 Local arrangements should be put in place for carrying out audits for Level 1 cases. It is recommended that as a minimum these audits should:

• Identify whether all agencies were present at all relevant meetings to allow the work of the MAPPA meeting to progress effectively;
• Identify effective processes and practices;
• Identify areas where practice and communication between agencies could be improved; and
• Ensure feedback and assurance to the SOG and relevant agencies.

MAPPA Significant Case Review

19.22 Where an individual subject to MAPPA has been the subject of a Significant Case Review (SCR), the SOG will, following the publication (in part or in full) of the SCR report;

• Establish a process for managing the delivery of the findings and recommendations of the report;
• Prepare action and implementation plans and establish a means of monitoring progress;
• Seek regular updates on the progress of actions for national agencies;
• Establish audit processes to ensure all findings and recommendations have been considered and an acceptable outcome and implementation process agreed; and
• Liaise with the Scottish Government and others as required.

19.23 When evaluating the performance of an SCR, it is expected that the effectiveness of each stage of the process would be considered:

• Development of the remit;
• Identifying and commissioning the Lead reviewer and the Review Team;
• Resources;
• Management of the process;
• Liaison with the victim and/or nearest relative/partner/family;
• Staffing;
• Production, handling and publication of the report (including communication strategy; and
• Identification of learning opportunities and good practice in order to promote continuous improvement and how these were communicated to interested parties.

19.24 Further detailed information about the SCR process can be found at Chapter 20.

Peer Review

19.25 There are a variety of ways to obtain information to assist and provide a method of benchmarking the effectiveness of local MAPPA arrangements and confirming the appropriateness of decision making. This is particularly useful for MAPPA Chairs and regular peer reviews can be organised by the MAPPA Co-ordinator.

National MAPPA Performance Indicators

19.26 Although the national MAPPA Performance Indicators measure actions rather than outputs, they have been designed to ensure that the critical pre-conditions are in place for the effective operation of MAPPA (e.g. the right people, from the right agencies, meeting at the right frequency). If these are not in place, MAPPA is unlikely to operate effectively as it should.

19.27 By summarising the extent of performance against each of the National Performance Indicators, it can also assist when viewing the performance of other areas. An analysis of areas where performance may be below target will indicate what further information is required in order to undertake remedial action. For example, not meeting the performance indicator for reviewing Level 2 and Level 3 cases, may be as a result of an increase in the number of MAPPA individuals across all categories for that area, which has impacted on resources.

19.28 This notwithstanding, SOGs will be aware of the importance of evaluating the effective operation of active and alert multi-agency management of those individuals assessed as a high or very high risk and managed at Level 2 or Level 3.

19.29 Every SOG should evaluate its performance against the following National Performance Indicators:

• 95% of Level 3 MAPPP cases reviewed no less than once every six weeks. Where cases fall outwith, this the reasons must be clearly detailed so that any commonalities can be identified. For example, is it the same case repeatedly not being reviewed and if so is that due to a specific agency or lack of recurring information being received in time?
• 95% of MAPPA Level 2 cases reviewed no less than once every twelve weeks. Where cases fall outwith this, the reasons must be clearly detailed so that any commonalities can be identified. For example, is it the same case repeatedly not being reviewed and if so is that due to a specific agency or lack of recurring information being received in time?
• Where invited, 90% attendance by each Duty to Co-operate (DTC) agency at an appropriate level of seniority for Level 2 and 3 meetings in community and custody. This includes attendance via telephone/video conferencing or a written report.
• Where invited, 90% attendance by Police Scotland at the agreed level of seniority for Level 2 and 3 meetings in community and custody including Scottish Prison Service (SPS) Risk Management Team (RMT) meetings.
• Where invited, 90% attendance by Community Based Social Work at the agreed seniority level for Level 2 and 3 meetings in community and custody including SPS RMT meetings.
• 100% written contributions to be provided by Responsible Authorities in cases of non-attendance (where invited) at Level 2 and Level 3 meetings in the community or custody. Upon receipt these contributions will be recorded in the minute and Risk Management Plan.
• 100% written contributions to be provided by Duty-to-Co-operate agencies in cases of non-attendance (where invited) at Level 2 and Level 3 meetings in the community or custody. Upon receipt these contributions will be recorded in the minute and Risk Management Plan.
• 100% of MAPPP initial Level 3 meetings for those in custody to take place in the community no less than four weeks prior to the person being released. In circumstances where this has not happened, the reasons should be clearly detailed so any commonalities can be identified. For example, the referral was not received within agreed national timelines, or there was no risk of serious harm assessment carried out.
• 90% of Level 2 and Level 3 meeting minutes and updating of Risk Management Plans to be issued and ratified within 10 working days following the review meeting.
• Disclosure to be considered and the decision to be recorded in the minutes at 100% of Level 2 and 3 MAPPA meetings.