Assessing the effectiveness of inspection activity: HMFSI report

While responsibility for delivery standards and for improving public services lies with the organisations that provide the services, public bodies are generally subject to external scrutiny as a check, challenge, and reassurance on how they are performing. External scrutiny can influence behaviour and culture, and this can encourage and stimulate improvement.

External scrutiny bodies must themselves be accountable for their activity and scrutiny challenges need to be effective. For external scrutiny to be effective, scrutiny activity should have a defined impact on the ground and the scrutiny body should be aware of the effectiveness and the impact of its activities on the service scrutinised.

Fire and Rescue Service scrutiny

Inspectors of the Fire Service Inspectorate have scrutiny powers1 to inspect the Scottish Fire and Rescue Service (SFRS) in respect of how it carries out its functions and its state and efficiency.

The Chief Inspector has issued a scrutiny plan which identifies how the Inspectorate carries out the scrutiny role. A principal component of this role involves undertaking two types of inspection:

1. cyclical inspections of geographical areas – the local area inspections (LAI) of the SFRS based on local authority area
2. thematic inspections of the SFRS on specific functional areas

After each inspection the Chief Inspector publishes a report with findings and, where appropriate, includes recommendations to the Service for future action. We issue our LAI reports to the relevant SFRS Local Senior Officer. We issue our thematic inspection report to the SFRS Chief Officer.

Our reports contain recommendations that we think will make a tangible difference to the work of the Service and it is our expectation that the Service should use our inspection reports to address highlighted issues.

A list of inspection reports, and inspection activity, can be accessed at: https://www.gov.scot/collections/hm-fire-service-inspectorate-in-scotland-reports/

We routinely follow up the recommendations in our LAI reports by returning to the area approximately 6 months after the inspection and meet with the relevant LSO to see what action the Service has taken locally in response to our report. This helps us assess the usefulness, effectiveness, and appropriateness of our LAI work and the progress made by the Service in meeting our recommendations.

Follow up work on our thematic reports and their recommendations, and on national impacts identified in LAI reports, has been less structured. There has been some ad-hoc assessment and follow up, especially where there may be an overlap with a later inspection. But we have not applied a structured approach and therefore do not have detailed knowledge of how effective our thematic inspection work is in delivering improvement in the Service. A structured follow up approach to our thematic inspection work is now being introduced which will follow a similar approach to LAIs, although the timescale for a follow up will vary and be influenced by the complexity of a report’s recommendations.

Proposal

We propose to conduct an inspection into the impact of our scrutiny of the SFRS. We will do this by examining the manner in which the SFRS implements our thematic inspection report recommendations, and any national issues arising out of LAI reports, and considering the value the Service places on this work.

During the course of this Inspection we will be focusing our attention on the general policy for implementing recommendations, the practice in implementing recommendations, and specifically how the Service deals with the recommendations in our inspection reports and what outcomes have been achieved.

By doing this, we will improve our understanding of the benefit and impacts of our inspection work and may highlight ways in which we can improve how we conduct our inspection activity. The results of this Inspection will provide an insight into the effectiveness of our thematic scrutiny activity and the corporate culture of the SFRS to external scrutiny. In addition it will enable us to give robust assurance to Scottish Government and Ministers as to the methodology and progress of the recommendations made within our reports.

Methodology

This inspection will follow the guidelines used for previous thematic inspections. The Inspection framework will provide a structure that will be risk based, proportionate and focused.

The inspection will involve a desk top review of the SFRS’s policy in this area. This will be complemented by face to face interviews.

The inspection will be delivered in four stages, some of which will be undertaken concurrently. These are:

Stage 1 – Scoping

Initial engagement with the SFRS to establish a single point of contact. A review of documents in relation to SFRS policy and practice.

Stage 2 – Fieldwork

The fieldwork will include:

• interviews with persons who determine or influence SFRS policy - Directors, other strategic managers and Board members
• interviews with persons charged with implementation of report recommendations and those responsible for monitoring and reporting on progress
• review the recommendations within our thematic inspection reports, together with national issues identified within the LAI reports, to specifically examine what changes have been introduced as a result of these reports

We will consider speaking to other scrutiny bodies regarding their experience in this area and to other users of our reports to gauge their views.

Stage 3 – Analysis of evidence

During this stage, Inspectors will review and evaluate the information and evidence collected and if necessary expand the area of enquiry to address any issues arising or areas of concern that are raised. This may result in further information being required from the SFRS and interviews with other members of staff.

Stage 4 – Publication and reporting

A copy of the report will be provided to the SFRS Board, the Chief Officer and laid before the Scottish Parliament. A copy will also be made publicly available on the HMFSI website.

The report will outline our methodology and approach, report on the Inspectorate’s key findings, conclusions and identify any recommendations we think appropriate.

Timetable

The proposed timetable is:

• September/October 2019: pre-planning and consultation on the inspection outlines
• October/November 2019: data collection and review of documentation
• December 2019/February 2020: fieldwork interviews with SFRS managers and board members
• March/April 2019: report drafting and consultation
• May/June 2020: report publication

The inspection team

The Inspection team members are:

• Simon Routh-Jones – Chief Inspector
• Brian McKenzie – Assistant Inspector – Lead Officer
• Graeme Fraser – Assistant Inspector

The team’s work is independently reviewed for quality assurance.

Respond to this inspection outline document

This Inspection Outline document has been prepared to describe why we are carrying out an inspection and how we will go about the work. It is also intended to support our consultation with Ministers, the SFRS management team and representative bodies. If you wish to make any comments or observations, these should be submitted to the Chief Inspector by emailing

The team members can also provide clarification on any content and can be contacted through the same email address.