Warmer Homes Scotland - first annual review

Scope

The scope of the review encompasses each of Warmer Homes Scotland's objectives to make an early determination of whether or not they are being met and whether or not the scheme is delivering in line with its guiding principles. The review considers operational delivery in the period from go live on 1 September 2015 to the end of that financial year on 31 March 2016. The review also examines policy decisions that were made as the scheme was being developed to consider issues including:

• Is the scheme reaching the people who need it?
• What behaviours are being driven by the contractual Key Performance Indicators?
• Does the scheme offer the right mix of measures? And
• Is there equity in delivery between rural and urban areas?

The review does not consider the period of mobilisation from May to September 2015 as this has already been the subject of an internal gateway review and audit by Pennington Choices.

The scope of the review was agreed by the HEEPS Delivery Board and Warmer Homes Scotland Strategic Board.

Methodology

In developing the scope of the review, a number of questions were raised against each of the objectives that would help to determine whether or not it is being met and to what degree. It was also established at this time what data or other evidence would be required to make these determinations.

Table 1 is an extract from the scoping document showing an example of the questions that were raised alongside the evidence requirement for each of the objectives.

Based on this, data and evidence, both quantitative and qualitative, were gathered from a number of sources. It was primarily gathered by the Scottish Government review team, overseen by Internal Audit and each of the Boards. Data for the technical review was also gathered by Pennington Choices.

Data Sources

In order to gain a comprehensive understanding on how the scheme is being implemented, the Scottish Government review team conducted field research engaging with members of the Warmer Homes Scotland supply chain, customers, Home Energy Scotland, and Warmworks surveyors & inspectors in order to cover all delivery aspects of the scheme (see Figure 2).

This qualitative data was used to assess key aspects around the customer service component, community engagement and supply chain delivery model.

In addition to field research, a set of "customer outcomes" data sourcing from both Warmworks' invoices and EST's Customers Records Management tools was used in much of the analysis regarding the fuel poverty objective.

As part of this review, Scottish Government commissioned Pennington Choices to undertake a Technical Review of the scheme which included examining which measures had been taken up by customers, which hadn't, why customer applications were cancelled, which measures they think should remain or be removed and which new measures they think should be included going forward. The Technical Review considered data from September 2015 to June 2016 unlike the rest of the review which covered the period from September 2015 to end March 2016) and can be found at Annex 1.

Figure 2: Qualitative data sources

SURVEYS

The Scottish Government review team accompanied surveyors on the job to observe the quality of the interaction with customers and to assess whether the protocol followed for customer/home data gathering is in alignment with contractual agreements and Scottish Government standards.

SUBCONTRACTORS

The Scottish Government review team met with various subcontractors working on the WHS scheme to discuss their experience of delivering measures in different parts of Scotland including challenges they encountered and consider their recommendations for future improvements.

INSPECTIONS

A number of works in progress were observed by the Scottish Government review team in order to determine the overall inspection robustness and customer service.

CUSTOMER SERVICE

HES customer advisors agreed to answer a survey detailing aspects of their experience in managing customers over the phone and made a series of recommendations that would optimise the impact of the scheme.