Procurement Efficiencies: Monitoring and Evaluation of Devanha Phase 2 - Research Findings

Housing, Refeneration and Planning Procurement Efficiencies: Monitoring and Evaluation of Devanha Phase 2

Main Findings

• The Devanha programme failed to meet the works cost target as set out in the Offer of Grant. On this measure, value for money has not been achieved. However, the basis of the original target is not clear and therefore it cannot be concluded how realistic it actually is.
• Although incentivisation mechanisms for performance improvement were part of the procurement process, these did not relate to the Offer of Grant works cost targets. As such, the most important target was not used to drive performance improvement and procurement efficiency. This undermined the importance and awareness of the Offer of Grant target.
• The Devanha programme met its HAG target as set out in the Offer of Grant. This is due to numerous and complex rent setting and HAG administration factors and is not attributable to the Devanha procurement process.
• The long term nature of the Devanha framework had the advantage of bringing price stability and predictability at a time of rising market conditions when the agreement was signed. However, as market conditions transformed at the later stages, traditional tendering was seen to provide more competitive pricing and better value for money. These conditions could not have been reasonably foreseen at the time of signing the agreement.
• There was no experience of the NEC form of contract used by Devanha within the project team at the outset. This contributed to the operational and implementation difficulties throughout most of the duration of the programme and served to undermine progress toward achieving value for money. The use of a procurement consultant was largely ineffective in mentoring Devanha on NEC.
• Devanha introduced a new procurement approach not previously experienced in the North East region. Innovative practices including greater collaboration, communication, open book costing and sharing of constructability experience were introduced into the procurement process. Devanha has developed procurement skills and experience that were previously absent in the region.
• Devanha lacked leadership; partly because it did not appoint a managing director and partly because of tensions between the Devanha Board and the individual RSLs. Devanha members recognised these problems would have to be addressed in any future collaboration.
• Although RSLs considered risk issues prior to involvement in Devanha, it has become clear that not all potential risks relating to financial liabilities were fully considered.
• Devanha suffered from a weak external profile and was a confusing organisation for local authorities to work with. Local authorities often felt that they were dealing with individual RSLs rather than Devanha.
• Devanha succeeded in building to a high quality but did not achieve significant savings through bulk procurement.

Background

Devanha Phase 2 was a groundbreaking approach in Scotland, putting in place new structures and new methods of working. It built upon earlier experience of Devanha Phase 1, a procurement initiative developed under Communities Scotland's 'Building a Better Deal' (BABD) agenda, which was a response to the Egan and Latham reports. Through greater customer focus, improving procurement, shortening the supply chain and using people and resources efficiently, Communities Scotland encouraged RSLs to become "best practice clients" with what the Egan Report (1998) described as a change of style, culture and process, rather than just a series of mechanistic activities. This collaborative approach underpinned the objectives for Devanha Phase 2.

The Devanha programme consisted of 51 projects with a mix of rental and shared equity units including projects which were delivered through a framework agreement with three contractors: Bancon, Chap, and Robertsons and through section 75 agreements with a variety of other developers. The programme took place in three local authority areas: Aberdeen City, Aberdeenshire and Moray. Most of the units in Moray were delivered by a non-framework contractor as part of the Moray Volume Procurement Initiative.

An important context for Devanha's performance is that it was conceived in very different economic circumstances than those which pertained once the programme was up and running. It was anticipated that the Devanha framework would provide price stability and predictability at a time of rising market conditions. As market conditions worsened dramatically in the later stages, traditional tendering was seen to provide more competitive pricing.

The Research

The research involved a four year monitoring exercise during which data was collected annually. The research objectives were to:

• Establish and implement a monitoring and evaluation framework,
• Evaluate and report on progress and impact,
• Set out key findings and learning points to aid sharing best practice,
• Provide evidence on transferable procurement practices,
• Provide evidence of outcomes against Devanha objectives,
• Provide evidence of whether those involved have appropriate policies, guidance and systems to manage change in procurement practice.

Key Lessons Learned

In addition to monitoring Devanha's performance, the research aimed to analyse transferable lessons which could be learned from Devanha's experience. The remainder of these findings set out key lessons which can be learnt from Devanha.

Value for Money

The most important cost target against which VFM is measured at a programme level should be directly linked to any gain share/pain share mechanism at the operational level. Devanha did not link the Offer of Grant target to driving procurement efficiency at scheme level. Additionally, a greater involvement of supply chain partners in establishing cost targets would be necessary to improve buy-in and improve their credibility.

Future procurement arrangements should consider a number of VFM metrics in addition to capital cost. Important value criteria including design quality, sustainability and life cycle costing should also be considered. Whilst these were all issues for Devanha they were not included in the Offer of Grant.

Any future framework arrangement should consider staged allocation of workload to supply chain partners over the duration of the programme based on their performance. Devanha's approach was to allocate the entire workload at the outset which limited the ability to review performance and competitiveness mid-programme. The changing market conditions experienced by Devanha underline the importance of such flexibility.

Procurement and Collaboration

Future procurement arrangements which involve inflation monitoring in target setting should consider independent third party assessment of inflation. This will ensure greater objectivity in any assessment or auditing of value for money of the procurement vehicle.

The experience and readiness of project teams to implement novel forms of contract should be carefully considered. The apparent advantages of NEC and other modern forms of contract will be very difficult to realise if there is little operational capability.

A thorough assessment of alternative procurement routes and their management of risk should be considered. Significant cost risks were retained by Devanha under NEC3 leading to cost escalation of schemes.

There should be more effort in establishing an agreed specification at the outset of any bulk procurement initiative in order that cost savings from bulk buying potential can be exploited. Opportunities were lost in Devanha to achieve such economies of scale.

The remit and terms of engagement for any procurement consultant to be used in the future should be carefully considered.

Efforts should be made to expand collaborative working beyond main contractor and involve subcontractors and suppliers in any future collaborative arrangement. This would help facilitate a common specification and bulk buying potential.

Governance

Any future collaborative programme should have a leader - Managing Director or equivalent post - in place from the start of the initiative. The MD should be supported by a development team reporting directly to her/him which may be seconded from partner RSLs but clearly badged as representing the collaborative organisation.

The Board composition should be reviewed and a simpler structure, with more independent scrutiny, developed. The MD should report directly to, and support, the Board.

There should be a clear demarcation of responsibility and decision making between the Board and partner RSLs, set out in a service level agreement. Once a project has been included within its programme, the collaborative organisation should have full responsibility for it until the completed scheme is handed over to the RSL.

Performance monitoring systems must be established before work commences to ensure an appropriate flow of management information to the MD and the Board.

RSLs should carry out a thorough and comprehensive risk analysis before entering into a collaborative development programme.

Local Authority Engagement

There is a need for more liaison with the local authority from the beginning and during the programme. Recognition should be given to the local authority's role as the strategic housing authority with responsibility for drafting the Strategic Local Programme in agreement with Scottish Government.

Devanha suffered from a weak external profile and was a confusing organisation for local authorities. The changes to governance recommended above, in particular the appointment of an MD and the badging of development staff as working for the collaborative organisation, should address this problem. A collaborative organisation should also monitor the quality of its relationship with the local authority as a key strategic partner.

Provided that a collaborative organisation has appropriate governance and a strong relationship with the local authority, it should be in a strong position to pursue collaborative working with a local authority. Its advantage lies in its likelihood to have capacity to deliver a large programme and in its members' widespread housing management resources and existing units across the area.

Quality and Standards

Projects should be brought into a collaborative programme at as early a stage in the design process as possible.

There should be greater emphasis on achieving the benefits of a standard specification throughout the programme. Early adoption of standard layouts, specifications and components would enable greater economies of scale and improved quality through repetition.

The standard specification should be sensitive to local needs but should prevent unnecessary variation. Local variation, particularly in external appearance, should be possible, to account for local building traditions but variation based on custom and practice amongst partners should not be allowed. Improvements to the specification based on experience of what works best should be applied consistently to the standard specification in all developments.

Collaboration should be used to facilitate innovative approaches. Devanha demonstrated that following a key theme, in their case energy efficiency, can be a driver for quality improvement.

Training and Employment

Training and employment benefits can be achieved even in a challenging economic climate.

Achieving maximum benefit requires the allocation of resources, for example, to reach more difficult groups, to engage with section 75 developers, or to include subcontractors and suppliers.

Conclusion

Devanha Phase 2 was a bulk procurement initiative which aimed to produce cost and grant efficiencies while delivering high quality development. It involved a collaborative approach to development. Through the use of a framework agreement and NEC form of contract, Devanha aimed to demonstrate a change of style, culture and process which would lead to greater efficiency.

Devanha did not meet its works cost targets as set out in the Offer of Grant. It had greater success as a vehicle for building to a high quality although success was limited by the lack of full implementation of a standard specification. The considerable change in the economic climate adversely affected Devanha's ability to achieve its targets and highlighted the importance of independent inflation monitoring in future programmes.

Through trialling an innovative approach, Devanha shed light on a variety of key lessons for the efficient delivery of an affordable housing programme. Perhaps the most important of these relate to the key role of leadership and governance structure of a collaborative organisation. The readiness of project teams to implement novel forms of contract needs to be considered if the benefits of these approaches are to be realised. Effective and timely data collection and monitoring systems for management and incentivisation were shown to be important. Devanha also highlighted the need for fuller implementation of a standard specification at an early stage to improve quality and take advantage of any bulk procurement opportunities.