Procurement Efficiencies: Monitoring and Evaluation of Devanha Phase 2

7 Process, Collaboration and Value for Money

Devanha Works Cost Target Setting

7.1 It is debatable whether the mechanism for calculating the Devanha Offer of Grant works costs target at scheme level was realistic, given the complexity and variability that existed across schemes. The Devanha target accounted for house type and was weighted accordingly for each scheme but did not account for the range of technical and design risks that existed and were subsequently accounted for in the adjusted target. These typically related to groundworks, utilities and infrastructure for which the contractor is compensated and the client carries the risk. This raises the more fundamental question of whether NEC form of contract where the client does bear much of the design risk is appropriate for social housing procurement programmes. A further significant design risk related to special needs adaptations which were not accounted for in the original design or cost targets, and again the client bore the cost and time risk resulting from redesign and rework on site. It is also open to question as whether the ambitious Devanha Offer of Grant works cost targets actually incentivized procurement efficiency and savings. Although there was a level of awareness of the Offer of Grant targets, the adjusted targets were considered the 'real' targets that contractors worked toward.

Devanha Process and Influence on Cost Savings and Procurement Efficiency

7.2 In addition to the monitoring of cost and grant targets, the evaluation of Devanha involved qualitative data collection through the Core Group focus group workshops and confidential semi structured one-to-one interviews. Both were valuable in understanding experiences of the Devanha participants and gaining an insight into how the procurement process worked from the various developer RSL, consultant and contractor perspectives. In particular, the interviews at the end of the programme provided a reflection on overall value for money and the influence that the Devanha process had on costs and value.

7.3 One of the aims of bulk procurement programme was to secure incremental improvements in efficiency gains/cost savings year on year from the Devanha framework approach. However this has not been borne out by the evidence in that there is no pattern of increased savings from tranche to tranche. A strong impression gained from the interviews across Devanha participants was that there was a large element of chance as to whether targets were met and performance was good or bad, and this resulted from the site and other technical characteristics. As such, the influence of the Devanha process on VFM and cost savings, at scheme level, is difficult to evaluate numerically.

VFM, Competition and the Devanha Process

7.4 The competitiveness of the Devanha process is a key determinant of VFM. The difficulty of any framework agreement is that, in securing the benefits of a long term framework of closer collaboration and supply chain integration, the price advantages of traditional lowest cost competitive tendering are not fully realised. Although the open book costing process ensured open market prices from subcontractors and suppliers were reflected in adjusted target costs, the level of price competition was arguably not as high as would have been secured through traditional tendering.

7.5 The approach in Devanha was to allocate all the workload to framework contractors from the outset. It may have been advisable to introduce a staged allocation of schemes which could have reviewed contractor performance mid way through the programme for the purpose of further allocation of work. This would provide an additional mechanism to ensure competition between framework contractors is maintained throughout a bulk procurement programme.

Value for Money and the views of Devanha

7.6 The ultimate question is whether Devanha bulk procurement offered VFM when compared to traditional procurement approach. Responses from across the range of participants varied from possibly to yes. No one was of the opinion that Devanha did not offer value for money compared to a traditional procurement approach with contractors more positive in response to this question. They readily pointed to the open book costing process which meant that RSLs did share in the benefits of keener material and supplier prices as the market cooled. Of course RSLs would also have benefited from market conditions through traditional open tendering procurement. RSLs were more equivocal/less convinced that they were able to benefit from increased price competition in the open market as the programme progressed. A key issue related to timing, in that economic and market conditions at the end of Devanha were completely different to those at the start and could not reasonably have been foreseen. As such what could bluntly be termed a 'sellers' market became a 'buyers' market - conditions where traditional competitive tendering in the open market provides the keenest prices. However, it should be borne in mind that VFM extends beyond keenest price and lowest capital cost.

Implementation and Operation of the NEC Approach.

7.7 The benefits and limitations of the Devanha bulk procurement process have been considered and reported on at each round of the research. The Core group asserted there was a higher level of collaboration than experienced through traditional procurement and this manifested in better communication, trust, open book costing and sharing of buildability experience. The January 2012 closeout interviews largely supported this, though there was a view expressed that not all framework contractors fully bought into the collaborative approach.

7.8 The lack of understanding and experience of the NEC procurement process that is fundamental to operational aspects of Devanha procurement were clear from early on in the process, despite the engagement of a procurement consultant from the outset. In Round 2 we reported on the difficulties related to the operational aspects of implementing the NEC Devanha approach. Poor preparation for the Devanha approach was also widely observed at the close out interviews. There was a strong and consistent view that the procurement consultant workshops and training failed to prepare the team adequately and the lack of time upfront for development work was widely considered a problem.

7.9 The importance of extending training and awareness of the Devanha process and associated KPIs to site level personnel was not fully realised until late on in Devanha. Only when the influence site personnel had on meeting KPIs, and therefore influencing gain share became clear was this fully realised as schemes were completed and signed off. In particular, the defects KPI had great importance in this respect. Communication of Devanha targets throughout the organisation was slow. Throughout the duration of Devanha, the Offer of Grant targets themselves were not generally known at Core Group level.

7.10 For the most part, partnering did not extend beyond the first tier contractors. This wasn't from a lack of willingness or conscious inaction. Rather, the small size and fragmented nature of subcontract activity meant that real benefits from this hadn't been identified. The lack of a true Devanha specification, uniformly applied over all projects from the start, meant that the opportunity for bulk buying and strategic involvement of subcontractors was severely limited.

Open Book Costing

7.11 Mid way through Devanha the impression was of contractors undergoing a cultural change in coming to terms with open book costing. By the end of Devanha, Chap and Bancon were fully engaged in open book costing and cited this as means of passing savings from keener price competition on to RSLs when the question of VFM was raised. The open book costing process is a contractual feature of NEC and the contractors have been compliant with this. As indicated in the report, Robertsons did not respond to any invitation to participate in interviews to explore their approach and experiences. It is the case that savings are passed on, but the contractor shares the savings whereas the client would secure all of these under full scheme by scheme competitive tendering. Evidence of this may be seen in non-framework schemes where prices were very keen. There is undoubtedly more market testing under traditional lowest cost competitive tendering, though at the expense of collaboration and other gains in value and quality.

Devanha Offer of Grant Works Cost Targets and Scottish Government Validated Targets.

7.12 In the concluding interviews the consistent view was that the Offer of Grant targets were unrealistically low and unachievable. Originally, the targets had been proposed by Communities Scotland and accepted by Devanha RSLs as aspirational targets which were the basis for acceptance of the Offer of Grant. However, interviewees felt that there was no apparent justification for the target and there was no involvement of consultants or contractors in target setting. Through time, and with inflationary adjustments, this target was seen as increasingly divorced from the reality on the ground. Clearly, given the views expressed by interviewees, the justification for the target did not permeate effectively to the level of those running the schemes.

7.13 A single target (with weightings to adjust for different house types) was overly simplistic and didn't take account of the many site and infrastructure factors that influence costs. Although there was some level of awareness of the Devanha targets by the contractors and consultants, the Scottish Government validated targets were considered as the 'real' targets. At scheme level the ability to meet the target related to the extent of difficulties with the site or other technical issues, rather than application of any Devanha process initiatives. Also a frequently cited reason for rising costs was late design changes, in particular adaptations for special needs not considered at the design stage. The waste, inefficiency and extra cost to the RSL of such rework are obvious. Variations are an almost inevitable feature in the course of construction projects. The key question is who bears the risk for these and under the NEC approach the RSLs absorb much of the risk through compensation events.

Compensation Events and the NEC Approach

7.14 The Devanha process had little bearing on the type or extent of compensation events in schemes. There was a consistent view that these variations would have happened anyway as they were largely a consequence of technical and site specific issues such as ground conditions or changing requirements and design changes as schemes progressed.

Gain / Pain Share and KPIs

7.15 Pain has been limited and there was mostly gain as a result of favourable market conditions following the validation of targets. Schemes ostensibly performed well because they were delivered below target but this was as a result of Scottish Government validated target being set before the impact of much keener subcontractor prices was felt. Falling prices rather than contractor behaviours or Devanha process was the reason for the performance. It is therefore arguable the extent to which contractors earned the gain. Nevertheless, the Gain/Pain share mechanism allowed for this.