Procurement Efficiencies: Monitoring and Evaluation of Devanha Phase 2

6 Costs and Grant

Introduction

6.1 This chapter considers whether the Devanha programme has achieved VFM with reference to the cost and grant data that has been collected from Scottish Government in the agreed format. Where reconciled completion data is available, actual HAG and works cost at scheme completion are compared with targets. For the remainder of schemes comparisons are made between Offer of Grant and validation stage targets. Additionally, data collected through workshops and interviews of a range of key participants in the Devanha programme has been used to inform the assessment of VFM.

Final and Interim reports on Cost and Grant

6.2 In previous years our reports on progress towards Devanha Offer of Grant works cost and HAG targets and evaluation of the Devanha procurement process have been partial due to lack of actual completion data. For this final report actual cost data is still incomplete, with data available for 757 units (56% of total to be constructed) in 35 schemes from the programme. Consequently, conclusions still have to be qualified though the quantitative and qualitative data that is available allows us to evaluate VFM

Works Costs and Grant targets

6.3 A number of Cost and Grant targets apply at different stages along the timeline of each scheme. This section reports on actual works cost and HAG expenditure performance against targets. However in assessing procurement efficiency and value for money of the design and construction process, works cost target is the more meaningful measure since there are a number of factors, e.g. to do with rent setting, that influence HAG expenditure but which are unrelated to procurement process.

6.4 Target measures relating to works cost for each scheme are set out as follows:

Devanha target	The works cost target contained in the Devanha Offer of Grant. This was the target cost benchmark which, if met, would have demonstrated Value for Money. The benchmark related to a particular type of scheme with weightings used to adjust it as appropriate.
Validated target	An estimate of cost specific to each scheme based on the latest design and market information at the point of grant approval, and before the works commence.
Adjusted target	The validated target is subject to revision and adjustment to reflect design changes and client risks that occur during construction period. These adjustments relate to compensation events and are a central feature of NEC form of procurement.
Actual cost	The actual cost is the final agreed signed off cost which is only known once construction is completed.

6.5 The Devanha Target: The research was commissioned to monitor and evaluate performance of Devanha against targets set out in the Offer of Grant. Part 6 of this states that Devanha agreed to make average works cost savings of 7.3% over the programme, based on assumed savings of 5% in year 1, 7% in years 2 and 10% in year 3. The original target grant cost benchmark was expressed in the Offer of Grant as a "target grant cost benchmark for a 4 person home at April 2006 of £68,648." However the actual target that was applied to all schemes in Devanha related to a benchmark figure of £67,284 which represented target works cost for a 3 person unit at April 2006. This is the accepted basis for target recorded in grant validation forms. This benchmark is used to calculate a target for each scheme in Devanha by applying the Scottish Government bedspaces weighting factor and Scottish Government agreed inflation uplift.

6.6 The Scottish Government Validated target is that which is approved by Scottish Government at tender stage and is based on the Offer of Grant target adjusted to take account of scheme specific design and risk variables. The validation is an approval that the target cost for each scheme had been prepared in line with the agreed process on the advice of CWC. The Offer of Grant and Scottish Government validated targets are recorded on the Scheme Cost and Grant validation forms which provided data for this section.

6.7 The Adjusted target works cost is known only at scheme completion stage. As each scheme is constructed, the target is reviewed and adjusted for agreed compensation events and their associated costs. Typically these relate to site risks and design changes initiated by the client and changing market conditions which affect contractors' costs. The final adjusted target is not known until the scheme is complete and all events have been costed. The adjusted target is crucial as it dictates the amount of gain or pain that is shared by the contractor according to the agreed mechanism.

6.8 The actual cost is that which is signed off at scheme completion stage against which the Offer of Grant and Validation stage targets are compared. As the data shows, the divergence between actual cost and validated targets at scheme level can be very considerable due to the varying levels of price and design risk experienced. Validated targets became out of date quickly at a time when the market was changing rapidly and difficult site conditions and uncertain technical requirements not apparent at validation stage were reasons for validated target and actual cost variance. Whilst RSLs shared in the benefits of a keener market, they bore much of the design risk through the NEC procurement approach.

Assessing Value for Money

6.9 The monitoring component of the research brief involves tracking and comparing numerical cost and grant data made available over the life of the programme with the original targets set out in the Offer of Grant. As such, the ability of Devanha to meet these targets is transparent, objective and readily determined by a straightforward comparison of target and signed off cost and grant at completion stage of the schemes. It has become apparent however that the evaluation of whether VFM has been achieved is more than a straightforward comparison to determine whether Offer of Grant targets have been met. Many factors including market conditions, individual scheme and site characteristics and the project team's ability to influence events all affect actual cost. The target set in the original Offer of Grant and its achievability has also been an issue and is discussed later in this report.

6.10 Probably the single most significant influence on Devanha performance has been the complete transformation of economic circumstances since the Offer of Grant was signed. These were obviously outwith the control of Devanha, but there are issues about how the Devanha process accounts for changing markets and prices compared to traditional procurement discussed in this report. Specifically, the ability of the framework approach to take advantage of a market which became very competitive as Devanha progressed is arguable and there are differing views on this within the Core group. Although the open book costing process allowed cost savings to be passed to the RSLs, the full benefit of traditional competitive tendering in a very keen market were probably not realized since the benefits were shared by both client and contractor, rather than solely by the client. However, this would only take into account costs and the criteria for comprehensive VFM assessment would go beyond capital cost and grant comparisons, and would ideally take account of life cycle costs, end user satisfaction and measures of product quality. The primary benchmarks set out in the Offer of Grant against which achievement of target is measured are cost and grant.

Inflation and Market conditions

6.11 The original target was to deliver 1563 houses for a fixed amount of grant which assumed a level of works cost inflation increases over the programme as set out below. These works cost inflation increases shown in Table 6.1 were from forecasts provided by CWC using BCIS All-in Tender price index. Although the detail behind these are not available a review of BCIS data shows that these were forecasts of year-on-year tender price increases. As such the forecast only showed a slowing down of the rate of price level increase toward the latter part of Devanha, but did not forecast the fall of tender price levels actually experienced. Subsequent BCIS forecasts were revised downward as the gloomier economic outlook was becoming clearer.

Table 6.1 Offer of Grant Inflation forecast

Year 06/07 = 3.44%

Year 07/08 = 5.33%

Year 08/09 = 8.39%

Year 09/10 = 6.86%.

6.12 As events have shown, the sharply rising market before onset of recession and its impact on inflation, subcontractor and supplier prices experienced by Devanha were not forecast at all and very quickly rendered the Offer of Grant inflation assumptions out of date. A comparison of the Offer of Grant figures in Table 6.1 with the original inflation and agreed inflation figures (Table 6.2 and Figure 6.1) which were applied at scheme validation stage to set targets at the level of individual schemes illustrates this. From the outset in tranche 1 these inflation rates outstripped the Offer of Grant assumptions.

Table 6.2 Original and Agreed Inflation recorded at SG Scheme Cost and Grant Validation stage (percentages are relative to the 2006 base year)

Tranche	Original Inflation	Agreed Inflation
1	8.95%	18.72%
2	8.95%	23.20%
3	8.95%	25.44%
4	18.09%	29.20%
5	18.09%	32.96%
6	18.09%	32.96%
7	18.09%	36.93%
8	18.09%	37.98%
9	18.09%	37.98%

Figure 6.1 Original and Agreed Inflation recorded at scheme validation stage

6.13 The market conditions and level of inflation actually experienced in the North East has been a major issue for Devanha and its ability to deliver the programme. It has been a source of repeated discussion throughout and features strongly in the Core Group workshops and end of process interviews carried out with Devanha participants. Clearly, since revised works cost targets are based on agreed inflation levels prevailing at scheme validation stage, the assessment of inflation is key to setting the target and the subsequent ability to meet it. Scottish Government commissioned a separate Inflation Monitoring peer review report from HWU in 2009 which contained a number of conclusions and recommendations relating to the process for assessing agreed inflation. One of the main weaknesses highlighted was that much of the data used to inform the inflation allowance was derived from within Devanha and could not therefore reflect open market conditions.

6.14 An important factor to note is the timing of when the Devanha Offer of Grant was prepared and signed in relation to national and regional construction market conditions. At this time the market was extremely buoyant and one of the benefits of a long term framework agreement in a rising market is that it can bring predictability to pricing and the capacity to deliver the programme. However, as the market turned to recession the extent to which the RSLs could take advantage of very keen pricing in open market conditions was arguable. The open book costing process did allow material and subcontract cost savings to be passed on, but probably not to the extent of traditional competitive tendering at individual scheme level.

Figure 6.2 Unit works costs comparison between different Local Authority areas

• Figure 6.2 is based on data supplied by Scottish Government and shows the average works cost trend across 4 regions in Scotland. Market pressures on works costs are seen to be broadly similar in each region. Throughout the nineties works costs were relatively flat before increasing from 2000 as the economy grew, with a steep rate of increase from around 2006 before the onset of recession around 2008. If the North East were to have followed these trends across the Devanha period, where costs started to level off in Glasgow, Edinburgh and Tayside, then Devanha should have been able to deliver greater cost savings. However, all indications from Devanha were that the market remained stronger for longer and that the onset of recession was delayed and less severe than the national picture. The inflation reports prepared by CWC and used as the basis for agreeing inflation uplifts supported this argument and are the source of evidence for market conditions being distinctly different in the north east. Since the inflation uplifts were used in adjusting cost targets at validation stage of schemes it is clear they are instrumental in the measurement of procurement efficiency. One of the key findings of this report is that any future procurement arrangements which involve inflation monitoring in target setting should consider independent third party assessment of inflation to ensure greater objectivity in any assessment or audit of value for money.

Approach to Final Report - Monitoring and Evaluation

6.15 At the time of writing of the report, complete and signed off data is only available for 35 of the 51 schemes which accounts for 757 units. Data up to Scottish Government validation stage has been made available for all 51 schemes which does allow for a level of analysis in all schemes to inform the findings and conclusions.

Analysis of Works Cost Data

6.16 Table 6.3 summarises the works cost data discussed in this section.

All schemes to completion stage - 35 schemes.

6.17 The actual total works cost as signed off by Scottish Government is 4.22% above the Offer of Grant target. For the 35 schemes for which data is available therefore the Devanha works cost target has not been achieved. These schemes include a mix of tenure (rented and NSSE^[3]), framework and non framework and are drawn across all 9 tranches. Deviation from target on individual schemes varies considerably, though the extremes are explained by scheme and site specific features and are not directly attributable to extent of procurement efficiency. For example, the near 50% reduction on Devanha target at one extreme is explained by a substantial reduction in scope of the works between target setting and validation stage on the scheme (a one-off circumstance). Similarly, the overrun of almost 43% is not due to extremely poor management or lack of efficiency, but difficult site conditions not accounted for in the notional Devanha target.

6.18 Thus, variations from Offer of Grant target cannot be directly attributed to any management or procurement efficiency from the Devanha process since the adjusted targets for individual schemes can vary considerable from calculated Offer of Grant target. That is to say, the Offer of Grant target is a notional benchmark that doesn't take into account scheme specific factors and risks.

Tenure - NSSE v Rental (works cost)

6.19 25 of the 35 schemes are for Rental and the remaining 10 are NSSE. The actual total works cost as signed off by Scottish Government for Rental schemes is 4.36% above the Offer of Grant target. For this rental subset of schemes the Offer of Grant target has not been met.

6.20 The actual total works cost as signed off by Scottish Government for NSSE schemes is 3.60% above the Offer of Grant target. For the NSSE subset of schemes the Offer of Grant target has not been met, though performance is slightly better than Rental schemes albeit the data sample is smaller.

Framework v Non-framework (works cost)

6.21 19 of the 35 schemes are framework and the remaining 16 are non-framework. The actual total works cost as signed off by Scottish Government for framework schemes is 9.36% above the Offer of Grant target. For this subset of schemes the Offer of Grant target has not been met.

6.22 For Non Framework schemes the Offer of Grant works cost target is 2.78% below the Offer of Grant target. For the Non Framework subset of schemes the Offer of Grant target has therefore been met and the performance is better than framework schemes. Many of these schemes will have benefitted from going to the marketplace at a more favourable part of the economic cycle and the MVPI schemes were subject to a separate negotiation.

Analysis by tranche (works cost)

6.23 The average works cost saving in the Devanha target is based on averaged assumed increasing savings through procurement efficiencies over the life of Devanha programme. The data however does not convincingly support this assumption. When broken down by tranche there is no real pattern of improving performance apparent as shown in Table 6.5 but the number of schemes and units available for analysis within each tranche does vary, meaning that a true like for like comparison cannot be carried out. It is encouraging though that the scale of the cost overruns experienced in tranche 2 and tranche are not repeated later in the programme,

6.24 As the figures in the Summary Table 6.3 show, the actual costs exceed the Offer of Grant works cost target at aggregate level for the framework schemes, but are below the Scottish Government validated target. This suggests that either efficiencies were not delivered as expected or the Offer of Grant target may have been too challenging for most schemes, or a combination of both (analysis at scheme level show most framework schemes do not meet the Devanha target). Regarding the SG validated target, interviews revealed a sense that the NEC gain/pain share process provided an incentive for contractors to be cautious in maintaining validation stage targets at a high level and that the Scottish Government validated figures may therefore have been artificially high.

Analysis of HAG Data

6.25 Table 6.4 summarises the HAG data discussed in this section.

6.26 This section reports on actual HAG expenditure against target HAG expenditure. However for reasons outlined at the start of the chapter HAG performance is not a good measure of procurement efficiency. Of course, works cost are a large component of HAG, but other non-construction related factors influence HAG.

All schemes to completion stage - 35 schemes.

6.27 The total Devanha Offer of Grant HAG target for the 35 signed off schemes, comprising 757 units, decreases by -8.02% at Scottish Government validation stage. The actual HAG as signed off by Scottish Government is 9.22% below the Offer of Grant target. For the 35 schemes for which data is available therefore the Devanha HAG target has been achieved. These schemes include a mix of tenure (rented and NSSE), framework and non framework and are drawn across all 9 tranches. Variance from target on individual schemes is considerable, from almost 80% reduction through to increase of almost 47%. It is to be expected that works cost variation will cause HAG variation in individual schemes, but there are other influencing factors contributing to variation relating to calculation of grant in NSSE schemes and rent setting issues.

Tenure - NSSE v Rental (HAG)

6.28 Twenty five of the 35 schemes are for Rental and the remaining 10 are NSSE. The Offer of Grant HAG target for the rental schemes decreases by -2.53% at validation stage. The actual total HAG as signed off by Scottish Government for Rental schemes is -8.98% below the Offer of Grant target. For this subset of schemes the Offer of Grant target has been met.

6.29 For NSSE schemes the Offer of Grant HAG target decreases by -15.81% at validation stage. The actual total HAG as signed off by Scottish Government is -10.56% below the Offer of Grant target. For the NSSE subset of schemes the Offer of Grant target has been met, and performance is slightly better than Rental schemes in percentage terms although the data sample is smaller and, as mentioned above, may reflect grant calculation and rent setting considerations which are outside Devanha's control.

All schemes - Framework v Non-framework (HAG)

6.30 Nineteen of the 35 schemes are framework and the remaining 16 are non-framework. The Offer of Grant HAG target for the framework schemes increases by 0.5% at validation stage. The actual total HAG as signed off by Scottish Government is -0.33% below the Offer of Grant target. For the framework subset of schemes the Offer of Grant HAG target has been met.

6.31 For Non Framework schemes the Offer of Grant HAG target decreases by -18.34% at validation stage. The actual total HAG as signed off by Scottish Government is -20% on the Offer of Grant target. For the Non Framework subset of schemes the Offer of Grant target has therefore been met and the performance is better than framework schemes. Many of these schemes will have benefited from going to the marketplace at a more favourable part of the economic cycle and the MVPI schemes were subject to a separate negotiation.

Analysis by tranche (HAG)

6.32 When broken down by tranche there is no pattern of improving HAG performance apparent as shown in Table 6.6. However the number of schemes and units for analysis within each tranche does vary considerably, particularly since there are a number of schemes in tranches 5 to 8 which have yet to be signed-off. The sample is smaller meaning that a true like for like comparison is not carried out.

6.33 It is apparent from the data that there are large variations between HAG targets, both at Offer of Grant and validation stage, and actual Grant expenditure at scheme level. In particular the variation exhibited between actual grant and validation stage grant is greater than for the same works cost comparison. These large variations at scheme level result from a number of factors unrelated to procurement efficiency and the Devanha process. Features affecting grant are not the same as for cost, e.g. rent setting and timing of rent setting.

TABLE 6.3

SUMMARY analysis comparing targets and actual Works Cost : COMPLETED & SIGNED-OFF SCHEMES

WORKS COST
	All	NSSE	Rental	Framework	Non Framework
schemes	35	10	25	19	16
units	757	163	594	399	358
D target	65,855,039	12,594,006	53,261,033	37,935,712	27,919,327
validated	69,503,884	13,906,845	55,597,039	42,462,540	27,041,344
actual	68,630,979	13,047,954	55,583,025	41,486,734	27,144,245
Validated on D target	5.54%	10.42%	4.39%	11.93%	-3.14%
Actual on SG validated	-1.26%	-6.18%	-0.03%	-2.30%	0.38%
Actual on D target	4.22%	3.60%	4.36%	9.36%	-2.78%

TABLE 6.4

SUMMARY analysis comparing targets and actual HAG : COMPLETED & SIGNED-OFF SCHEMES

HAG
	All	NSSE	Rental	Framework	Non Framework
schemes	35	10	25	19	16
units	757	163	594	399	358
D target	58,301,343	8,910,628	49,390,715	31,942,884	26,358,459
validated	53,625,949	7,501,695	46,124,254	32,101,963	21,523,986
actual	52,925,348	7,969,552	44,955,796	31,839,047	21,086,301
Validated on D target	-8.02%	-15.81%	-6.61%	0.50%	-18.34%
Actual on SG validated	-1.31%	6.24%	-2.53%	-0.82%	-2.03%
Actual on D target	-9.22%	-10.56%	-8.98%	-0.33%	-20.00%

TABLE 6.5

ANALYSIS by TRANCHE, WORKS COST - : COMPLETED & SIGNED-OFF SCHEMES

Works cost
	Tranche 1	Tranche 2	Tranche 3	Tranche 4	Tranche 5	Tranche 6	Tranche 7	Tranche 8	Tranche 9
Number of schemes	13	1	3	7	2	3	1	3	2
Number of units	288	27	53	141	58	44	22	116	8
Devanha target	21,270,977	2,434,930	4,434,682	13,699,880	5,869,727	4,488,710	2,341,074	10,513,492	801,567
SG validated target	22,953,255	2,884,424	5,469,185	14,659,120	6,258,246	4,991,282	2,411,037	9,073,598	803,737
Actual	23,158,590	2,987,029	5,858,323	14,725,281	5,184,321	4,499,980	2,225,745	9,115,602	876,108
Validated on D target	7.91%	18.46%	23.33%	7.00%	6.62%	11.20%	2.99%	-13.70%	0.27%
Actual on SG validated	0.89%	3.56%	7.12%	0.45%	-17.16%	-9.84%	-7.69%	0.46%	9.00%
Actual on D target	8.87%	22.67%	32.10%	7.48%	-11.68%	0.25%	-4.93%	-13.30%	9.30%

TABLE 6.6

ANALYSIS by TRANCHE, HAG - : COMPLETED & SIGNED-OFF SCHEMES

HAG
	Tranche 1	Tranche 2	Tranche 3	Tranche 4	Tranche 5	Tranche 6	Tranche 7	Tranche 8	Tranche 9
Number of schemes	13	1	3	7	2	3	1	3	2
Number of units	288	27	53	141	58	44	22	116	8
Devanha target	18,612,194	2,185,055	4,164,860	11,531,877	5,091,500	3,847,643	1,929,282	10,361,646	577,286
SG validated target	16,366,177	2,618,313	4,863,618	11,191,675	4,543,908	3,948,912	1,348,563	8,179,540	565,243
Actual	16,041,767	2,574,824	4,812,921	11,689,429	4,174,552	3,830,629	1,233,487	7,879,644	688,095
Validated on D target	-12.07%	19.83%	16.78%	-2.95%	-10.76%	2.63%	-30.10%	-21.06%	-2.09%
Actual on SG validated	-1.98%	-1.66%	-1.04%	4.45%	-8.13%	-3.00%	-8.53%	-3.67%	21.73%
Actual on D target	-13.81%	17.84%	15.56%	1.37%	-18.01%	-0.44%	-36.06%	-23.95%	19.19%

Works Cost and HAG analysis for ALL schemes to validation stage

All schemes to Validation stage - 51 schemes. (Works cost, Table 6.7)

6.34 The total Devanha Offer of Grant works cost target for the 51 signed off schemes, comprising 1383 units, amounts to £131,964,986 adjusted for agreed inflation. At Scottish Government validation stage this rises to 138,785,681 - an increase of 5.17% on the Offer of Grant target. On this basis, unless savings are made and the signed off actual costs are reduced the Offer of Grant target will not be achieved. These schemes include a mix of tenure (rented and NSSE), framework and non framework and are drawn across all 9 tranches.

Tenure- NSSE v Rental (works cost)

6.35 35 of the 51 schemes are for Rental and the remaining 16 are NSSE. The Offer of Grant works cost target for the rental schemes equates to £104,854,314. At Scottish Government validation stage this rises £109,899,020- an increase of 4.81%. For this subset of schemes, and based only on a comparison between Offer of Grant and validation stage, the Offer of Grant target would not be met.

6.36 For the 16 NSSE schemes the Offer of Grant works cost target equates to £27,110,672. At Scottish Government validation stage this rises to £28,886,661 - an increase of 6.55%. For the NSSE subset of schemes, and based only on a comparison between Offer of Grant and validation stage, the Offer of Grant target would not be met.

Framework v Non-framework (works cost)

6.37 32 of the 51 schemes are framework and the remaining 19 are non-framework. The Offer of Grant works cost target for the framework schemes equates to £96,089,793. At Scottish Government validation stage this rises to £104,442,389 - an increase of 8.69%. For this subset of schemes, and based only on a comparison between Offer of Grant and validation stage, the Offer of Grant target would not be met.

6.38 For Non Framework schemes the Offer of Grant works cost target equates to £35,875,193. At Scottish Government validation stage this reduces to £34,343,292- a decrease of 4.27%. For the Non Framework subset of schemes, and based only on a comparison between Offer of Grant and validation stage, the Offer of Grant target would be met. Thus the performance is better than framework schemes.

All schemes to Validation stage - 51 schemes. (HAG, Table 6.8)

6.39 The total Devanha Offer of Grant HAG target for the 51 signed off schemes, comprising 1386 units, amounts to £120,864,158 adjusted for agreed inflation. At Scottish Government validation stage this reduces to £113,128,516 - a decrease of 6.40% on the Offer of Grant target. On this basis the Offer of Grant target will be achieved. These schemes include a mix of tenure (rented and NSSE), framework and non framework and are drawn across all 9 tranches.

Tenure- NSSE v Rental (HAG)

6.40 35 of the 51 schemes are for Rental and the remaining 16 are NSSE. The Offer of Grant HAG target for the rental schemes equates to £100,436,736. At Scottish Government validation stage this reduces to £95,662,554- a decrease of -4.75%. For this subset of schemes, and based only on a comparison between Offer of Grant and validation stage, the Offer of Grant target would be met.

6.41 For the 16 NSSE schemes the Offer of Grant works cost target equates to £20,427,422. At Scottish Government validation stage this reduces to £17,465,962- a decrease of -14.50%. For the NSSE subset of schemes, and based only on a comparison between Offer of Grant and validation stage, the Offer of Grant target would be met.

Framework v Non-framework (HAG)

6.42 32 of the 51 schemes are framework and the remaining 19 are non-framework. The Offer of Grant works cost target for the framework schemes equates to £86,966,044. At Scottish Government validation stage this reduces to £85,407,374- a decrease of -1.79%. For this subset of schemes, and based only on a comparison between Offer of Grant and validation stage, the Offer of Grant target would be met.

6.43 For Non Framework schemes the Offer of Grant works cost target equates to £33,898,114. At Scottish Government validation stage this reduces to £27,721,142- a decrease of -18.22%. For the Non Framework subset of schemes, and based only on a comparison between Offer of Grant and validation stage, the Offer of Grant target would be met.

TABLE 6.7

SUMMARY analysis comparing Devanha target and Validated Works Cost : ALL Schemes

	Works Cost
	All	NSSE	Rental	Framework	Non Framework
schemes	51	16	35	32	19
Units	1386	311	1075	990	396
D target	131,964,986	27,110,672	104,854,314	96,089,793	35,875,193
Validated	138,785,681	28,886,661	109,899,020	104,442,389	34,343,292
Validated on D target	5.17%	6.55%	4.81%	8.69%	-4.27%

TABLE 6.8

SUMMARY analysis comparing Devanha target and Validated HAG : ALL Schemes

	HAG
	All	NSSE	Rental	Framework	Non Framework
schemes	51	16	35	32	19
units	1386	311	1075	990	396
D target	120,864,158	20,427,422	100,436,736	86,966,044	33,898,114
validated	113,128,516	17,465,962	95,662,554	85,407,374	27,721,142
Validated on D target	-6.40%	-14.50%	-4.75%	-1.79%	-18.22%

Trend in performance - Validation stage

6.44 Validation stage data is available for all 51 schemes of the programme. Table 6.9 below shows that for those signed off schemes where actual costs have been reconciled, the Devanha works cost target overrun at validation stage was only slightly less than for those schemes which remain to be signed-off (i.e. where actual cost and grant data not available at the time of writing). This suggests that if actual data at sign-off stage were available for all schemes, the pattern would be similar to that for the 35 schemes where completion data is available i.e. the actual cost would be slightly below validation stage works cost target, but slightly above Offer of Grant target. It can reasonably be concluded that the data on which this report is based is representative of the Devanha programme as a whole, and that the basis for the conclusions and recommendations being presented are robust. Extrapolating trends for target and actual HAG across the whole programme are even more problematic due to the number of variables affecting HAG.

Table 6.9 Trend between Validated and signed-off

	SG validated against Devanha target
	Works Cost	HAG
All schemes (51)	5.17%	-6.40%
Signed-off schemes (35)	4.22%	-8.02%
Schemes not signed-off (16)	4.80%	-4.89%

Evaluation of Value for Money - Works cost target

6.45 Of the 35 completed and signed-off schemes only 12 have actually met the Devanha works costs target, of which 8 are non framework. Therefore only 4 of the 35 framework schemes (11%) have actually met works cost target (Table 6.10). However, it would be an oversimplification to conclude that this in itself is evidence of exceptional performance or procurement efficiency for these four schemes. Subsequent revisions to targets have been carried out and agreed between contractor and project manager to account for particular scheme and site characteristics and changing market conditions. A clear example from the table below is Denmark St Fraserburgh NSSE, which shows a reduction on Offer of Grant target, but was actually one of the few schemes subject to pain share by the contractor since the actual cost exceeded the final adjusted target.

Table 6.10 - Framework schemes achieving Works Cost target

Scheme	Actual cost on Devanha target	Actual cost on Validated target	Adjusted target met (Yes/No)
Deveron Rd, Huntly NSSE	-30.03%	-34.61%	Yes
Denmark St, Fraserburgh NSSE	-14.04%	-27.11%	No
Deveron Rd, Huntly R	-7.11%	-12.80%	Yes
Hillside, Portlethen (Drum)	-4.93%	-7.69%	Yes

Gain and Pain share on Scheme Completion costs

6.46 Of the 35 completed and signed off schemes, 16 of the schemes show gain share and 5 show pain share as a result of being above or below adjusted target. The remaining 14 delivered to target. On this evidence, with only 14% of schemes (5 out of 35) in a 'pain' situation (Table 6.11) at sign off then the majority of schemes have performed to adjusted target or better at final sign off stage. Comparing this to Devanha Offer of Grant target where 66% of schemes (23 out of 35) are in excess of this it is clear that the targets do not reconcile, which serves to confuse the overall VFM assessment. On the one hand there is the Devanha Offer of Grant works cost target for each scheme and on the other hand there is the adjusted target which is only known when the risks become apparent and all compensation events are accounted for. The Devanha Offer of Grant target is a notional benchmark and the adjusted target is a reflection of actual site and scheme specific conditions. If the Devanha Offer of Grant target is being overshot in most cases then arguably the target is unrealistically low and unachievable even with an efficient procurement process. Of course the counter to this is that targets were deliberately challenging and that Devanha programme should have delivered.

6.47 The actual cost of 3 of the schemes were clearly below both the Offer of Grant and Scottish Government validated targets, but the fact that a scheme incurring pain share is apparently below these targets (Denmark St, Fraserburgh NSSE) demonstrates that procurement efficiency and good practice cannot be concluded from the Scottish Government target figures.

Table 6.11 Schemes with Pain share (Framework and Non Framework)

Tomintoul (NSSE) Denmark St, Fraserburgh NSSE Distillery Site, Oldmeldrum NSSE Distillery Site, Oldmeldrum Rent Tomintoul (Rent)

Non-Framework Framework Framework Framework Non-Framework

Conclusions

6.48 Performance by disaggregated elements of the programme: rented/shared equity, framework/non-framework in relation to HAG and cost targets are set out in the conclusions in Chapter 13. In summary, the only element of the programme to meet the Offer of Grant works cost target was the non-framework schemes. The programme as a whole and all of the above mentioned sub-sets met the target Offer of Grant HAG target. However HAG performance is not a reliable indicator of construction or procurement efficiency due to the other factors which influence its final calculation.

6.49 The expectation was that the Devanha process would provide for incremental improvement in works cost performance. However, the evidence for this was very weak when performance is analysed by tranche and it is concluded that continuous improvement in procurement efficiency was not demonstrated by Devanha over the duration of the programme. The number of schemes and units within each tranche does vary considerably though, meaning that a true like for like comparison cannot be carried out.