Analysis of the impact and value of community benefit clauses in procurement

5. Conclusions and Recommendations

Conclusions

1. Through the e-survey of procurement organisations across Scotland and the in-depth analysis of 24 contracts, the research has found that CB clauses are increasingly being used in public sector contracts across Scotland.

2. The evidence suggests that there remains scope to continue building awareness and understanding of CB clauses, particularly around the use of CB clauses in service contracts.

3. In terms of the impact of CB clauses, analysis of the 24 contracts shows that the targets around job opportunities, apprenticeships, work placements and training for priority groups have been exceeded.

4. Capturing the additionality and sustainability of the CB outcomes is harder to calculate given that procuring organisations have not typically required their contractors to monitor the sustainability and additionality of CB outcomes. Best estimates suggest that although apprenticeships and work placements have the greatest additionality, employment sustainability levels are 75% for the priority groups recruited through CB clauses.

5. Across the contracts evaluated in depth, there is a big variation in the numbers recruited from priority groups for each £ of contract value. Bringing all CB contracts up to the standard of the better performing ones would significantly increase their overall impact.

6. The research findings strongly point towards the need for a more comprehensive evidence base to be developed around the longer-term impact of CB clauses. This requires a more systematic monitoring of CB clauses and their impacts in future contracts so that the use of CB clauses can be fully justified. It is in this context that the recommendations below focus on how the monitoring and evaluation of CB clauses in public sector procurement can be improved upon.

Monitoring and Evaluation Framework Recommendations

There are four parts to the monitoring and evaluation framework recommendations. The first two parts - Monitoring Information and Outcomes Information - relate to the indicators that should be captured. The third part provides guidance on the Monitoring System needed to collect the monitoring information. The final part considers the Reporting Arrangements to ensure the CB data collected is used and acted upon.

In relation to the first two parts, the indicators suggested have been developed to allow procuring organisations to understand the scale and nature of CBs they are achieving and how this is changing over time. By using a consistent set of indicators across public sector organisations, this will allow the information to be reported to the Scottish Government on an annual basis and for an overall picture of the contribution of public procurement to the National Outcomes to emerge. A number of indicators have been included that do not specifically relate to CBs but will enable CBs to be benchmarked against total procurement activity.

Monitoring Information

For each individual contract that uses CB clauses, we recommend that four different types of CB activity indicators are collected for monitoring purposes. The four types of indicators, which are set out in Figure 5.1 and explained in turn below, are:

• Key Contract Information.
• Community Benefits Clause Indicators.
• Short-Term Sustainability Indicators.
• Additionality Indicators.

Key Contract Information

The first set of indicators focus on recording the total value of the contract, the type of organisation that is awarded the contract (i.e. the main contractor), and the types of organisations that make up the contract supply chain. By including Indicator 2, procuring organisations should recognise that SMEs and social enterprises can be main contractors in their own right, and not just form part of the supply chain (as captured by Indicators 4 and 5).

Community Benefit Clause Indicators

We recommend that eight indicators are collected that in combination will measure the different types of training and employment benefits secured. These relate to job opportunities, apprenticeships, work placements, and qualifications achieved through training.

The recommended indicators are intended to provide both consistency across contracts and some flexibility to reflect local needs. For example, in relation to the training and employment indicators, we recommend using standardised 'priority group' categories from which procuring organisations can select their CB focus. The 'priority group' categories could be:

• Protected characteristics under the Equalities Act 2010.
• Employment status - e.g. unemployed, school leavers.
• Other priority groups as set out in the organisation's Outcome Agreement^[13] with the Scottish Government.

In relation to work placements for priority groups (Indicator 11), there is a need for greater consistency in how these are defined by procurement organisations. We recommend that work placements refer to:

• Placements targeted at individuals post-school, i.e. individuals in higher or further education, or in the labour market but not in employment.
• Placements that last for a minimum of 60 hours^[14].

Short-Term Sustainability Indicators

Monitoring the sustainability of the CBs achieved is the most difficult and resource intensive aspect of the monitoring system - particularly if seeking to monitor long-term sustainability (which is discussed later in this chapter). With this in mind, we recommend that the initial focus is placed on measuring the short-term sustainability of CBs, concentrating specifically on job opportunities, apprenticeships and work placements. For jobs and apprenticeships, sustainability has been defined as employment at 26 weeks after the job/apprenticeship start. The rationale for this is that:

• 26 weeks is a significant period of work that allows individuals to develop skills and demonstrate their commitment to work, thus improving their overall employability.
• Most contracts are still ongoing at 26 weeks enabling contractors to provide the majority of this data as part of the contract monitoring arrangements. Contractors should also be required to record the destinations of leavers as far as they know them.

It is vital that a consistent definition is used for 'employed at 26 weeks'. In relation to these two indicators (Indicators 14 and 15), this is defined as employment at 26 weeks after the individual has started the job or apprenticeship and can relate to employment with the contractor, sub-contractor or with a subsequent employer. The point is that sustainability relates to an individual entering and remaining in employment, rather than remaining in a specific job.

Additionality Indicators

To monitor additionality, we recommend focusing on whether or not the job opportunities, apprenticeships, work placements, and qualifications for priority groups, along with the sub-contracting opportunities to SMEs and social enterprises, would have happened anyway if the CB clauses had not been in place. The rationale for this is purely to have a simple and easy to use system which does not place excessive burdens upon procuring organisations or contractors. Additionality can be either or both:

• Additional opportunities (e.g. apprenticeship places, work experience placements, etc.) that would not have existed had the CB clause not been in place.
• Opportunities that would have existed if the CB clause had been in place, but that have been targeted on priority groups as a result of the CB clause.

Figure 5.1: Contract Monitoring Indicators (CMI)

KEY CONTRACT INFORMATION
CMI1. Total Value of Contract
CMI2. Main Contractor by Type of Organisation - SME; Social Enterprise; Other
CMI3. Total Value of Works Sub-Contracted
CMI4. Total Value of Works Sub-Contracted to SMEs
CMI5. Total Value of Works Sub-Contracted to Social Enterprises
COMMUNITY BENEFIT CLAUSE INDICATORS
CMI6. Total Number Recruited to Deliver Contract
CMI7. Number Recruited from Priority Groups
CMI8. Total Number of Apprentices Recruited to Deliver Contract
CMI9. Number of Apprentices Filled by Priority Groups
CMI10. Number of Work Placements for School Pupils, College and University Students
CMI11. Number of Work Placements for Priority Groups
CMI12. Number of Qualifications Achieved Through Training by Priority Groups
CMI13. Number of Qualifications Achieved Through Training by Other Employees
SHORT-TERM SUSTAINABILITY INDICATORS
CMI14. Number of Recruits from Priority Groups Employed at 26 Weeks After Job Start
CMI15. Number of Apprenticeships from Priority Groups Employed at 26 Weeks After Apprenticeship Start
CMI16. Number of Work Placements for Priority Groups Subsequently Recruited by Contractor/Sub-Contractor
ADDITIONALITY INDICATORS
CMI17. Number of Recruits from Priority Groups that would have been taken on if the CB Clause had not been in place
CMI18. Number of Apprentices from Priority Groups that would have been taken on if the CB Clause had not been in place
CMI19. Number of Work Placements for School Pupils, College and University Students that would have been offered if the CB Clause had not been in place
CMI20. Number of Work Placements for Priority Groups that would have been offered if the CB Clause had not been in place
CMI21. Number of Qualifications Achieved Through Training by Priority Groups that would have been offered if the CB Clause had not been in place
CMI22. Number of Qualifications Achieved Through Training by Other Employees that would have been offered if the CB Clause had not been in place
CMI23. Total Value of Works Sub-Contracted to SMEs that would have been awarded if the CB Clause had not been in place
CMI24. Total Value of Works Sub-Contracted to Social Enterprises that would have been awarded if the CB Clause had not been in place

In terms of who collects the indicators set out in Figure 5.1, it is recommended that:

• Key Contract Information is recorded by the procuring organisation and the information can largely be taken from the tender/contract documentation.
• Community Benefit Clauses Indicators are collected by the main contractor (ensuring that sub-contractor CB delivery is also captured) and reported to the procuring organisation on a monthly basis as part of the contract monitoring arrangements.
• Short-Term Sustainability Indicators should in the main be collected by the main contractor (again ensuring that sub-contractor CB delivery is also captured) and reported to the procuring organisation on a monthly basis as outlined above. However, where an individual has left and the contractor has no record of their destination, the procuring organisation should aim to contact that individual themselves.
• Additionality Indicators are finalised immediately at the end of the contract through a joint meeting between the procuring organisation, the main contractor and other sub-contractors as appropriate. Additionality is calculated by subtracting these from the total outcomes and then dividing by the outcome.

Longer-Term Sustainability Indicators

Ideally the monitoring system would also capture sustainability over the medium- to long-term, i.e. whether the individual or apprentice is still working after the end of the contract. The suggested indicators would be:

• Number of recruits from priority groups employed 26 weeks after end of contract.
• Number of apprenticeships from priority groups employed 26 weeks after end of contract.

There are two options to how this longer-term sustainability data could be collated:

• For the contractor to provide this data. However, given that the contract would have been signed off, there is no obvious motivation for the contractor to collect and supply this information. In addition, they would likely provide a partial picture as they could only provide information on those recruits still employed with them, but not on those employed elsewhere.
• For the procuring organisation to provide this data. This option requires the procuring organisation following up with each recruit/apprentice individually to capture their current employment status. To do this the procuring organisation would first need to collect the personal contact details of each individual (while the contract is ongoing), and then dedicate significant resources to contacting these individuals. Irrespective of the resources allocated, recognition is needed that response rates may be low and complete data will be difficult to secure.

In addition, the Scottish Government should explore alternative ways of monitoring long-term sustainability through data sharing agreements with other organisations holding employment and skills data, such as the DWP, Scottish Funding Council and SQA. A good example of this would be the Modern Apprenticeship data held by Skills Development Scotland. By 'tagging' apprentices recruited under CB clauses in the Corporate Training System (CTS), long-term sustainability data in terms of the number of apprentices that have completed their apprenticeship and the number of apprentices that have changed employer during the course of their apprenticeship could potentially be captured.

Outcomes Information

At the organisational level, we recommend that a series of headline indicators are collected that demonstrate the use and impact of CB clauses. These headline indicators - as set out in Figure 5.2 - are grouped into four types and explained in turn below:

• Use of Community Benefits Clauses in Contracts.
• Community Benefits Clause Indicators.
• Short-Term Sustainability Indicators.
• Additionality Indicators.

It should be noted that the number of indicators may appear extensive but Indicators 9 to 21 are not additional indicators and will have been captured as part of the monitoring information.

Use of Community Benefits Clauses in Contracts

On an annual basis, the number and value of contracts let by procuring organisations should be recorded to help procuring organisations complete their annual procurement report under the Procurement Reform (Scotland) Act 2014 and understand the extent to which they are adopting CB clauses across all the contracts they let.

There are also opportunities to monitor variations on the above depending on the different priorities of procuring organisations. For example, individual procuring organisations may wish to monitor by:

• Different types of contract (e.g. construction new build, construction repair and maintenance, provision of goods and provision of services) to assess progress on the roll out of CB use beyond construction.
• Different contract values at intervals below the £4 million level specified within the Procurement Reform (Scotland) Act 2014 (recognising that many contracts fall below the £4 million level).

Key Contract Information - Short Term Sustainability - Additionality Indicators

As mentioned earlier, Indicators 9 to 21 are not additional indicators and have already been captured. However, as well as being important monitoring indicators, these are also important CB performance measures at the organisational level that the Board and Senior Management Team should demand. For example, it is important that they know what impact the use of CB clauses in the contracts they let is having in terms of jobs, apprenticeships, work placements, qualifications and business opportunities for the communities they serve.

Figure 5.2 is not intended to be too prescriptive as the specific set of headline, outcomes indicators the Board and Senior Management Team choose to monitor may vary relating to the specific aims and objectives of the organisation (e.g. some organisations may want more indicators relating to contracts to SMEs and social enterprises). Furthermore, there is also the opportunity to aggregate the data for each contract to provide the total number of CBs achieved across all contracts by a procuring organisation, which might be more meaningful for the Board and Senior Management Team.

Figure 5.2: Outcomes Indicators (OI)

USE OF COMMUNITY BENEFITS CLAUSES IN CONTRACTS
OI1. Total Number of Contracts Let
OI2. Total Number of Contracts Let Over £4 million
OI3. Total Number of Contracts Let with Community Benefit Clauses
OI4. Total Number of Contracts Let Over £4 million with Community Benefit Clauses
OI5. Total Value of Contracts Let
OI6. Total Value of Contracts Let Over £4 million
OI7. Total Value of Contracts Let with Community Benefit Clauses
OI8. Total Value of Contracts Let Over £4 million with Community Benefit Clauses
KEY CONTRACT INFORMATION
OI9. Total Value of Contract
OI10. Number Recruited from Priority Groups
OI11. Number of Apprentices Filled by Priority Groups
OI12. Number of Work Placements for Priority Groups
OI13. Number of Qualifications Achieved Through Training by Priority Groups
SHORT-TERM SUSTAINABILITY INDICATORS
OI14. Number of Recruits from Priority Groups Employed at 26 Weeks After Job Start
OI15. Number of Apprenticeships from Priority Groups Employed at 26 Weeks After Apprenticeship Start
OI16. Number of Work Placements for Priority Groups Subsequently Recruited by Contractor/Sub-Contractor
ADDITIONALITY INDICATORS
OI17. Number of Recruits from Priority Groups that would have been taken on if the CB Clause had not been in place
OI18. Number of Apprentices from Priority Groups that would have been taken on if the CB Clause had not been in place
0I19. Number of Work Placements for Priority Groups that would have been offered if the CB Clause had not been in place
OI20. Total Value of Works Sub-Contracted to SMEs that would have been awarded if the CB Clause had not been in place
OI21. Total Value of Works Sub-Contracted to Social Enterprises that would have been awarded if the CB Clause had not been in place

In terms of who collects the data set out in Figure 5.2, Indicators OI9 to OI21 will be collected as part of the contract monitoring process. The relevant indicators, along with the Use of Community Benefits Clauses in Contracts indicators, should be collated and reported by the organisation's head of procurement on at least an annual basis to the Board and Senior Management Team.

Monitoring System

To collect the indicators outlined above requires each procuring organisation to have sufficient resources in place to collect, analyse and report on the indicators across all contracts let with CB clauses. Furthermore, where the data collection responsibility lies with the contractor for specific indicators, the procuring organisation must have the skills and expertise to request and indeed encourage contractors to provide the information required in a timely and accurate manner. This task will be made simpler by using a monitoring spreadsheet (like that used by CITB Construction Skills) that can effectively record progress made and outcomes against each indicator.

Reporting of Community Benefits

The monitoring of the CB indicators recommended above will generate a significant amount of data that can help inform (and increase) the future use of CB clauses by procuring organisations across Scotland. However, to be of real value the data needs to be widely reported and then acted upon. We recommend that the data is reported at the organisational level, but also collated by the Scottish Government to form a national picture of the use and impact of CB clauses over time.

In each procuring organisation, we recommend that a dedicated group with strategic responsibility for overseeing the use of Community Benefits is established. Its task will be to meet at least annually to review the progress made by the organisation in its use of CB clauses. This would require those responsible for monitoring the individual contracts to produce an annual procurement report that details performance by individual contract and at the aggregate, organisational level. The group can then assess what changes are required to secure greater Community Benefits.

In the Scottish Government, there is scope to consider whether the annual procurement reports that we recommend each procuring organisation should produce should form part of the organisational return required under the Procurement Reform (Scotland) Act 2014. More widely, the Scottish Government should commit to publishing a Community Benefits Annual Report based on the annual performance reports provided by each procuring organisation, highlighting illustrative and informative examples of good practice. This will allow a body of evidence on the usage and impact of CB clauses to be developed and inform future policy in this area.