Building standards - digital competency assessment system: options appraisal

5. Options Appraisal

5.1 Option 1: Adapt existing CAS content

1. User feedback on the existing CAS from local authority building standards surveyors shows that it is too large and intimidating. Some adjustments to the content of the existing CAS framework spreadsheet could be realised quickly and cheaply which would improve user experience. While these would not resolve the identified issues, they might be worth putting into practice in the short term as an interim solution to help end users.

2. The following are some examples of improvements that could be made:

• Moving specific pieces of guidance held in the CAS handbook into the relevant competency items within the CAS framework spreadsheet
• Filtering the competency items within the CAS framework spreadsheet to only those that are relevant to the user filling it out
• Reviewing the CAS framework to merge duplicate and remove unnecessary content

Cost

3. We would expect this option to attract the following costs:

• License and maintenance: Free
• Implementation: Potential small amount of consultancy from a specialist

Implementation Considerations

4. We would expect this option to take a maximum of 1-2 months to implement.

5. Adaptation of the existing CAS content could be implemented by the existing Building Standards Hub / LABSS stakeholders with the assistance of a specialist with experience in improving the useability of the existing Excel spreadsheet.

Strengths

6. This option has the following strengths:

• Low cost and effort
• Easy to implement quickly
• Minimal training or changes to the current process required since users are already used it
• Can easily adapt the toolkit to remove and/or add competency items

Weaknesses

7. This option has the following weaknesses:

• Does not address all user requirements
• Still quite a cumbersome and time-consuming process
• Does not address all of the issues that have been reported with the CAS
• Use of PDF is not accessible
• Substantial changes would need to be made in order for users to feel the impact
• Users with low Excel confidence/knowledge will continue to struggle using the toolkit without proper Excel training

5.2 Option 2: Leverage existing Virtual Learning Environment functionality

1. The existing Virtual Learning Environment includes a competency management system module. This module would allow for the creation of a competency framework which could be populated with the circa 700 competency items included within the CAS. The module does not allow for full customisation, so it would not be possible to capture all of the data points that are currently captured within the CAS for each competency item.

2. However, it would be possible to capture the most important information, i.e.:

• Description of the competency item
• Links to relevant training items
• The employee's compliance level (achieved/not achieved/not applicable) with the competency item

3. Further investigation would be needed to prove the feasibility of this option.

Cost

4. We would expect this option to attract the following costs:

• License and maintenance: Potentially already included within existing VLE costs, subject to confirmation with existing vendor
• If using an external identity provider, this would attract an additional indicative cost of £20-100 per month
• Implementation: Potential small amount of consultancy fees if needed, to be confirmed by Building Standards Hub in discussion with existing vendor

Implementation Considerations

5. Estimation of timescales for this option would require further investigation with the current VLE provider, Learning Pool

6. This option is likely to be outside of the scope of the existing contract with Learning Pool and may therefore require amendments or a separate contract.

Strengths

7. This option has the following strengths:

• Low cost and effort
• Easy to implement quickly
• Single platform for both virtual learning and competency management

Weaknesses

8. This option has the following weaknesses:

• Does not allow for capture of all of the CAS data points
• Potential vendor lock-in
• No scope for custom or conditional workflows, for example allowing for questions or sections to be skipped
• A change in the CAS process is needed to use this option, which may add implementation time to establish the competency assessment process

5.3 Option 3: Bespoke .Net development

1. A bespoke development would allow complete control over a customised solution which is tailored directly to the requirements of the Digital CAS therefore providing the best end result. This option gives the greatest flexibility and best functionality, and therefore will also require the most up-front effort and cost of the options considered, due to the additional development effort required.

2. Microsoft's .Net platform is recommended as the platform of choice due to the following reasons:

• .Net is free and open source
• It has a wide variety of use cases within public sector organisations
• There is good support and integration with the Azure cloud platform, which is available as a hosting option via Scottish Government's Cloud First team
• It is aligned to the tech stack in use in other Scottish Government developments
• It has a significant market share and support community, meaning that there are many providers available for support, maintenance and implementation of any future updates

3. While the proposed functionality of the Digital CAS does not appear to be particularly technically challenging, the volume of competency items may raise some challenges in terms of design. Any custom development option should therefore include a suitable amount of design time to work out the best way to display and capture the data.

4. Data held within a .Net solution would be made available in a standardised format which facilitates reporting within tools that SG may have already, such as Power BI.

5. A potential bespoke developed platform would comprise the following components:

• Web Application User Interface
  • Provides a user friendly experience for local authority users to manage their own competencies and those of their direct reports
  • Provides a user friendly experience for Building Standards Hub administrators to administer competencies in line with changing legislation
  • Allows for single sign-on between the Digital CAS, the VLE and the LABSS website
  • Provides reporting and data export for managers and administrators
• Application Service
  • Includes back-end logic relating to user processes and competency management
• Data Store
  • Stores all user and competency data in a standardised format so that it can be used as a data source for common reporting

Cost

6. For a bespoke .Net development of a Digital CAS platform we would expect the following costs to be incurred:

• Indicative Recurring fees: £500 per month for Azure hosting
• Indicative Implementation Cost: £150k for bespoke development, including additional overheads outlined in Implementation Considerations below
• If using an external identity provider this would attract an additional indicative cost of £20-100 per month

7. Implementation costs and associated timescales can be roughly broken down into:

• 30% for Alpha
• 50% for Beta
• 20% for Live

8. The above costs could increase if:

• Full DSSS panel assessment is required. This can increase the effort by up to 30%
• Additional project phases are required
• Additional rounds of user testing or performance testing are required.

Implementation Considerations

9. We would expect a bespoke .Net implementation of a Digital CAS platform to take about 6-9 months to implement, assuming the implementation team are allocated 100%. This includes all activities needed to get to a live product including development (frontend, backend and cloud infrastructure), testing, design, content design, user research, analytics, deployment, consultancy and project management.

10. For implementation of a bespoke .Net development, we would expect a multi-disciplinary implementation team to consist of:

• Front end, back end and cloud infrastructure developers
• Tester
• Designer
• Content Designer
• Analytics Consultant
• User Researcher
• Project Manager
• Digital Consultant
• Technical Consultant

Strengths

11. This option has the following strengths:

• Tailored exactly to requirements
• Provides the flexibility to produce a solution that strongly aligns with user needs
• Developed in line with SG development and security standards
• Options for hosting including the SG Cloud First platform
• Data is held in-house so vendor lock-in is avoided
• Lower recurring costs

Weaknesses

12. This option has the following weaknesses:

• Most effort and therefore a higher implementation cost
• Potentially longer time to develop

5.4 Option 4: Low-code development

1. Low-code platforms allow for rapid development of custom solutions. They do not provide quite the same level of customisation as a bespoke development, but they do allow for a good level of customisation which would support the requirements of the Digital CAS. Low-code development is generally quicker than traditional development due to foundational components already being in place and the use of templates.

2. A number of low-code platform options are available which could be used to build a Digital CAS system, including:

• Specific low-code development platforms, such as Microsoft Power Platform or OutSystems
• CRM systems such as Microsoft Dynamics or Salesforce
• Open source form builders

3. Of the low-code platforms reviewed, the Microsoft Power Platform is recommended as the platform of choice:

• Applications built with Power Platform are often familiar to users who already use Office 365 products
• There is a good market share and support network available
• There are a wide variety of use cases already in place throughout the public and private sectors
• Power Platform is scalable by default based on demand and therefore would not need any intervention should load increase
• Development and reasonable operational use are included in a Microsoft 365 Enterprise license, which SG could potentially use to reduce costs

4. As with the custom .Net option, while the proposed functionality of the Digital CAS does not appear to be particularly technically challenging, the volume of competency items may raise some challenges in terms of design. Any custom development option should therefore include a suitable amount of design time to work out the best way to display and capture the data.

5. Similar to a bespoke .Net application, a Power Platform implementation of the Digital CAS would be tailored directly to user needs. There will be some areas where there may be less flexibility than a .Net application, such as how screens can be structured and how data can be stored, but these limitations are minor.

6. Data held within a Power Platform solution would be available for reporting via Power BI and other reporting tools.

Cost

7. For a Power Platform implementation of a Digital CAS platform we would expect the following costs:

• Indicative recurring costs: £500-1000 per month (depending on how often users log in). These costs relate to the cost per user for Power Pages authenticated users, which attract a cost of £150 per 100 users who log in per month. This is a centrally managed cost which is liable to be paid by the organisation which hosts the application.
• Indicative implementation costs: £100k for Power Platform development, including additional overheads outlined in Implementation Considerations below
• If using an external identity provider this would attract an additional indicative cost of £20-100 per month

8. Implementation costs and associated timescales can be roughly broken down to:

• 30% for Alpha
• 50% for Beta
• 20% for Live

9. The above costs could increase if:

• Full DSSS panel assessment is required. This can increase the effort by up to 30%
• Additional project phases are required
• Additional rounds of user testing or performance testing are required.

Implementation Considerations

10. We would expect a Power Platform implementation of a Digital CAS platform to take about 4-6 months to implement, assuming the implementation team are allocated 100%. This includes all activities needed to get to a live product including Power Platform development, testing, design, content design, user research, analytics, deployment, consultancy and project management.

11. For implementation of a Power Platform Digital CAS solution, we would expect a multi-disciplinary implementation team to consist of:

• Power Platform Developers
• Tester
• Designer
• Content Designer
• Analytics Consultant
• User Researcher
• Project Manager
• Digital Consultant
• Technical Consultant

Strengths

12. This option has the following strengths:

• Rapid development
• Lower development time and therefore cost than bespoke .Net
• Good degree of flexibility (with limitations)
• Tailored exactly to requirements (with limitations)
• Developed in line with SG development and security standards
• Data is held in-house so vendor lock-in is avoided

Weaknesses

13. This option has the following weaknesses:

• Monthly per-user cost for building standards users would increase costs as more people enter the profession as set out by the workforce strategy
• Not as customisable/flexible as bespoke .Net development

5.5 Option 5: Purchase existing Software-as-a-Service (SaaS)

1. A review was carried out on available SaaS products which include capabilities for competency management. Several Learning Management System (LMS) platforms include such functionality, however we found no competency management products which could be procured on their own.

2. If there were an appetite to replace the incumbent VLE with another supplier, then a SaaS provider who provides a robust competency management system could be considered, but this is unlikely the most cost-effective solution as the VLE implementation is already underway with the current supplier.

Cost

3. License and maintenance:

• SaaS options are generally licenced on a per user basis
• Costs vary by supplier, but one example gave a cost of £2.50 per user licence per month, which would give an approximate cost of £1500 per month if all 600 local authority verifier users were provided with a licence.
• If using an external identity provider this would attract an additional indicative cost of £20-100 per month

4. Implementation:

• Implementation costs would consist of professional services from the chosen supplier to assist with the configuration of the SaaS product to suit the DCAS and VLE requirements
• It has not been possible to provide an estimate of this cost at this stage.

Implementation Considerations

5. Timescales for this option would vary dependent on the supplier chosen. It is expected that an implementation team would be provided by the chosen SaaS vendor.

Strengths

6. This option has the following strengths:

• Low time-to-implement
• Low capital expenditure

Weaknesses

7. This option has the following weaknesses:

• Not as flexible as custom options
• May not meet all user requirements
• Won't be interoperable with existing VLE
• Can only be procured as part of a larger LMS system, so not likely to be cost-efficient
• In general, SaaS licensing costs are recognised as being high per month, per user, therefore costs would increase as more people enter the profession as set out by the workforce strategy

5.6 Cost Comparison

1. The below table presents the approximate costs and timescales of each option for comparison.

Table 5.6 – Cost Comparison

	Option 1 (Amend existing)	Option 2 (VLE)	Option 3 (Bespoke .Net)	Option 4 (Low-code)	Option 5 (SaaS)
Recurring cost	Free	Included	£500 per month	£1000 per month	£1500 per month
Identity Provision	N/A	£20-100 per month	£20-100 per month	£20-100 per month	£20-100 per month
One-off cost	Small consultancy fee	Small consultancy fee	£150k	£100k	(varies)
Time to implement	1-2 months	TBC	6-9 months	4-6 months	(varies)

5.7 Scoring

1. The below table presents each of the above five approaches and how they compare against the defined assessment criteria. Scoring is relative between the approaches. The minimum score for each criterion is 1 and the maximum score is 5. All options are scored out of 40.

Table 5.7 – Scoring of Options based on Assessment Criteria

	Criteria	Option 1 (Amend existing)	Option 2 (VLE)	Option 3 (Bespoke .Net)	Option 4 (Low-code)	Option 5 (SaaS)
Primary	Alignment with Requirements	1	2	5	4	3
	Costs	5	5	3	3	2
	Security	2	3	5	5	3
Secondary	Timescales	5	4	2	3	4
	Extensibility	1	2	5	4	2
	Ease of Maintenance	2	3	4	4	4
	Interoperability	1	2	5	4	2
	Reuse of Services	4	4	3	3	1
	Total	21	25	32	30	21

5.8 Hosting Considerations

1. Options 3 (Custom Development) and 4 (Low-code development) must be hosted within a cloud platform. It would be recommended to choose the same provider for Identity Provision and Hosting. Approximate hosting costs are included earlier in this report.

2. The following components would require to be hosted:

• Application front end
• Database back end
• Identity Provider

3. The chosen cloud platform and development platform will determine which exact components are hosted and how they are provisioned. The following hosting options are available:

• Scottish Government
  • The Scottish Government Cloud First platform offers hosting on AWS and Azure platforms.
• Improvement Service
  • The Improvement Service offer hosting on the AWS cloud platform and have a track record of providing digital services to all 32 local authorities in Scotland, though not specifically within the same remit as the Digital CAS.
• Third Party
  • An independently procured cloud solutions hosting provider could be chosen to host the Digital CAS. This would provide the most flexibility, but would be subject to additional security assessments.

5.9 Identity Provision

1. Any of the options could allow users to authenticate using basic username/password authentication, however in order to fulfil the requirement of single sign-on between the Digital CAS and the Virtual Learning Environment (VLE) with a single identity, an identity provider is required.

2. The current VLE, Learning Pool, supports single sign-on using the SAML 2.0 authentication protocol. Therefore, any Digital CAS system should ideally support this protocol.

3. During the process of this research, we were not able to identify any pre-existing identity provision platform which is already federated with the 32 local authorities, therefore it will not be possible to allow LA users to log in using their own corporate IDs. However, it would be possible to allow LA users to create an identity with the chosen Identity Provider, which is then used to log into both the VLE and the Digital CAS.

4. There is a stated desire for users to be able to login to the LABSS website using the same credentials they use to log into the VLE and the Digital CAS. This can be accomplished using an external identity provider which supports SAML 2.0, however it may require additional effort from the LABSS website provider to enable external identity provision and single sign-on.

5. The following options are available for identity provision:

• Scottish Government
  • The Scottish Government iTECS department maintain an identity provider which can be used by users external to Scottish Government. This identity provider supports the SAML 2.0 protocol and could therefore be used to allow LA users to create accounts and be authenticated for both the VLE and the Digital CAS.
• Improvement Service
  • The Improvement Service maintain an identity provider which, while primarily used for businesses and members of the public, can be used by LA users as well. This identity provider supports the SAML 2.0 protocol and could therefore be used to allow LA users to create accounts and be authenticated for both the VLE and the Digital CAS.
• Third Party
  • A cloud-based identity provider can be procured from and hosted by a third party which would have the same features as those offered by Scottish Government and Improvement Service.

6. Indicative costs for an external identity provider are £20-100 per month, however this may be reduced depending on the chosen vendor due to volume discounts or negotiated rates.