Independent Review of Audiology Services in Scotland

Quality Assurance Sub-Group

Key points

• Quality assurance is the process of checking standards of good practice are met and encouraging continuous improvement.
• The Lothian report prompted scrutiny of quality assurance of audiology services across the UK.
• KPIs were selected to provide a sample view of the quality of service provision and revealed widespread shortfalls in performance against recognised service quality measures.
• A peer review-based sampling audit of clinical skills in two key areas of the paediatric hearing loss pathway identified the need for urgent remedial measures to improve skills and establish ongoing assurance arrangements.
• Survey results indicated limited senior oversight or interest in quality assurance of audiology services by Health Boards. At service level there was, in most cases, no evidence of regular or recent clinical audit activity.
• Overall, findings point to the need for a suite of measures to assure service quality on an ongoing basis.
• There is little evidence of collaboration relating to quality assurance at a national level across Scotland between health boards, SCIP or UNHS, or with other UK countries.
• Establishment of effective and nationally coordinated quality assurance systems for audiology is achievable, as evidenced elsewhere in the UK.
• There needs to be one nationally recognised body with a remit to oversee quality assurance across audiology pathways.

Introduction

The BAA Lothian review identified a series of serious, significant issues, particularly within the early years (under 5) age groups of the paediatric audiology service. It was recommended that a comprehensive quality assurance programme should be established for the clinical aspects of the service – to include peer review and a reporting/oversight mechanism to director level, with arrangements for a suitable peer reviewer to be identified.

Informed by these findings, and within the wider scope of the National Review, the remit of the Quality Assurance Sub-Group was to:

• provide a quality assurance appraisal of audiology services (all ages), surveying key elements of existing service provision, with a particular focus on issues impacting on patient outcomes;
• review existing quality assurance arrangements, making recommendations necessary to establish robust quality assurance processes, while progressing service quality, improvement and outcomes for patients on a permanent basis.

Quality assurance is the process of checking that standards are met and encouraging continuous improvement. Assuring and driving up the quality of services is essential if audiology is to achieve the intended benefits to population health, while minimising unintended harms to those receiving services. Further information and definitions are provided on the Review website (1).

The Sub-Group was aware of previous national recommendations and arrangements for quality assurance organised on a national basis, although not the current provision across Scotland. It was agreed that a variety of types of information should be obtained to determine the current position, addressing the second bullet point above.

In context, many other areas of the UK currently have no robust systematic external quality assurance against evidence-based national service quality standards, although a nationally coordinated approach is well-established in Wales. The findings of the Lothian report have generated activity in many parts of the UK, leading to scrutiny of care and with the goal of improving quality assurance to those commissioning and responsible for audiology services.

The Sub-Group also received the findings of the Reference Group’s stakeholder engagement exercises conducted as part of this Review and the survey of audiologists in Scotland carried out by the BAA. These provided information pertinent to the remit of the Sub-Group and recommendations.

It is important to note that while all Health Board audiology services were within scope of the Review, services delivered by the SCIP are subject to quality assurance scrutiny by NSS. For UNHS, the NSS provides collation of primary screening data and performance against KPIs, while the Health Boards are responsible for delivery against the related targets.

Methodology

Survey of KPIs

The Sub-Group devised a set of KPIs selected to provide a sample view of quality of service provision across adult and children’s audiology services, with reference to national service quality standards (8, 9). Each NHS Board was asked to complete a questionnaire survey (Appendix E) based on the KPIs, via heads of audiology services. The SCIP and the UNHS programme were also surveyed, the latter in relation to performance against existing KPIs.

It is important to note the limitations of this survey. Data were gathered through a one-off desk-based exercise. There was no onsite verification of submitted responses; and the depth and range of investigation of service quality would not match that of a robust external audit process against a wider range of criteria based around the patient pathway. There was no direct observation of practice; use of a site-visit-based approach, with briefing of all stakeholders and interactive scrutiny of local practice, would be expected to improve the accuracy and scope of outcomes reported from the limited desktop audit exercise conducted here.

Peer-review exercise of audiology skills

The Sub-Group devised two peer-review exercises of clinical skills for two key elements of the paediatric hearing loss pathway: ABR assessment following referral from UNHS; and technical aspects of hearing aid management.

ABR assessment is used to diagnose hearing loss. It provides information on the level and type of hearing loss and is used to guide decisions on clinical management – for example, fitting of hearing aids where indicated. Subsequently, hearing aids need to be fitted accurately, with measurements made to confirm this, in order to ensure optimum amplification, with best access to speech sounds in particular. Such audiological procedures are complex and require application of high-level skills while adhering to evidence-based professional guidelines.

Audiologists who perform such procedures were requested to submit materials for peer review. The exercise performed was to audit existing practice against BSA professional good practice guidance (25). Although of limited scale, this audit exercise and analysis report (Appendix F) provided useful insight into the training needs of audiologists based on care of individual patients.

A case sampling approach was adopted reflecting limitations of time and resources. Outcomes relating to individual cases were reported to audiologists and the Scottish Government to an agreed process. The exercise also had the benefit of providing familiarity with external peer-review practices for participants. However, the exercise did not explore audiologists’ perception of their skills or levels of training/knowledge, or the adequacy of resourcing and organisational support for these demanding and complex activities.

Survey of quality assurance arrangements at Health Board level

The Sub-Group devised questions to survey heads of service on the current arrangements for quality assurance at Health Boards and their collaboration with other services, including ear, nose and throat (ENT) and specialist services (SCIP and UNHS). The questions were informed by established good practice in services in other UK countries and a definition of clinical audit provided by the National Institute of Health and Care Excellence. Health Board audiology services were then surveyed, via heads of services.

Limitations on scrutiny of quality assurance through the Review

Due to limitations of time and resources, peer-review exercises and cases-level audits were not performed for other key elements of audiology pathways, notably behavioural assessment of pre-school-age children and adult pathways (including balance, tinnitus and implantation). Lack of close scrutiny of case-level practice for such pathways means they cannot be reported upon here.

Findings

Survey of KPIs

The limitations of the approach taken to gather information on performance against KPIs should be noted (see above). However from a detailed analysis of a limited number of KPIs sampled, it is evident that there were shortfalls across all Health Board audiology services. As the KPIs were derived as a sample from across existing evidence-based Scottish Government children’s hearing services and adult rehabilitation service quality standards, it is reasonable to conclude that Health Boards might currently fall short of acceptable levels of compliance against these national standards if a more extensive site-based external audit were to be conducted. The outcomes of this survey point towards the need for robust external audit of services against existing service quality standards. A list of the surveyed KPIs can be found at Appendix G.

Peer-review exercise of audiology skills

The peer-review exercise to explore audiology skills identified shortfalls across those Health Boards delivering these services.

There was a general lack of adherence to professional best practice guidance to provide assurance of competence in diagnostic ABR assessment and hearing-aid fitting, which may reflect current specialist skills and previous training. It should be noted that there is no existing external peer-review scheme in place for these key elements of the patient pathway. When introducing a robust external peer review exercise of this type, it is likely that issues/concerns at different levels will be revealed. This has been the case: there were at least minor shortfalls against good practice guidance identified at all Health Boards offering ABR assessment and/or hearing-aid fitting.

Given the scale and significance of audit findings for the ABR audit (shortfalls in practice against guidance) it was decided to expedite development and submission of three recommendations from the Sub-Group (see recommendations 52-54). The hearing-aid peer-review exercise revealed reports of equipment shortfalls and lack of training to provide for fitting of hearing aids to best practice standards. Specific recommendations were escalated to respective Health Boards. For further detailed outcomes of analysis of both exercises, see Appendix F.

Survey of quality assurance arrangements in Health Boards

For the majority of services there is no evidence that regular or recent clinical audit has been undertaken. Some measures of performance (for example, waiting times, referral rates) were reported and there were some examples of service evaluation to inform or assess service change.

There is minimal evidence of joint clinical audit with ENT. There was no evidence of joint clinical audit with other Health Boards, SCIP or UNHS services.

There is no evidence of external audit or review of services in recent years, save for paediatric audiology services in NHS Lothian in 2021.

There is no evidence from the majority of Health Boards of inter-Health Board quality-related performance benchmarking activity.

There is no evidence of non-ABR-related peer review for the majority of Health Boards and no evidence of external peer review of clinical practice.

The majority of Health Board audiology services submit regular (monthly) reports within their Board related to referral to treatment (RTT) and clinical activity performance. This is often limited to specific pathways and it is unclear where this data is ultimately reviewed. There were also limited reports of submission of any quality-related data other than that related to RTT access times.

No Health Boards report having an audiology service quality policy/manual/system. There was some reference to continued use of service quality standards. There is little evidence of collaboration relating to quality assurance at a national level across Scotland between health boards, SCIP or UNHS, or with other UK countries.

For a list of questions used for the survey of Quality Assurance Arrangements, see Appendix H.

Views from heads of service on measures required to improve quality assurance and services

There was a clear recognition from heads of service about the existing gaps and variation in quality assurance and they were able to identify a range of mechanisms to address these. However, they also identified a lack of resources as a significant barrier to making progress.

More specifically, there was support for:

• implementation of a formal process for quality assurance using Scottish service quality standards and peer review. To include an external audit element, and requiring a national coordinated approach and Scottish Government support;
• additional (human) resources to enable service managers to have sufficient and protected time to review and improve services. Training in clinical audit within an audiology context;
• development of resources (materials) to support quality assurance activity across Scotland and ensure alignment – for example, report templates, guidance documents;
• a coordinated and collaborative approach across audiology in Scotland – for example, establishment of a national audiology quality assurance team; audiology quality leads identified within each Health Board working together across Boards and linking with the Scottish Government; regular heads of service quality assurance meetings; heads of service working together in an agreed way;
• improved patient management system and tools for improved data collection and extraction;
• re-establishment of audiology patient reference groups;
• improved audiology accommodation;
• improved profile of audiology – recognition of audiologists as independent professionals to enable service improvement, including through role extension and raised profile aligned to the importance of audiology services;
• quality standards that are patient-centred;
• increased collaboration with other services (for example, speech and language therapy, education, audiology outside Scotland).

Summary

While there were areas of good practice, shortfalls in quality and quality assurance were identified in each of the surveyed approaches and were evident across Health Boards. Although the root causes were not explored systematically, there is reference to resourcing, lack of coordination, lack of collaboration, poor profile of services and absence of organisational focus on quality (other than access times). There is acknowledgement among those leading and others delivering services that there is a need for improvement and an appetite to do so.

The Sub-Group is keen to emphasise that shortfalls reported do not reflect adversely on the commitment of individuals aspiring to deliver high-quality care.

Going forward, quality assurance should be pursued using a variety of approaches, since each has different strengths and weaknesses. Evidence-based service quality standards, even if audited robustly, are not sufficient in isolation to assure quality. It is recognised, for example, that audit against standards needs to be augmented with reporting against KPIs at a health board and national level. This will provide ready availability of data to benchmark against service standards compliance and KPIs to drive forward quality improvement systematically, across the country.

Collaboration in pursuit of quality assurance and supportive activity is highly desirable. Priority should be given to recommendations that feature collaboration to help realise the benefits: namely, to provide efficiencies, encourage sharing of good practices with respect to quality assurance; to encourage a culture of openness; to ensure wider engagement in quality assurance activity (within and across audiology teams); and to contribute towards a unified national identity/profile for audiology.

The Quality Assurance Sub-Group recognises the inter-relationships and interdependencies with the remits of the other Sub-Groups. Indeed, provision of high-quality and quality-assured services is reliant on effective leadership operating within governance structures that are fit for purpose and on the availability of staff with appropriate skills.

It is considered critically important that structures and dedicated resources are available to oversee:

• the development of service quality standards;
• robust external audit process;
• escalation where indicated;
• national planning for service quality improvement;
• national reporting.

There is a need for openness with respect to reporting on service quality and performance of services. It is by doing so that remedial steps can be taken to address shortfalls in service quality as and when they occur, or, more positively, to demonstrate improvement to services.

Recommendations 42 to 55 are especially relevant to the work of this Sub-Group.