Professionalism in nursing, midwifery and the allied health professions: report

4. Measuring professionalism

A subgroup of the working group, including academics from Dundee, Durham, Edinburgh Napier and Stirling universities, reviewed and explored the literature on mechanisms to recognise and measure manifestations of professionalism and professional behaviours. The subgroup identified an array of potentially suitable measurement tools (proxy measures) that might be appropriate to measure professionalism and to identify development needs. Essentially focused on behaviours, relationships, practice, context and culture, the tools cover diverse approaches that may facilitate quality improvement through professionalism. This chapter provides a brief summary of some of the key findings from the research: the complete research report can be accessed at
http://www.knowledge.scot.nhs.uk/qualitycouncils/support-workstreams/professionalism-(and-regulation).aspx.

Characteristics such as "care" and "compassion" are top priorities for health service users and are key to the professional principles that guide NMAHPs. Many of the caring and enabling behaviours are those which define "professional" behaviours, and their presence can be measured. The Care Governance Measurement Framework^[44] proposes a range of measures including variables that describe caring and enabling behaviours, their interrelationships and influences, and their impact on staff and patient experiences and outcomes.

Informatics principles should also assist in ensuring that data collected are fit for purpose, cost-effective and are used to drive quality improvement. Key principles include the reporting of data in real time (or near real time) to enable those who are accountable for the delivery of care to make informed decisions that promote and sustain quality patient care.

Potential tools

It is important to emphasise that no one tool or approach will suffice: one size does not fit all.^[45] It will be for individual organisations to determine the most appropriate tools to apply in specific contexts and to provide feedback to staff accordingly.

Collated views of co-workers: multi-source feedback

Multi-source feedback (MSF) involves collecting data from different sources - peers, clinical supervisors, non-clinical staff - to develop a broad indicator of practice patterns.^[46] One of the aims of MSF is to raise self-awareness of performance. It also seeks to encourage improvement and utilise feedback from both clinical and non-clinical peers^[47]. The ECO model (emotions, content, outcomes) is a three-step process developed from the counselling literature to facilitate feedback acceptance and use in MSF.^[48] Those who have used the model have found it useful and simple: it engages participants to reflect upon their feedback and performance and explore emotions, and clarifying content appeared integral to accepting and using the feedback. Using the ECO model to engage individuals in feedback discussions may prove useful in the future.

Patient opinion/satisfaction

Patient and service user satisfaction questionnaires can be another useful source of feedback for professionals.^[49] The Consultation and Relational Empathy (CARE)^[50] questionnaire focuses on relationship empathy as perceived by patients within healthcare consultations. The CARE measure has been validated with doctors, and the Healthcare Quality Strategy for NHSScotland^[51] is committed to pursuing its introduction in all clinical appraisals and with other healthcare professionals.^[52]

Conscientiousness index

The trait of "conscientiousness" can be measured objectively and has been shown to be an effective predictor of workplace performance in a number of settings. Data on undergraduate medical student performance on routine tasks show that the "conscientiousness index" data correlated positively with staff and peer understandings of professionalism.^[53] This work has been independently replicated elsewhere and is currently being explored for use in a number of postgraduate settings, including those for AHPs. A conscientiousness index is inexpensive to calculate and is reliable and objective. While it does not capture all elements of professionalism, it may reflect a significant part. There is potential for the conscientiousness index to be used with other healthcare groups.

Measurement of insightful practice

Measurement of insightful practice is a new concept that is being tested in the Tayside In-Practice Portfolio (TIPP) study.^[54] The method focuses on encouraging productive reflection (engagement and insight) and necessary action (lifelong learning and response to audit).

External and objective feedback based on patient care governance outcomes

Data on patient outcomes relevant to specific roles have been used within the NHS to promote quality improvement. These data could be developed to feed back to different professional groups on their personal and team performance.

Personal qualities assessments

The revised NEO Personality Inventory (NEO-P-IR) is an internationally recognised instrument to measure personality. It is a 240-item measure of a five-factor model - extraversion, agreeableness, conscientiousness, neuroticism, and openness to experience - that measures six sub-domains across each of these personality factors, including leadership, decision-making and stress management.^[55]

Independent measurement of the professional (such as situational judgement tests)

Situational judgement tests (SJTs) are used to assess performance in solving problems in the workplace. They usually present users with hypothetical and challenging scenarios they might face in the work environment, offering different approaches they may want to take to solve the problem (generally in multiple-choice format). SJTs have been described as valid and reliable tools with which to select professionals within organisations^[56] and have been used to reduce the number of applicants taken forward to more resource-intensive selection processes. This suggests that they may be effective as a filter in recruitment and appraisal processes.

Internal self-assessment of the team by its members

Questionnaires may be used to capture the perceptions of team members, allowing sharing of views and identification of opportunities to develop team cultures and improve quality.

External independent assessment of the team by others

Questionnaires completed by those working with the team and those experiencing care could be developed to provide feedback on culture.

A feedback exercise [full measurement report] shows that tools perceived as likely to demonstrate a strong breadth of performance as potential formats of measurement include:

• patient opinion/satisfaction questionnaires
• situational judgement tests
• external objective feedback on patient care outcomes (governance)
• team questionnaires on culture.

There are other potential methods of testing that were not included in the research, such as observation processes like the Workplace Culture Critical Analysis Tool (WCCAT),^[57] which supports continuous quality improvement and provides real-time feedback on quality interactions and culture. Observational and simulation processes were also not tested.

Making choices

The real test of any successful measure of professionalism is whether it supports the workforce to achieve improved patient, service user and carer experiences and outcomes. Successful implementation should reassure the public and help deliver the Healthcare Quality Strategy for NHSScotland^[58] quality ambitions.

Means of enabling individuals and teams to consider their professionalism need to be flexible and practical. When making choices on tools, it will be important to consider if they adequately cover necessary professional attributes, include a measure(s) of patient/service user feedback and are endorsed through personal development planning/review mechanisms.

The inclusion of patient/service user opinion offers challenges. Assuring anonymity will be important if valid feedback is to be captured, and this has been successfully achieved with the CARE process with patients.^[59] Team questionnaires are potentially more difficult, but improved technology and better web access should make it possible to feed back increasingly sophisticated information to teams on measurable patient outcomes.

The working group's suggestions on possible tools for measuring professionalism in the NMAHP workforce are shown in Table 4. Consideration needs to be given to the infrastructure required to train and support a range of people to use the tools and give the necessary feedback as part of the improvement cycle.

Table 4. Working group's suggestions on possible tools for measuring professionalism in the NMAHP workforce

NMAHP group	Suggested tools
Pre-registration students	Situational judgement tests Personal qualities assessments Conscientiousness index
Newly qualified practitioners	Conscientiousness index Situational judgement tests Observation of practice and clinical skills
Established practitioners	Conscientiousness index Peer feedback (such as MSF) Patient feedback (such as CARE) Team (culture) questionnaires (possibly based on care governance) Observation of practice and clinical skills
Support workers	Peer feedback (such as MSF) Patient feedback (such as CARE) Team (culture) questionnaires (possibly based on care governance) Observation of practice and clinical skills Personal qualities assessments
Managers	Peer feedback (such as MSF) Patient feedback (such as CARE) Team (culture) questionnaires (possibly based on care governance)

Recommendation 13. Ensure that information-gathering and measurement tools are appropriate to needs and are endorsed and facilitated through systems such as personal development planning and review processes.

In summary:

• there is no single measurement tool that can be relied upon to measure professionalism, given the diverse nature of the concept and the multiple roles in which high standards are important
• there is a need for a range of tools to meet local circumstances and needs
• there is a need to explore the validation of approved team (cultural) measures
• measurement of outcomes should be linked to professionalism to ensure that any implemented process is successful
• any adopted system(s) will need to align with the Scottish Government's Quality Measurement Framework
• current work on care governance may produce useful cultural measures of professionalism.