Health and social care staff experience: report 2019

Response Rates

All questions on the iMatter questionnaire are mandatory to eliminate partial loss of data.  Only those questionnaires that have every question answered can be included within the analysis.  The response rate shows the number of staff issued with the questionnaire (Recipients) and the number of staff who responded (Respondents) as an overall percentage.  A response rate of 60% is required for teams of 5 or more and 100% for teams of 4 or less to generate a report.  This is to ensure anonymity and also the higher the response rate, the more realistic the feedback of how staff feel about working in their team.  In total 179,453 questionnaires were issued and 111,512 usable responses were received.  This equates to an overall response rate of 62%.

2019 Board Response Rates

While the overall response rate for Health and Social Care for 2019 is 62%, there is considerable variation in response rates across the Boards, ranging from 53% to 93%.

There are 5 Boards, all of which are National Boards, that stand out in terms of high response rates:

- NHS Health Scotland (93%)
- Healthcare Improvement Scotland (90%)
- NHS Education for Scotland (87%)
- NHS National Services Scotland (82%)
- The State Hospital (79%)

The highest response rate in a Geographic Board is NHS Forth Valley (68%). NHS Borders has the lowest response rate at 53%, with a further 3 Boards not reaching the 60% threshold:

- NHS Western Isles (56%)
- Scottish Ambulance Service (59%)
- NHS Greater Glasgow and Clyde (59%)

Comparing 2019 Response Rates to Previous Years

The table below shows the response rates for the three years that iMatter has been in place.  The overall response rate for Health and Social Care has risen to 62% this year, an improvement of 3 percentage points over 2018. It is now only 1 percentage point below the 63% achieved in 2017.

Across the 22 Boards, 17 have an improved response rate in 2019 over 2018, two have remained unchanged and three have declined.

Board	2017	2018	2019	Response Rate Movement 2019-2018 (pp)
Health and Social Care	63%	59%	62%	+3
Golden Jubilee Foundation	68%	63%	67%	+4
Healthcare Improvement Scotland	80%	86%	90%	+4
NHS 24	67%	70%	65%	-5
NHS Ayrshire & Arran1	64%	59%	60%	+1
NHS Borders	61%	53%	53%	0
NHS Dumfries & Galloway	63%	59%	66%	+7
NHS Education for Scotland	81%	84%	87%	+3
NHS Fife	62%	53%	62%	+9
NHS Forth Valley	65%	62%	68%	+6
NHS Grampian	64%	60%	62%	+2
NHS Greater Glasgow & Clyde	58%	54%	59%	+5
NHS Health Scotland	85%	91%	93%	+2
NHS Highland	58%	51%	60%	+9
NHS Lanarkshire	65%	62%	65%	+3
NHS Lothian	65%	63%	63%	0
NHS National Services Scotland	76%	77%	82%	+5
NHS Orkney	73%	83%	66%	-17
NHS Shetland	61%	56%	63%	+7
NHS Tayside	65%	58%	61%	+3
NHS Western Isles	52%	52%	56%	+4
Scottish Ambulance Service	64%	64%	59%	-5
The State Hospital	78%	77%	79%	+2

 The largest increases from 2018 to 2019 are

- NHS Highland up 9 percentage points from 51% to 60%
- NHS Fife up 9 percentage points from 53% to 62%
- NHS Dumfries and Galloway up 7 percentage points from 59% to 66%
- NHS Shetland up 7 percentage points from 56% to 63%

All four of these Boards succeeded in increasing their response rates to above the 60% threshold. Case studies below illustrate actions taken to achieve these increases:

¹The NHS Ayrshire and Arran response rate for 2017 was amended following the 2018 report publication from 63% to 64%.

Case study 1: Improving Response Rate

NHS Fife Tea Break Campaign 2019 recognised that as well as staff knowing about iMatter, they also need to have access to the survey in an environment that was conducive to participation.

“We co-ordinated a series of visits direct to staff workplaces with a tea trolley, tea urn, biscuits, fruit (had to ensure our healthy working lives credentials were sound) and access to the iMatter survey. We had a number of laptops, ipads and access to local follow on printers for staff who were keen on completing paper copies. Our two trolleys were supported by Communications staff, HR staff and partnership representatives who volunteered to assist.”

An example of the process can be seen here at https://youtu.be/hadKaVWUqFk

Case Study 2: Improving Response Rate

NHS Highland started by providing greater resource in the Preparation and Confirmation Stages. This led to a considerable reduction in the number of teams. There were a series of iMatter Awareness sessions and weekly reporting of response rates.

Actions were taken to reduce the volume of paper surveys and an increased response rate was achieved among those responding on paper. The most significant change was the level of support from senior management in the Board. The interim Chair and the new CEO. The weekly CEO bulletin was used to publicise key aspects of iMatter and considerable support came from the Director of Communications who ensured important iMatter messages were distributed via the Team Brief.

Case Study 3: Improving Response Rate

NHS Dumfries & Galloway took a number actions, building on previous years activity including a considerable increase in Action Plans in 2018.  They delivered Staff Awareness Sessions aimed at highlighting the importance of staff engagement the cycle of improvement, stages of the iMatter cycle and how to effectively action plan. 

They increased communications throughout the cycle – all staff emails, articles in Workforce Briefing paper and weekly staff Core Briefing and created an iMatter video.

Specific action was taken to reduce the number of paper surveys by offering Support Services staff the opportunity to participate via providing a personal email address instead of paper questionnaires. This resulted in 25% of Support Services staff opting to complete online.

Looking at progress from 2017 there are three Boards that have increased their response rates each year:

- Healthcare Improvement Scotland (80%, 86%, 90%)
- NHS Education for Scotland (81%, 84%, 87%)
- NHS Health Scotland (85%, 90%, 91%)

In contrast, the response rate in NHS Orkney has dropped from 83% in 2018 to 66% in 2019. NHS Orkney’s relocation to a new hospital led to some internal delays and technical issues. The additional workload to implement this move and maintain patient care are likely to have impacted staff’s availability to complete the iMatter questionnaire this year.

Method Effect – online and paper response rates

In order to ensure all staff have the opportunity to take part in iMatter, paper questionnaires are distributed to those without access to the online survey.

In 2019 86% of surveys were issued online and 14% were on paper. The proportion of paper surveys issued has continued to drop slightly from 16% in 2018 and 18% in 2017.  The share of the responses received is 92% online and 8% paper. This is due to the higher response rate to the online survey, leading to online being a higher proportion of the responses.

Health and Social Care 2019	Volume issued	% of Volume Issued	Usable Response Volume	% of Responses Received	Response Rate by Method
Online	153,989	86%	102,099	92%	66%
Paper	25,464	14%	9,413	8%	37%
Total	179,453		111,512		62%

Both online and paper response rates have increased from 2018. The online response rate has improved by 2 percentage points and the paper response rate by 6 percentage points.

Typically the Geographic Boards make more use of paper surveys than their National counterparts. The highest use of paper surveys is in NHS Grampian and NHS Greater Glasgow and Clyde, with both issuing paper surveys to 19% of their staff.

Further detail of response volume and response rates by method are contained in Appendix 4.

As seen in the overall response rates there is considerable variation across Boards in both the online and paper response rates.

The three Boards with the highest overall response rate only used the online survey, as did NHS Orkney and NHS 24.

The State Hospital achieved a 73% response rate on the paper survey, coming close to their online survey response rate of 79%. Golden Jubilee Foundation also achieve a high paper response rate (65%), similar to the Board’s online response rate of 68%.

Case Study 4: Migrating from paper surveys

The State Hospital made a concerted effort in 2019 to reduce the number of paper surveys used particularly in teams where paper was the majority response method. 

Discussions between the iMatter Board Administrator and iMatter Team Leaders resulted in additional support being given to staff to set up valid e-mail accounts before the survey went live. Time was then allocated for staff to visit the Learning Centre where they were given the opportunity to use a PC to complete their e-mail survey.

Case Study 5: Migrating from paper surveys

The Golden Jubilee Foundation; “Based on previous response rates from paper questionnaires, the Op Lead had contacted all those areas who had previously opted for paper questionnaires over the last few years.  Recognising that many of the staff in these areas did not frequently access their work e-mail accounts, each of the managers were provided with the option to use personal e-mail addresses.  The managers communicated this to their teams and many staff opted to have the questionnaire sent electronically to their personal e-mails.  This in turn reduced the number of paper questionnaires sent.  The mangers within the departments who utilised the paper questionnaires this year continue to be committed to the continuous improvement model and have continued to encourage and engage with their teams which resulted in the increase in responses.”

The Future 

As noted, there has been a small reduction in the volume of paper questionnaires issued and several Boards have taken proactive steps to reduce their reliance on paper surveys. 

The Care at Home Services team from West Dunbartonshire HSCP took an innovative approach to reducing paper questionnaires. Key actions are highlighted below and their Team Story “Text for Success” provides full details.

Case Study 6: SMS Pilot

As a team of home carers, most had previously received paper questionnaires, but lacked engagement in iMatter. The team garnered support from NHS GG&C, the Scottish Government and Webropol to embark on a mobile technology supported pilot, that involved text messages to home carers informing them about iMatter and providing the link to the survey. Not only did this pilot generate an 85% iMatter response rate, but it has also served as a catalyst for using technology to enhance learning and communications across the team

“We are thrilled with the success of the project, and excited about the potential of using SMS technology to engage staff in the improvement process even more widely in the future.”

West Dunbartonshire HSCP, Care at Home Services. SMS Pilot

Looking to the future, the combination of encouraging staff towards the online questionnaire and use of technologies such as SMS will continue to reduce the volume of paper surveys and should increase the overall response rate.

Unusable Responses

A prerequisite of iMatter is that every question on the questionnaire is answered.  If there are any errors on the questionnaire then it is not processed.  

For the online questionnaire this is monitored within the script and so non-completion is not an issue. However, in 2019 there were 6,630 recipients (4% of the total online survey volume) who clicked on the survey invitation but did not then finish the survey. This is an improvement on 2018, when 8,625 did not complete the online survey. In addition, 1,362 people reached the end of the survey but did not hit the “Submit” button and so their surveys are not included in the results. Some of these people may have made a conscious decision not to submit their responses, but others may simply not have realised that they needed to ‘submit’ their response for it to be processed. Some design enhancements are under consideration for next year to reduce the volume who accidentally miss the ‘Submit’ button.

On the paper questionnaire it is not possible to monitor responses as they are being completed and so it is only when returned questionnaires are processed that incomplete or incorrectly completed responses are identified. 

Of the 25,464 paper surveys issued, 9,413 (37%) were input and a total of 2,278 (9%) were rejected for the reasons shown here. 

Of the partial responses 79% were only missing a response to one question.

Health and Social Care 2019	Volume	% of paper surveys sent
Paper Surveys Sent	25,464
Responses Processed	9,413	37%
Responses Rejected	2,278	9%
Reasons for Rejection
Partial Response	1,587	6%
Completion Errors	282	1%
Duplicate	167	1%
Past Deadline	242	1%

Summary

Overall the response rate has increased 3 percentage points from last year to 62% meaning the reporting threshold of 60% has been achieved. This increase is reflected across most of the Boards, with only three having a lower response rate in 2019 than 2018. Of those, NHS Orkney saw a considerable drop, but there are mitigating circumstances with a major hospital relocation impacting all staff during the iMatter fieldwork period.

The increase in response rates has occurred in both online and paper surveys, though there has also been a small reduction in the share of paper surveys issued.

Several Boards have shown considerable increases in their response rates and as the Case Studies show those improvements are as a result of concerted effort across the whole iMatter process, raising awareness and engaging with staff at all stages as well as senior management demonstrating their support throughout.