NHSScotland Staff Survey 2015 National Report

Appendix A: Background

1 Types of survey questions

There were various types of questions used in the NHSScotland 2015 Staff Survey questionnaire. This section describes the types of questions that were included and highlights any important information that the reader should be aware of in relation to each question type.

1.1 'Top‐level' and 'sub' questions

'Top‐level' questions are defined as questions to which all respondents were expected to provide an answer.

Example:
Q5‐5 During the past 12 months while working for your health board have you experienced bullying/harassment from your manager?

'Sub‐questions' are questions that respondents were expected to answer only where relevant to a previous response.

Example:
Q5‐6 Did you report the bullying / harassment you experienced?

Note that the number of responses received for sub‐questions will always be much lower than that received for top‐level questions as only a sub‐set of respondents will have provided an answer to each sub‐ question.

1.2 Attitudinal questions on a five point scale

Many of the survey questions were designed to capture the views of respondents in relation to a particular statement. These questions were all phrased positively and invited participants to respond on a scale between one and five, one being the most positive response and five being the least positive.

Example:

1.3 Attitudinal questions on a six point scale

Two additional questions have been included in 2015 with a six point scale. These questions have been taken directly from iMatter which will allow for some national benchmarking across Health Boards. These are:

'Q7 I have confidence and trust in my direct line manager'
'Q8 I feel senior managers responsible for the wider organisation are sufficiently visible'

These questions, similar to the five point scale, were phrased positively and invited participants to respond on a scale between one and six, one being the most positive response and six being the least positive.

Example:

As with previous surveys, the main unit of measurement is the percentage of staff who answered positively to each question. For example, for questions where respondents were asked to indicate their level of agreement or disagreement, responses of strongly agree and agree were considered positive. Full details of the response categories are shown below:

For each attitudinal question, the percentage of respondents who selected each response option is reported along with the total percentage of positive responses. Note that it is important to look at these detailed results, across all the response categories, when interpreting the survey findings.

1.4 Non‐scale questions

There were a number of non‐scale questions in the survey where respondents were invited to respond either 'Yes' or 'No'. Where the question was positive, 'Yes' was considered to be a positive response.

Example:
Q2‐4 Did you agree a Personal Development Plan (PDP) or equivalent?

Please note that there were seven questions in the survey where the question was negative (Q4‐4, Q4‐5, Q5‐5, Q5‐6, Q5‐10, Q5‐11 and Q5‐12). For these questions, 'No' was considered to be a positive response.

Example:
Q5‐5 During the past 12 months while working for your health board have you experienced bullying/harassment from your manager?

There were two non‐scale questions in the survey (Q2‐1 and Q5‐4) where there was more than one positive response option. In these cases, both positive responses were considered to be equally positive.

Example:

2 Calculation and reporting of results

This section describes how the results in the National Report were calculated.

2.1 Percentage of positive responses

As with previous surveys, the key unit of measurement provided throughout the report is the percentage of staff who responded 'positively' to each question.

For each question, the percentage of positive responses was calculated according to the number of respondents who provided a valid answer to that question. Respondents who did not provide a valid answer were excluded (e.g. no answer given, multiple answers on a paper questionnaire where a single a response is required, illegible written marks).

Example:

Total Number of Responses	Number of Valid Responses	Number of Positive Responses	% of Positive Responses
1,023	1,000	800	80%

The total number of valid responses received for each question is shown in the report.

For ease of reading, all percentages are reported to the nearest whole number. Please note that all reported differences between results are based on rounded results.

Example:

2015 Result (unrounded)	2014 Result (unrounded)	2015 Result (reported)	2014 Result (reported)	Difference (reported)
80.3	80.6	80	81	‐1%

Rounding percentages to the nearest whole number occasionally results in total percentages that do not add up to exactly 100% (in some charts / tables percentages may total 99% or 101%).

In order to ensure maximum accuracy, all formal statistical testing was performed on unrounded results.

2.2 Protecting the anonymity of respondents

The survey asked respondents to provide information relating to their employment (e.g. staff group, working pattern) and socio‐demographic profile (e.g. age, gender, ethnicity). This information has been used to present the results for different groups of staff in Appendix D and E of this report. The purpose of these appendices is to help highlight any variation in response amongst staff groups.

In order to preserve anonymity, sub‐groups of staff comprising of fewer than ten respondents have not been reported separately, however their responses have been included in overall NHSScotland and NHS Board results.

Where possible, small staff groups were amalgamated. For example, there were very few respondents who selected Student/Trainee as their staff group, therefore for the purposes of reporting this group has been merged with the 'other' staff group.

3 Guide to the National Report

The survey questionnaire was structured around the five key elements of the NHSScotland Staff Governance Standard.

The Staff Governance Standard requires all Boards to demonstrate that staff are:

1. Well Informed
2. Appropriately Trained and Developed
3. Involved in Decisions
4. Treated Fairly and Consistently, with Dignity and Respect in an Environment where Diversity is Valued
5. Provided with a Continuously Improving and Safe Working Environment Promoting the Health and Wellbeing of Staff, Patients and the Wider Community.

The results presented in the national report are set out according to these five elements, along with findings relating to the overall experience of working for NHSScotland.

There are also specific sections relating to unfair discrimination, bullying and harassment, and emotional / verbal or physical abuse. Broadly speaking, questions relating to unfair discrimination are aligned with the 'Treated Fairly and Consistently' Staff Governance strand, whereas questions relating to bullying and harassment and emotional / verbal or physical abuse are aligned with the 'Provided with a Continuously Improving and Safe Working Environment' strand.

3.1 Survey response

On page ten of the national report, you will find the percentage of staff who participated in the survey within each NHS Board in 2013, 2014 and 2015.

It is important to note that the results in the national report relate only to staff who responded to the survey; the views of these staff may not necessarily represent all staff employed by NHSScotland. If the response rate for a Board is particularly low, please bear this in mind when interpreting the results.

You should also refer to the 'Participant Profile' (Figure 13 of the national report) when considering response rate. This profile shows who responded to the survey within NHSScotland. If you know that around 40% of the staff within your Board are nurses, but only 20% of the survey respondents were nurses, then this group may be under‐represented in your survey results.

If any of the staff groups are under‐represented then it may be useful to investigate whether these staff groups face any barriers to participation that Boards could address in order to encourage participation in future surveys. It is found that there are barriers outwith the Board's control (e.g. staff objected to certain questions within the survey) it would be useful to feed this information back to the survey sponsors so that this can be considered in the design of future surveys.

3.2 Key findings and summary of results

On pages 12 ‐ 16 (section 6) of the national report, the main findings of the survey are summarised, presented under sub‐headings for each of the five Staff Governance Standard strands; and sub‐heading for the 'overall experience' section of the survey.

Pages 14 and 14 (section 6.1 and 6.2) present the five most and least positive results for NHSScotland. Figure 2 ranks the results of all the attitudinal questions from most to least positive: this will help you to begin to identify areas of strength or weakness within your survey results.

The information on these pages is shown in a summary format that could easily be used within a presentation for communicating results to staff. More detailed information is presented within the report using alternative formats that will be useful for other situations (e.g. presentations to specific groups or bodies).

Please note:
‐ Only attitudinal questions on a five‐point or six‐point scale are included in the key findings and Figure 2. Non‐scale questions with only two response options (Yes / No) tend to score the most positively / negatively in surveys, so their inclusion in this analysis would have skewed the results.
‐ There are some Yes / No type questions where one might expect the percentage of positive responses to be extremely high (e.g. discrimination, bullying & harassment); including these questions in the most and least positive results could be misleading.

3.3 Comparison to the 2014 Staff Survey

Figure 3 on page 20 of the report shows the change in the percentage of positive responses since the 2014 NHSScotland Staff Survey. The results are ranked from most to least improvement. This figure will help you to gauge where responses have most improved or deteriorated since 2014. Only the attitudinal questions on a five‐point scale are shown.

Comparisons with the 2014 results are colour coded according to whether the changes are statistically significant. The green bars on the graph represent a statistically significant positive change in response since the 2014 survey, whereas the red bars show a statistically significant negative change. Amber bars indicated that any change observed is not statistically significant.

Note that the staff who responded to the 2014 survey may not be the same staff who responded in 2015. Where there is a notable difference in the Participant Profile (Figure 13 of the national report) in 2015 compared to 2014, this will be highlighted in the summary on page 8 of the report.

Question Block 8 Q1 has been revised to help improve the quality of data collected by ensuring staff understand the question better. The response options have been reduced to 6 for 2015. Staff will indicate whether they are full time or part time; full time (shift worker) or part time (shift worker); Sessional or they prefer not to say.

Two additional questions have been included (Question Block 6 Q7 and Q8). These questions have been taken directly from iMatter which will allow for some national benchmarking across Health Boards.

'Q7 I have confidence and trust in my direct line manager'
'Q8 I feel senior managers responsible for the wider organisation are sufficiently visible'

There may be instances in Figure 3 where, for example, a 1% difference for one question is shown to be significant, but a 2% difference for another question is not. While this may seem strange there are a number of reasons why this may occur; the most likely reason relates to the fact that, while all percentages and percentage differences are reported to the nearest whole number, all statistical testing was performed on unrounded results. The statistical significance calculations are also affected by the number of people who provided a valid answer to each question and by results which approach the extremes (e.g. the percentage of positive responses is >90% or <10%).

3.4 Identifying opportunity for improvement

Section 6.5 presents a 'RAG' (Red, Amber, Green) analysis of survey findings in 2015 and 2014. Pages 21 and 22 present an explanation of how to read and interpret the RAG table (Table 1) which is presented on pages 23 and 24.

This table concisely presents a lot of useful data from the current and previous survey, ordered from the most to least positive responses in 2015. It therefore allows the reader to see the change in response between years, upon which statistical significance testing has been performed using the 95% confidence interval. A summary count is provided of the number of statistically significant improvements, statistically significant deteriorations, and questions for which there was no significant change between 2014 and 2015.

The red/amber/green colour coding is designed to act as a guide to interpreting the results and to help to identify areas of strength and areas that present opportunities for improvement. The parameters for the 'red', 'amber' and 'green' colour coding have been set by Capita for this survey in line with the parameters used in our analysis of the results for other public sector staff surveys.

While Capita tends to use a numeric scoring system to analyse scale questions, to assist the reader the colour coding for this survey is determined by the overall 'positive' percentage (in line with other forms of analysis presented in this report). When using a 5‐point scale the cut‐off for 'areas of strength' is usually questions or statements generating an average score of over 4.00 on a scale of 1 to 5. Analysis of the national data led to converting the threshold to questions or statements where 65% or more agree or tend to agree.

Similarly, the cut‐off for 'areas for improvement' is usually questions or statements generating a score below 3.00 (mid‐point on the scale from 1 to 5) and this is converted into 40% or fewer staff agreeing.

It should be remembered that this colour coding is simply a guide, and that your knowledge of NHSScotland, its workforce and local challenges will also influence which factors you feel are priorities for implementing actions aimed at improving future staff survey perceptions.

3.5 Main results charts

On pages 25 to 40 you will find the results of the survey (Figures 4 to 11), set out according to the five strands of the Staff Governance Standard. Figure 12, on page 42 and 43, presents findings from the set of questions about overall experience of working in NHSScotland.

All the main results charts (for scale response questions) are structured in a similar way:

1. Question: The text of the question that was asked.

2. Response chart: The percentage of respondents who selected each response option for this question, rounded to the nearest whole percentage, is presented in a stacked bar chart. It is important to look at this detailed breakdown, not just the percentage of positive responses, when interpreting the survey results. Note that the rounding percentages occasionally results in the total percentage not adding up to exactly 100%.

3. Total Responses: The number of valid responses that were received for this question.

4. % Positive 2015: The total percentage of positive responses that were received for this question in 2015. Note that the sum of all the positive responses shown in the bar chart will occasionally not equal the figure shown under the '% Positive 2015' heading. This is related to the rounding of results and is not an error.

5. Change from 2014: The change in the percentage of positive responses to this question since the 2014 NHSScotland Staff Survey. This data may be presented as a positive (+) or negative (‐) percentage according to whether the overall percentage of positive responses was higher or lower than in 2014.

On pages 34, 38 and 40 of the report, the reasons that respondents gave for not reporting unfair discrimination, bullying / harassment and physical violence or emotional / verbal abuse are shown. Respondents were asked to indicate whether they agreed or disagreed with each of the reasons for non‐ reporting that were listed in the survey. Note that respondents may have agreed with all, some or none of the reasons listed.