Scottish Health and Care Experience Survey 2015/16 - Technical Report

8 Analysis and Reporting

Introduction to analysis

8.1 The survey data collected and coded by Quality Health Ltd were securely transferred to ISD Scotland, where the information was analysed using the statistical software package SPSS version 21.0

Reporting Patient Gender

8.2 Analysis of survey response rates by gender was done using the gender of the sampled patients, according to their CHI record at the time of data extraction (20 October 2015).

8.3 For all other analyses by gender, where survey respondents had reported a valid gender in response to question 46, this information has been used in reporting. Where the respondents did not answer the question or gave an invalid response, gender information from the sampled patient's CHI record was used.

8.4 In total, 109,326 responders (98 per cent) provided a valid response to the question on gender (question 46). Of these, there was a difference between self-reported gender of the respondent and the gender of the originally sampled patient in 1,408 cases (1.3 per cent). Amongst this group it was more frequently the case that a survey questionnaire originally sent to a male was responded to by a female (n = 865), than it was that a questionnaire sent to a female was answered by a male (n = 543). As practice contact rates are generally higher in females than males, one possible reason for this is that some male survey recipients may not have been to their practice in the past 12 months and passed their questionnaire to a female member of their household.

Reporting patient age

8.5 Analysis of survey response rates by age was done using the age of the sampled patients, according to their CHI record at the time of data extraction.

8.6 For all other analyses by age where survey respondents had reported a valid age in response to question 47, this information has been used in reporting. Where the respondents did not answer the question or gave an invalid response, age information from the sampled patient's CHI record was used.

8.7 Valid age was taken to be anything between 17 and 108 years. A small proportion of cases where age was reported as less than 17 were treated as invalid responses to the question, although it is likely that in at least some of these instances the respondents were giving their feedback about their experience at the practice when making an appointment for their child, and in doing so reported the child's age rather than their own.

8.8 In total, 108,146 responders (97 per cent) provided a valid response to the question on age at last birthday (question 47). Of these, the self-reported age and the age calculated from the CHI record differed by two or more years in 2,149 cases (2 per cent). In a further 8,403 cases (8 per cent) there was a difference of one year. This is not unexpected, however, as many recipients would have had a birthday between 20 October 2015 and the date they responded to their questionnaire (November 2015 - February 2016).

8.9 In many instances where the age calculated from the CHI record differed from the age reported by the survey respondents, the associated age group used in the national report remained the same, whether based on CHI or based on the survey response. In 2,998 cases the record was however counted under a different age group for response rate analysis to the one used for all other analyses. Of these, 2,527 (84 per cent) were in an older group for the main analysis of results than for analysis of response rates. Some of this relates to individual recipients having a birthday and "moving up" by a single age group. In other instances this reflects the respondent being a different individual to the person sent the questionnaire and being more likely to be somewhat older than the originally sampled patient; older people were more likely to respond to the survey than younger people.

Table 11 Where reported age and CHI age groups are different

Age group derived from survey responses (Oct 2015 - Feb 2016)	Age group derived from CHI records as at 20 Oct 2015
	17 - 34	35 - 49	50 - 64	65 and over	Total
17 - 34	0	87	41	39	167
35 - 49	121	0	98	60	279
50 - 64	184	196	0	146	526
65 +	255	591	1,180	0	2,026
Total	560	874	1,319	245	2,998

Reporting deprivation and urban/rural status

8.10 Patient postcodes were used to match records to deprivation and urban/rural status information as defined by the Scottish Government. The versions used were:-

• The Scottish Government urban rural classification 2011/12. Further information on the classification is published at www.gov.scot/Topics/Statistics/About/Methodology/Geography

• The Scottish Index of Multiple Deprivation 2012. Further information on the index is published at www.gov.scot/Topics/Statistics/SIMD

8.11 A small minority of records were not matched to deprivation or urban/rural information, for example because the postcodes were not valid or recognised by the reference files used in the matching. Table 12 below shows the numbers and percentages of records that were not assigned to a deprivation or urban/rural category.

Table 12 Patients that could not be assigned urban/rural or deprivation categories

	No. of all responders	% of all responders	No. of sampled patients	% of sampled patients
Patient not assigned to a classification or quintile	211	0.2	1,989	0.3

Number of responses analysed

8.12 The number of responses that have been analysed for each question is often lower than the total number of responses received. This is because not all of the questionnaires that were returned could be included in the calculation of results for every individual question. In each case this was for one of the following reasons:-

The specific question did not apply to the respondent and so they did not answer it. For example if they did not see a nurse in the previous 12 months and therefore did not answer questions about their experience with the practice nurse(s)

The respondent did not answer the question for another reason (e.g. refused). Patients were advised that if they did not want to answer a specific question they should leave it blank

The respondent answered that they did not know or could not remember the answer to a particular question

The respondent gave an invalid response to the question, for example they ticked more than one box where only one answer could be accepted.

8.13 The number of responses that have been analysed nationally for each of the per cent positive questions are shown in Annex A: Per cent positive and negative results.

Weighting

8.14 Results at Scotland, NHS Board and Health and Social Care Partnership level are weighted. Weighted results were calculated by first weighting each GP Practice result for each question by the relative practice size. The weighted practice results were then added together to give an overall weighted percentage at Scotland, NHS Board and Health and Social Care Partnership level. The weight for each practice is calculated as the practice patient list size (of patients aged 17 or over and therefore eligible for being included in the sample survey) as a proportion of the entire population (Scotland, NHS Board or Health and Social Care Partnership) of patients eligible for inclusion in the survey.

8.15 Weighting the results in this way provides results more representative of the population (at Scotland, NHS Board or Health and Social Care Partnership level) than would be the case if all practices (small and large) were given equal weighting in the calculation of aggregation results.

Percentage positive and negative

8.16 Per cent or percentage positive is frequently used in the reporting. This means the percentage of people who answered in a positive way. For example, when patients were asked how helpful the receptionists are, if patients answered "Very helpful" or "Fairly helpful", these have been counted as positive answers. Similarly those patients who said they found the receptionist "Not very helpful" of "Not at all helpful" have been counted as negative. Annex A details which answers have been classed as positive and negative for each question.

8.17 Percentage positive is mainly used to allow easier comparison rather than reporting results on the five point scale that patients used to answer the questions. There is also a belief that differences between answers on a five point may be subjective. For example there may be little or no difference between a person who "strongly agrees" and one who "agrees" with a statement. In fact some people may never strongly agree or strongly disagree with any statements.

Outcomes of NHS treatment indicator

8.18 The Quality Strategy emphasises the importance of measurement, and a Quality Measurement Framework has been developed ^[3] in order to provide a structure for describing and aligning the wide range of measurement work with the Quality Ambitions and Outcomes. As part of this framework, 12 national Quality Outcome Indicators have been identified, which are intended to show national progress towards achievement of the Quality Ambitions.

8.19 One of these twelve Quality Outcome Indicators relates to Patient Reported Outcomes. This is reported in section 11 of the national report.

8.20 An average score is calculated for each respondent based on the outcomes questions they have answered. (Patients answering none of the 3 questions are not included.) These average scores are weighted by the number of patients registered at each GP practice to give scores for NHS Boards and Scotland.

8.21 The three outcomes questions and how the responses were scored are presented below.

• In the last 12 months, have you received NHS treatment or advice because of something that was affecting your ability to do your usual activities? …how would you describe the effect of the treatment on your ability to do your usual activities?

Table 13 Scores for outcomes for something affecting ability to undertake usual activities

Outcome	Score
I was able to go back to most of my usual activities	100
There was no change in my ability to do my usual activities	50
I was less able to do my usual activities	0
It is too soon to say	Don't include

• In the last 12 months, have you received NHS treatment or advice because of something that was causing you pain or discomfort

Table 14 Scores for outcomes for something causing pain or discomfort

Outcome	Score
It was better than before	100
It was about the same as before	50
It was worse than before	0
It is too soon to say	Don't include

• In the last 12 months, have you received NHS treatment or advice because of something that was making you feel depressed or anxious?

Table 15 Scores for outcomes for something making patients feel depressed or anxious

Outcome	Score
I felt less depressed or anxious than before	100
I felt about the same as before	50
I felt more depressed or anxious than before	0
It is too soon to say	Don't include

Quality assurance of the national report

8.22 A small group of Scottish Government policy leads were sent a draft version of the national report for quality assurance. Feedback included suggestions on ways in which to report data as well as comments about the context for the survey. These were taken into account in finalising the national report. In addition ISD Scotland carried out quality checks of all figures used in the report.

8.23 A statement on data quality for all of the patient experience surveys is available at www.gov.scot/Resource/0049/00493714.pdf

Revisions to previous publication

8.24 A copy of our revisions policy is available at: www.gov.scot/Resource/0049/00490163.pdf.

8.25 Some revisions have been made to the Health Board level results for the Quality Outcome Indicator on Care Experience for 2013/14, due to an error in the previous calculation. The size of the revisions are shown in the table below. The main impact was for NHS Orkney and NHS Western Isles. In particular, the latter was incorrectly presented as 'significantly worse' between 2013/14 and 2011/12, but there was in fact no statistically significant difference between the scores for the two years.

Table 16 Revisions to the Quality Outcome Indicator by NHS Board, 2013/14

	2013/14 previously published figure	2013/14 revised figure	Difference
NHS Ayrshire & Arran	74.0	74.1	-0.1
NHS Borders	78.3	78.2	0.1
NHS Dumfries & Galloway	75.7	75.9	-0.2
NHS Fife	74.9	74.8	0.1
NHS Forth Valley	74.9	75.1	-0.2
NHS Grampian	76.4	76.4	0.0
NHS Greater Glasgow and Clyde	74.3	74.3	0.0
NHS Highland	77.4	77.3	0.1
NHS Lanarkshire	73.3	73.4	-0.1
NHS Lothian	76.2	76.2	0.0
NHS Orkney	82.8	80.8	2.0
NHS Shetland	78.4	78.4	0.0
NHS Tayside	75.7	75.6	0.1
NHS Western Isles	75.3	76.1	-0.8
Scotland	75.2	75.2	0.0