Scottish Cancer Patient Experience Survey 2018: technical report

7. Analysis and Reporting

The survey data collected and coded by Quality Health Ltd were securely transferred to ISD Scotland, where the information was analysed using the statistical software package IBM SPSS version 24.0.

Reporting the Age of Respondents

Respondent date of birth was taken from their CHI record at the time of data extraction. This source was used for all stages of the analysis.  The age of respondents used for reporting purposes was calculated as at 1 August 2018.

Reporting the Sex and Gender of Respondents

Analysis of survey response rates by sex was undertaken using the sex of people in the sample according to their CHI record at the time of data extraction. This source was also used in the calculation of the survey weights (more information about this is provided later in this section).

All other analyses has been undertaken using gender, the respondents’ answer to question 62 “What best describes your gender?”. In total, 4,926 responders (99 per cent) provided a valid response to Q62.

Number of Responses Analysed

The number of responses that have been analysed for each question is often lower than the total number of survey responses received. This is because not all of the questionnaires that were returned could be included in the calculation of results for every individual question. In each case this was for one of the following reasons:

• The specific question did not apply to the respondent and so they did not answer it. For example, if they had not had an operation for their cancer and therefore did not answer questions about their experience of this.
• The respondent provided an answer indicating that a question did not apply to them. For example, if they did not need or want help from the third sector.
• The respondent did not answer the question for another reason (e.g. refused).  People were advised that if they did not want to answer a specific question they should leave it blank.
• The respondent answered that they did not know or could not remember the answer to a particular question.
• Responses may be removed following validation checks, for example if a respondent selected an invalid combination of responses. Improved validation checks were introduced for this survey to ensure consistency between online and paper responses. 

The number of responses that have been analysed nationally for each of the positive / negative questions are shown in Annex A.

Weighting

When conducting a survey, it is important to have a representative sample of the population you are interested in. Applying weighting methods reduces potential bias by making the results more representative of the population.

Survey weights are numbers associated with the responses that specify the influence the various observations should have in the analysis. The final survey weight associated with a particular response can be thought of as a measure of the number of population units represented by that response.

Results for the 2015 SCPES were presented unweighted, however, weighting was introduced for the 2018 survey to take account of any non-response bias in the survey and ensure responses were representative of the sample population. A paper setting out the details of this review and the weighting methodology applied to the 2018 results is available at www2.gov.scot/Resource/0054/00542173.pdf.  

Results at national, Regional Cancer Network and NHS Board levels have been weighted. Results at Cancer Centre level continue to be presented unweighted.

Backdating of Previous Surveys

Due to the methodological changes introduced for this survey, results from the 2015 survey have been backdated where appropriate to ensure comparisons over time are available. Reports specifically relating to the 2015 survey will not be updated to include the backdated figures.

As part of the backdating process, the improved validation checks brought in for the 2018 survey have been applied to the 2015 survey responses. This means the total number of responses analysed for some questions will differ from those previously published. 

Percentage Positive and Negative

Per cent or percentage positive is frequently used in reporting results from this survey. This means the percentage of people who answered in a positive way. Annex A details which answers have been classed as positive, neutral and negative for each question.

Percentage positive is mainly used to allow easier comparison, particularly where responses are on a scale where there may be multiple responses being classed as positive or negative. There is also a belief that differences between answers on scales with five or more points may be subjective.  For example there may be little or no difference between a person who “strongly agrees” and one who “agrees” with a statement. In fact some people may never strongly agree or strongly disagree with any statements.

As described in Section 4 of this report, these results are based on a sample of patients and are therefore affected by sampling error.  The effect of this sampling error is relatively small for the national estimates. However, when comparisons have been made in the analysis of the survey results, the effects of sampling error have been taken into account by the use of confidence intervals and tests for statistical significance. Only differences that are statistically significant are reported as differences within the analysis and all significance testing is carried out at the 5% level.

More information on confidence intervals, significance testing and how they’re calculated can be found at: www.gov.scot/Resource/0052/00522932.pdf. 

Quality Assurance of the National Report

A restricted number of colleagues at Macmillan Cancer Support and ISD were sent a draft version of the national report for quality assurance purposes. Feedback received was taken into account when finalising the national report. 

Revisions to previous publications

A copy of our revisions policy is available at: www.gov.scot/Resource/0052/00522934.pdf