Scottish Cancer Patient Experience Survey 2024: technical report

Annex D – Weighting methodology

When conducting a survey, it is important to have a representative sample of the population you are interested in to reduce bias in the estimates produced. Applying weighting methods reduces the potential bias by making the results more representative of the population but also increases the variation in the estimates.

Survey weights are numbers associated with the responses that specify the influence the various observations should have in the analysis. The final survey weight can be thought of as a measure of the number of population units represented by the response.

The first Scottish Cancer Patient Experience Survey (SCPES) was run in 2015 and the results for this survey were presented unweighted. Feedback from a UK Statistics Authority (UKSA) review of the Care Experience Survey publications prompted a review of the weighting methodologies employed by the other Care Experience Surveys in the programme in 2018. Following these reviews, it was agreed that weighting for SCPES should be implemented.

Weighting to account for non-response bias enables us to make the results more representative of the survey population in a statistically robust way that is consistent with the methodologies applied to other Care Experience Surveys.

Each respondent has one weight. This differs slightly to the methodology applied to the other Care Experience Surveys which have had one weight per routed section. There is no administrative data available to estimate population totals for the routed sections in SCPES. Additionally, exploratory analysis suggests that using the national survey response to estimate population totals – as has been done for some routed sections in other Care Experience Surveys where population total estimates were not available from other sources – would not be appropriate here due to variation between areas.

The weight will be calculated to account for non-response by weighting up to the sample frame population based on age, sex and tumour group.

Trimming will be applied such that the maximum weight is calculated as:

Mean + (x * Standard Deviation)

where x is an integer such that less than 5% of respondents have their weight trimmed for each reporting level.

SCPES results are published at National, Network (of treatment and of residence), NHS Board (of treatment and of residence) and Cancer Centre levels. The weighting strategy described has been applied to all levels except Cancer Centre level, which are presented unweighted. The following groupings will be used at each reporting level:

• National – Two sex groupings (male and female), three age bands (16 to 64, 65 to 74 and 75+) and twelve tumour groups (Breast; Colorectal / Lower Gastrointestinal; Gynaecological; Haematological; Head & Neck; Lung; Prostate; Skin; Upper Gastrointestinal; Urological; Other, including Brain / Central Nervous System and Sarcoma; and Tumour Group Unknown).
• Network – Two sex groupings (male and female), three age bands (16 to 64, 65 to 74 and 75+) and twelve tumour groups (as for National level).
• NHS Board – Two sex groupings (male and female), two age bands (16 to 64 and 65+) and twelve tumour groups (as for National level).

In the 2024 CPES, the Cancer groups were reviewed and updated to reflect the interest in less survivable cancers in the 2023 to 2033 Cancer Strategy for Scotland. The tumour groups used to calculate the weights were not changed and are consistent with those used in 2018 to ensure methodological consistency over time and so that the overall survey results are comparable with previous rounds of the survey.