Cancer strategy 2023-2033 and cancer action plan 2023-2026: monitoring and evaluation framework - August 2023

Appendix 2: Metadata and data development for key headline indicators

Strategic Vision: By 2033 we will improve cancer survival

Indicator(s): Estimates of overall survival and age-standardised net survival at 1-year and 5-years; Age-adjusted mortality rates to capture changes in both incidence (rates) and net survival.

Method(s) of data collection: Extraction of cancer registration data (PHS), and population and deaths data (NRS).

Data source(s): Cancer registration (Scottish Cancer Registry) and NRS population and deaths datasets. This technical report on Cancer Survival in Scotland shows technical documentation including methodology, data sources and clinical coding information. This update on Cancer Mortality in Scotland, Appendix 1 explains age-adjusted mortality rates.

Data definitions (e.g. numerators, denominators, standardisation):

Cancer - Survival.

An individual patient's survival time is the time from date of diagnosis to date of death or censoring. Censoring is when a patient was lost to follow-up and their last known vital status was 'alive' (censoring may occur due to embarkation from Scotland or because the patient remained alive at the time analysis was performed, so-called 'administrative censoring'). Survival analysis accounts for the fact we do not know the (eventual) survival times of these censored patients. Survival analysis estimates parameters from the distribution of survival times. Overall survival is an estimate of the probability a patient will be alive at a given time after diagnosis. It is an estimate of survival from all causes of death, including non-cancer causes. Net survival is an estimate of the probability a patient will be alive at a given time after diagnosis, after making an adjustment for the impact of non-cancer causes of death. It is a useful measure for comparing cancer survival between populations, sub-population groups, and time periods, between which 'background' non-cancer mortality rates may differ. Overall and net survival are usually expressed as percentages between 0-100% and often interpreted as proportions. These measures are typically presented in both non-standardised and age-standardised forms. Age-standardisation is used to adjust for the effect of any differences in the age profiles of the populations being compared.

Cancer - Mortality.

Age-adjusted mortality rates; based on the number of death registrations in each of the calendar years, the following rates are calculated for cancer mortality. Crude rate: The total number of people with an illness (or who die) in a country or region, divided by the total population of that country or region, and is normally expressed 'per 1,000', 'per 10,000' or 'per 100,000'. Making comparisons on the crude rate can be misleading if the age structures of the populations of the countries or regions are quite different. Areas with larger percentages of younger people are unlikely to have as high levels of death as areas with larger percentages of older people – and therefore if there is no adjustment for these differences the wrong conclusion may be drawn about the health of an area simply because of the age-structure of the population. European Age-Sex Standardised Rates (EASRs) allow us to make comparisons between different geographical areas as they allow the effects of having different age structures in either the same population over time or different geographies to be removed. European Age-Sex Standardised Rate (EASR) uses European Standard Population (ESP) 2013 for each 5-year age group, the crude rate is calculated and then the weighted average of all age groups is taken based on the weightings of the 2013 ESP, to give the overall EASR.

Baseline/ comparability across time:

Cancer - Survival.

Estimates are best compared within studies to ensure the same methods are applied to different groups/time periods. PHS survival reports typically focus on new registrations but include a time series of cohorts to ensure methodological updates are applied to previous data. Survival is typically reported for single-year and five-year cohorts (to ensure sufficient cohort sizes for robust estimates). The pre-pandemic cohorts of 2019 and 2015-2019 provide natural baselines. Underdiagnosis and delayed diagnosis due to the pandemic will complicate analysis and interpretation for 2020 and 2021 data.

Cancer - Mortality.

Reports data since 1995. NRS moved from the World Health Organisation International Classification of Disease (ICD) version ICD-9 to ICD-10 in 2000. ICD codes have been back-mapped to 1995 as accurately as possible for continuity of reporting. Comparisons across the UK are produced by Cancer Research UK, and the most recent mortality statistics can be found on their Cancer Statistics for the UK page.

Comparison of Scottish and UK cancer data to that of other countries is a complex process because of the wide variation in data collection and coding practices, as well as variation in the quality and completeness of data. The International Agency for Research on Cancer maintain an online resource, the Global Cancer Observatory, that is searchable for comparative data. It may be misleading to focus too much attention on any apparent changes in mortality between 2020 and 2021; it is more informative to examine trends observed over a number of years. Striking changes from one year to the next may occur in the case of rare cancers, but these are likely to reflect random fluctuation caused by small numbers of cases - in such cases, it is even more important to examine mortality rates for a number of years aggregated together, rather than focusing on a single year.

Collection frequency and details (including time lag): To be confirmed.

Publication: To be confirmed.

Data breakdown:

Cancer - Survival.

Data can be broken down by sex, age group, and Scottish Index of Multiple Deprivation (SIMD). Estimates of overall survival and age-standardised net survival at 1-year and 5-years are available for most cancer groups by sex and age group for the period 2015-19, published in 2022. (Note a modified update focussing on the impact of the pandemic, for periods 2018-19 and 2020, was published in May 2023). Estimates of overall and age-standardised net survival at 1-year and 5-years are available for the period 2013-17 by sex and SIMD (for cancers with sufficient cohort sizes), last published in 2021. PHS will explore whether it is possible to estimate net survival by stage at 1-year and 5-years.

Cancer - Mortality.

Data can be broken by sex, age group, and SIMD. European and world age-standardised, as well as crude rates, are available for 45 different cancer types, broken down by sex and 5-year age group from 1995-2021.

Robustness and data limitations:

Cancer - Survival.

Overall survival and net survival estimates are not reported if the population at risk is too small. Age-standardised estimates are not reported if survival could not be estimated robustly for the age-specific groups required for their calculation. Age-standardisation is first attempted with five age groups, but, if any of these cohorts are too small, age groups may be merged to form four age groups. The COVID-19 pandemic has had unusual and complex impacts on cancer registration data, including through under-diagnosis and delayed diagnosis of cancers. Understanding these impacts will require careful analysis and interpretation.

Cancer - Mortality.

Registry data are subject to validation at data entry and quality assurance procedures. See the Cancer Information FAQs. Reported data are compared to previous years' figures and to expected trends. At time of extraction, data for the most recent year are estimated to be complete.

Outcome: Reduced relative population burden of disease

Indicator(s): Burden of disease (disability adjusted life years) relative to all causes of other disease, infection and injury in the overall population.

Method(s) of data collection: Extraction of electronic health records (Scottish Morbidity Records and National Records of Scotland Vital Events).

Data source: Scottish Burden of Disease (SBoD) study dataset. Technical documentation, including methodology, data sources and clinical coding information, is published by the Scottish Public Health Observatory.

Baseline/ comparability across time: Baseline year 2019; future data points will be comparable with baseline.

Collection frequency and details (including time lag): Data estimation and publication cycle to be confirmed.

How data can be broken down: Data available by sex and 5-year age group. Data may be available by SIMD quintile, if deemed suitable following statistical disclosure control.

Robustness and data limitations: In order to provide a measure of accuracy and relevance of the estimated disease disability adjusted life years (DALYs) to users, a measure of data quality has been developed for the SBoD study. This measure assigns a RAG (Red; Amber; Green) status to each disease or injury indicative of the accuracy and relevance of the estimates. Estimates of cancer burden are classed as Green: highly accurate and relevant. This indicates that the estimates have been derived using relevant and robust data sources with only a small degree of adjustments performed to the input data. Success in areas such as cancer survival may increase the measured non-fatal burden of disease: more screening and earlier diagnosis may increase the incidence of cancer, and better survival may increase the prevalence of cancer.Outcome: Reduced later stage diagnosis

Indicator(s): Diagnosis at disease stages III and IV (incidence by stage for 16 cancers).

Method(s) of data collection: Extraction of Cancer registration data (PHS).

Data source: Cancer registrations. Cancer Incidence in Scotland data up to 2021 is published by PHS.

Data definitions (e.g. numerators, denominators, standardisation): Data as per 2021 incidence publication released in March 2023 contains the number of diagnoses at stages I, II, III, IV, and unknown for the 16 most common cancer types diagnosed in Scotland.

Baseline/ comparability across time: Baseline year 2021; future data points will be comparable with baseline.

Collection frequency and details (including time lag): Incidence publication approximately each April with a table included on numbers and percentages for each stage by deprivation, in line with the earlier diagnosis vision.

Data breakdown: Data available by sex, stage and SIMD quintile.

Robustness and data limitations: The earlier diagnosis vision is for later stage disease (stages III and IV) to be reduced. A focus will remain on reducing the health inequality gap, particularly those from areas of deprivation. This vision currently looks at all tumour groups combined but there will be differences across groups in relation to the level of reduction. It is recognised that not all cancers can be conventionally staged so additional measurements, such as emergency presentations, will be required to track progress and improvements in other cancer types, including blood and neurological cancers. Outcome: Timely access to treatment

Indicator(s): Cancer waiting times.

Method(s) of data collection: Cancer waiting times (CWT) statistics – adjusted and unadjusted waits (PHS).

Data source: National Cancer Waiting Times Data.

Data definitions (e.g. numerators, denominators, standardisation):

Cancer Waiting Times - 62-day standard.

Numerator = The number of patients receiving their first treatment within 62 days of the Board receiving the urgent suspicion of cancer (USC) referral.

Denominator = The number of eligible referrals made under the performance standard.

Cancer Waiting Times - 31-day standard.

Numerator = The number of patients receiving their first treatment within 31-days of a decision to treat.

Denominator = The number of eligible referrals made under the performance standard.

Note: See the CWT Data & Definitions Manual for further detail on how the numerator and denominator are defined for both standards depending on the standard, source of referral and type of first treatment.

National Cancer Waiting Times Data: 95% of all eligible patients should wait no longer than 31 or 62 days for cancer treatment.

Baseline/ comparability across time: To remain relevant to the changing set of targets (as published in the Scottish Government's Better Cancer Care, An Action Plan in 2008), the cancer waiting times statistics published previously by PHS were replaced with a new series of figures. The first set of these new figures relating to these targets were published in June 2010. Performance against these targets was achieved by March 2011, the timescale agreed by the Scottish Government. These targets were considered as National Standards from April 2012 and continue to be published on a quarterly basis.

Collection frequency and details (including time lag): CWT data is submitted monthly, based on the previous month of treatment, and quarterly.

Cancer Waiting Times - Monthly Submissions.

This is for Performance Management purposes only. These data are submitted monthly and are based on patients treated within a specified monthly time period.

Cancer Waiting Times - Quarterly Submissions.

This is for publication purposes and submitted quarterly. These data are based on patients treated within a three-month time period.

Data breakdown: See CWT Data & Definitions manual for all variables collected and ways data can be broken down. As these are patient level data, they can also be linked by postcode to derive SIMD.

Robustness and data limitations: The quality of these statistics is considered fit for publication; data quality aspects are described within each publication. Fit for Purpose (FFP) exercises have been carried out by PHS for the 62-day performance and have shown that completeness of the 62-day cohort is within an acceptable range and is fit for publication. Case ascertainment is assessed each quarter for the 31-day standard. The latest figures can be found within Table 1 in the list of tables on the report publication page. PHS regularly carries out data quality exercises to ensure that data are recorded in an accurate and consistent manner across NHS Scotland. Information on these exercises can be found on the PHS website. In early 2012, PHS Cancer Waiting Times undertook a data quality project to assure that data submitted for Bowel Screening patients are recorded accurately and consistently. The Data Quality Assurance findings from this project were published by the former NHS Scotland Information Services Division (ISD). Responsibility for collating and submitting the data to PHS lies with the NHS Board that received the patient's initial referral to secondary care. Information on data quality, service issues and accuracy specific to this publication can be found in Appendix 2 of the CWTs quarterly report. The most recent such report is Cancer Waiting Times in NHS Scotland: 1 January - 31 March 2023.

The Data Quality Assurance team within PHS carry out data quality exercises on cancer waiting times data. Completeness: patients will only be included in the database if they have a valid Community Health Index (CHI) number. A patient will be excluded from reporting against the Cancer Waiting Times standards for the following reasons:
1. The patient chooses to have any part of their pathway outwith NHS Scotland. If this is before the decision to treat, they will be excluded from the 62-day standard; if after the decision to treat, they will be excluded from both standards.
2. The patient died before treatment.
3. The patient refused all treatment.
4. The patient was deemed a clinically complex case by the lead cancer clinician of the responsible NHS Board.

Outcome: More people receiving curative treatment

Indicator(s): In development.

Data development: Data development is an action in the first plan. Number of curative treatments recorded in national Systemic Anti-Cancer Therapy (SACT) data could be a potential indicator to provide some relative assessment of progress. Radiotherapy data are also available. Work will be undertaken by PHS to assess the feasibility of measuring surgical treatments from acute care data and multidisciplinary team data.

Data source: The national SACT dataset which combines and standardises data from the five local instances of the Chemotherapy Electronic Prescribing and Administration Systems (CEPAS) ChemoCare in Scotland.

Datadefinitions (e.g. numerators, denominators, standardisation): Number of patients receiving curative treatment out of all patients receiving treatment; increase in number of patients receiving treatment recorded as curative.

Baseline/ comparability across time: Data are considered to be complete from 2014 onward. Changes in the definition of the term 'curative' may influence data over time. A good baseline would be to reach national consensus on what should be considered curative and measure the first year post consensus as baseline.

Collection frequency and details (including time lag): Annual.

Data breakdown: Data can be broken down by geography, tumour type.

Robustness and data limitations: The National SACT dataset is still being validated by PHS and local analysts. An ongoing quality assurance process has been put in place and there is a continuous evaluation process of which variables are fit for reporting. Treatment intent is currently not considered to be of robust enough quality for reporting, however, quality has improved over time and awareness has been raised with prescribers in the preparation of the SACT 30-day mortality publication.

Vision: Excellent care

Outcome: Improved experience of services, across all areas of care

Indicator(s): In development.

Data development: Measurement will be defined and developed during the first action plan. A new Scottish Cancer Patient Experience Survey (SCPES) will be completed in 2024 and will be used as the basis for measuring experience of services and care. Exact indicators (specific question responses) will be agreed at this time. Examples could include overall experience, travelling to appointments, emotional and psychological support received, or receiving adequate information. Looking ahead, either a SCPES or a similar survey tool will be repeated during the next Cancer Action Plan. Other potential sources of data on experience include Care Opinion, and Patient Reported Experience Measures (PREMs) collected as part of specific evaluation activities.

Data source: SCPES or equivalent survey dataset.

Datadefinitions (e.g. numerators, denominators, standardisation): Respondents are individuals aged 16 or over, who had an inpatient hospital record with a mention of cancer and a confirmed cancer diagnosis within a specific timeframe.

Baseline/ comparability across time: Baseline year 2024; we will be looking back at the 2015 and 2018 surveys when we report in 2024 to understand progress/ areas for focused improvement.

Collection frequency and details (including time lag): Ad hoc, every 3-5 years with approximately 1 year time lag.

Data breakdown: Data can be broken down by age group, sex, sexual orientation, ethnicity, SIMD and rurality.

Robustness and data limitations: Some small numbers when data broken down, e.g. ethnicity. Outcome: Optimised quality of life for each individual

Indicator(s): In development.

Data development: Measurement will be defined and developed during the first action plan. Potential methods include building on available data such as Euro-QoL (EQ-5D), pulling data nationally from Holistic Needs Assessments.

Data source: Survey results, Holistic Needs Assessment reports, Patient Reported Outcome Measures (PROMs).

Data definitions (e.g. numerators, denominators, standardisation): Respondents will potentially be individuals aged 16 or over and with a confirmed cancer diagnosis.

Baseline/ comparability across time: Will depend on appropriate source. Unlikely to have comprehensive baseline.

Collection frequency and details (including time lag): Ad hoc. Potential time lag depending on agreed methodology.

Data breakdown: Some disaggregation likely. Will depend on agreed methodology.

Robustness and data limitations: Will depend on agreed methodology. Given there is no single standalone tool, there are likely to be limitations.

Outcome: Embedded research, innovation and data capture in all services

Indicator(s): In development.

Data development: Measurement will be defined and developed during the first action plan. Options include looking at access to clinical trials using data from the EDGE clinical research management system; or measuring the range of data available on the PHS Cancer Intelligence Platform. This work will build on recommendations by the Equity of Access Short Life Working Group.

Data source: Potentially EDGE; PHS dashboards.

Data definitions (e.g. numerators, denominators, standardisation): Will depend on measure(s) and methodologies chosen. Clinical trials' data will potentially include number of trials and number of participants in trials.

Baseline/ comparability across time: The baseline will depend on the indicators and methodologies chosen: baseline is likely to be 2023. Comparative data should be available going forward.

Collection frequency and details (including time lag): Annual, minimal time lag.

Data breakdown: The ability to provide breakdowns will be considered when determining suitable indicators.

Robustness and data limitations: There is no current standardised measure for this outcome.

Vision: Equitably accessible care

Cross-cutting aim: Reduced health inequalities in all areas above

Indicator(s): In line with this cross-cutting aim, wherever possible, we will monitor data broken down by equalities, socioeconomic and geographic characteristics. This will include analysis of: sex and age group, SIMD quintile, and geography if possible, subject to statistical disclosure control. The data breakdowns that are currently possible for each key headline indicator are provided above. In addition to monitoring equalities data for the headline indicators specified above, we will monitor cancer incidence amongst those aged under 75 years and cancer deaths for those aged 45-74 years. This will help us to understand absolute and relative inequalities between the most and least deprived areas in Scotland. See Appendix 3 for a summary of planned strategic work.

Data source: Annual updates of the long-term monitoring of health inequalities headline indicators are published by the Scottish Government. Scottish Cancer Registry (SCR) and Public Health Scotland. Technical documentation about the methodology of long-term monitoring of health inequalities has been published by the Scottish Government.

Data definitions (e.g. numerators, denominators, standardisation): European age-standardised rates of new cases of cancer amongst those aged under 75 years. European age-standardised rates of deaths from cancer amongst those aged 45-74 years. The relative index of inequality (RII) indicates the extent to which health outcomes are worse in the most deprived areas compared to the average throughout Scotland. It looks only at the income and employment domains of the SIMD, called the Income Employment Index (IEI). Absolute inequalities are measured by looking at changes in the gap between those living in most and least deprived areas in Scotland. It is possible for absolute inequalities to improve, but relative inequalities to worsen. Rates are age-standardised in order to show patterns over time on a consistent basis, taking account of changes in the age distribution of the Scottish population, therefore more clearly showing any underlying trend.

Baseline/ comparability across time: Baseline year 2020 for cancer incidence data. Baseline year 2021 for cancer deaths; future data points will be comparable with baseline.

Collection frequency and details (including time lag): Annual.

Data breakdown:

Cancer - Incidence Rate aged under 75 years.

All Cancers - cancer defined as all malignant neoplasms excluding non-melanoma skin cancer. The following ICD coding was used: ICD10 'C00-C96' excluding 'C44' (the Scottish Cancer Registry does not use code 'C97').

Prostate cancer (males only) - ICD-10 C61

Breast cancer (females only) - ICD-10 C50

Cancer of the trachea, bronchus and lung- ICD-10 C33-C34

Colorectal cancer- ICD-10 C18-C20

Cancer - Deaths aged 45-74 years.

All cancers - cancer defined as all malignant neoplasms excluding non-melanoma skin cancer. The following coding was used: ICD10 (2000 onwards) 'C00-C97' excluding 'C44'.

Prostate cancer (males only) - ICD-10 C61

Breast cancer (females only) - ICD-10 C50

Cancer of the trachea, bronchus and lung- ICD-10 C33-C34

Colorectal cancer- ICD-10 C18-C20

Robustness and data limitations: Aggregate data are provided by PHS for cancer incidence and cancer deaths. Scottish Government statisticians carry out quality assurance checks on the aggregate data, comparing it with past trends and against other published data, such as national level data published by NRS or PHS.