Scottish Cancer Patient Experience Survey 2015/16: Analysis of Free-text Comments

Discussion

Analysis of the free-text comments from the Scottish Cancer Patient Experience Survey provided a detailed insight into the importance, to people with cancer, of 'feeling confident within the system' and 'being treated as an individual'. This is the first survey of Scottish cancer patients which has included open-ended comments, and has added valuable information to our understanding of the many positives, but also the issues which patients encounter during their cancer journey. In total 2,663 (55%) respondents left at least one comment in the survey.

While participants gave more positive than negative comments, the positive comments were often generic, and lacked detail. The English (Wagland et al, 2015), London (Wiseman et al, 2015) and Welsh (Bracher, Wagland and Corner, 2014) Cancer Patient Experience Surveys also found that positive comments tended to lack detail of the specifics of favourable experiences. However, despite the lack of detail, several distinct themes emerged from the analysis of the positive comments, with many patients describing a good experience overall, and particularly valuing clear information, speedy and efficient processes, good support, good clinical care and feeling that they could put trust in the system.

Negative comments provided a much greater insight into the specific problems participants had encountered during their cancer care. The proportion of negative to positive comments (0.78:1) was greater in this survey than in either the Welsh (0.60:1) or London (0.51:1) surveys, however, this may be because the Scottish survey, unlike the London and Welsh surveys, did not specifically ask 'what patients found good about their cancer care'. Quantitative analysis of the tick-box questions in the Scottish Cancer Patient Experience Survey (Scottish Government and Macmillan, 2016) found that overall perceptions of care were very positive, with 94% of patients rating their care as '7' or higher on a scale from 0 (very poor) to 10 (very good).

However, analysis of the free-text comments highlights issues that are not revealed in the numerical analysis and provides greater understanding of the significant issues which have impacted on some aspects of patients' care. Participants may use many different criteria to arrive at a number to rate their care, however, the detail allowed in free-text comments provides respondents with the opportunity to express criticism over particular aspects of care which may not be captured in a single numerical rating. It is a well-documented feature of patient experience surveys that quantitative measures tend to be rated higher than qualitative surveys, where participants have the option to describe their concerns with the care they received (Williams, 1994). This is a discrete report of the interpretation of the free-text comments and has not compared or related findings from the free-text comments with the numerical findings.

Analysis of the free-text comments was inductive, and creation of the sub-themes was entirely driven by the data. The subthemes built up a picture of two key aspects of experience that were critically important, and generated an understanding of what feeling 'confident' and 'being treated as an individual' appear to entail. They also illustrated how issues with structures and processes can impact negatively on patients' experiences. The importance of these themes is highlighted in both the negative and positive comments made by respondents.

The overwhelming impression from the free-text comments was that patients' experiences and impressions of cancer care were vulnerable to particular features of the way in which health care professionals communicated and the way in which health services were organised. If they were treated with kindness, and had clear expectations of what was going to happen to them, which were met, then they were positive about their care. However, if things did not go to plan, if they felt ignored, or if they started to doubt the competence of the people or system that was caring for them, they quickly became stressed and anxious and had a negative experience of care. The comments indicated that it was often aspects of the organisation of care which led patients to feel that nobody saw what was different about them, or treated them as a person with needs.

Responses to the free-text comment boxes in the survey gave a clear picture of the different aspects of care which influenced patients' feelings of confidence, or not feeling confident. Negative comments demonstrated that aspects of care at all points of cancer treatment could impact on feelings of security, from difficulties with being diagnosed with cancer, to inadequate aftercare. These negative incidents around quality and access to the system, affected patients' confidence in their cancer care.

The other major issue for participants was feeling that they were not being treated as an individual, with patients feeling isolated and alone within the system of care. Factors which contributed to patients feeling this way included poor communication, lack of involvement in decision making, lack of information and poor emotional support. The results of this survey highlight that patients wanted to feel that they were being treated respectfully and recognised as a person, yet instead in some cases felt depersonalised and alienated by the people who were caring for them.

Strengths and Limitations

The analysis of the free-text data within this survey followed a structured approach and was rigorous and thorough, reducing the risk of bias in analysis. Recognised techniques were used to structure the large data-set including using structural coding to allow analysis by comment box, and measuring code frequencies to give an indication of the prominence of different sub-themes (Namey et al, 2007). Reliability of the application of codes to raw data was monitored by an independent researcher. The large number of comments increased the likelihood that unusual circumstances would be captured in the data. The comment box questions were designed to ask patients about their experience rather than their satisfaction, asking participants to 'tell us' about their experience rather than to 'evaluate'. This removed the risks inherent in making assumptions about how patients evaluate satisfaction (Williams, 1994). The data was analysed by experienced qualitative researchers, independent from the NHS and Scottish Government, who had no preconceptions about the data or results.

A real strength of this survey was that free-text boxes were placed throughout the questionnaire, giving participants a chance to reflect and comment on different stages of the cancer journey. This enabled the analysis to focus on particular problems which arose at different points in the patients' treatment, and identified that different subthemes had prominence at different stages.

Although most of the issues within the subthemes were covered at some point in the tick-box questions in the survey, participants often brought up issues, unprompted, in the qualitative comments before they had arisen in the questionnaire. For example, although the first three comments boxes and their preceding quantitative questions didn't ask specific questions about continuity of care, or being treated as an individual, both came out strongly in participants' comments. The content of the comments boxes also reflected specific issues covered in the preceding questions. However, within the comments, participants gave much more depth and description about the issues.

A great richness of personal experience was conveyed through the comments in this survey. A report of this nature cannot do justice to these personal accounts due to the volume of comments. Clearly every comment is important, and condensing the number of comments of people who have had real lived experiences is by its very nature reductionist, and loses the powerful impact of multiple, deeply personal messages about patients' experiences. Nevertheless, the methods used to analyse the data attempted to capture the breadth and depth of difficulties patients experienced.

There is the risk of an inherent bias in free-text responses towards patients who are more literate, have English as a first language, and who do not have learning difficulties. A helpline was available for participants to seek assistance in completing the survey. However, we acknowledge that this method of gathering views on patient experience may not have been the most suitable for all potential participants.

Ideas for future consideration

Many of the comments marked as factual/neutral during the process of splitting comments into top level categories, related to participants commenting that the name of the hospital on the survey letter was not actually the hospital where they had received all their treatment. Many participants received treatment across multiple hospitals, across primary and secondary care, and in some cases, across several Health Boards. The London survey noted a similar problem, with many participants expressing confusion over how to complete the questionnaire given they were treated by multiple trusts (Wiseman et al, 2015). This issue was pre-empted in the Scottish survey by (i) removing the instruction to complete the survey only if the experience happened at the hospital named on the covering letter, and (ii) asking whether or not the treatment took place at the hospital named on the covering letter. However, the feedback from respondents suggests that even this solution does not allow participants to feedback on their care in as much detail as they would like.

Analysis of themes by comment box indicated that there were more negative than positive comments related to the lead up to diagnosis. Confusion over the role of the GP practice in the provision of aftercare also emerged as an issue. Future surveys could consider including a separate free-text comments box on the patient's experience of primary care to provide additional insight into these issues.

Cancer patient experience surveys in the UK have been designed to gather evidence on the experiences of care provided within the NHS, in order to inform NHS improvement strategies. Many SCPES participants indicated that cancer charities and other specialist services also had a role to play in their cancer experience. Patients' accounts in this survey indicate that their cancer care is often reliant on and directly affected by the availability and quality of these services, and as such, questions to gain more information about this relationship could be considered for future surveys.

As previously mentioned, positive statements about care were often generic and lacked detail. This may be because patients found nothing in particular to remark upon if the treatment they received met their expectations. However, understanding patients' expectations, and the specifics of what makes a good experience is important to preserve quality services. Future surveys could consider including questions to elicit the specifics of what patients value or find positive about their care.

Finally, further analysis of the coded qualitative dataset could look at whether sub-themes cluster into groups of co-occurring issues, and also analyse differences in sub-themes by demographic and clinical characteristics.

Conclusions

The inclusion of free-text comments in the first Scottish Cancer Patient Experience Survey has provided a rich and informative analysis of the experiences of people with cancer in Scotland, therefore enhancing the usability and relevance of the quantitative survey results. Following the publication of the Scottish Government's new Cancer Strategy in March 2016, this report provides meaningful evidence to underpin developments in care, coordination of services and research across the country, so as to achieve the ambitions of truly patient centred, safe and effective care for the growing number of people affected by cancer in Scotland.