Data and methodology

The data used for this analysis was originally collected as part of the Health and Care Experience (HACE) survey, for each year. Further information on the methodology and background to the survey in each year can be found in the relevant technical reports, which are accessible from the HACE survey collection page.

The following sections contain information related to the analysis presented in this report. They also explain why some of the figures presented in this report will be different to those presented in the main HACE national reports for the same years.

In 2023-24, HACE received 107,000 total responses. Of these, 19,091 respondents reported some kind of need for help care and support with everyday living  - whether met or unmet - and 17,747 respondents reported that they provide some amount unpaid care per week. It should be noted however, that response rates to individual questions in the survey varied significantly. 

Comparability over time

Comparing people’s experiences of help, care and support they have received, over time

There were changes to the opening question about whether people had received care, in the 2021-22 and 2023-24 HACE surveys. These changes have affected the cohort (group) of people answering subsequent questions about their experiences receiving care, compared to previous years. This meant it was not always possible to compare responses over time in the HACE national results.

In order to conduct the time series analysis presented in this report, and ensure comparability of responses across the years, we first filtered the dataset for each year, so that in each year they included an equivalent cohort of respondents.

The HACE questionnaire’s opening question to the section on people’s experiences of receiving care asks people whether they have received help or support with everyday living in the last 12 months. From 2015-16 to 2021-22, one possible response option was ‘Yes, help to look after someone else’. This option was removed from the 2023-24 questionnaire, and replaced with a response option of ‘Yes, emotional/community/peer support’. For this analysis, responses from anyone where these options were the only response options they had selected in that year were removed from the datasets, before we carried out any subsequent analysis.

Similarly, in the 2021-22 questionnaire, a new response option was added: ‘Yes, an alarm service (e.g. an electronic device I wear) that can get me help’. Responses to the 2021-22 and 2023-24 surveys where this was the only response option people had selected -  as well as responses where this option was ticked alongside ‘Yes, emotional/community/peer support’, or ‘Yes, help to look after someone else’ were also filtered out, for this analysis.

All analysis presented in this report for people’s experiences of care therefore only included responses from people who had selected any, some or all of ‘Yes, help for me with personal tasks’, ‘Yes, help for me with household tasks’, ‘Yes, help for me for activities outside my home e.g. learning, working, socializing’ or ‘Yes, help for me with adaptations, and/or equipment for my home’. Responses from people selecting ‘Yes, help to look after someone else’ (2015-16 to 2021-22), ‘Yes, emotional/community/peer support’ (2023-24), and/or ‘Yes, an alarm service (e.g. an electronic device I wear) that can get me help’ (2021-22 or 2023-24) continued to be included, as long as the respondent had also selected one of these other four options.

This filtering approach allows us to directly compare the results to subsequent questions about people’s experiences of care, in different years.

The 2015-16 to 2021-22 HACE surveys also asked people who stated they had received care in the last 12 months whether they agreed with the statement ‘I was treated with compassion and understanding’. In the 2023-24 survey this statement was replaced with the statement ‘I was treated with dignity and respect’. These are deemed to be substantively different questions, so answers to these questions have been excluded from the analysis presented here.

Comparing results to other questions, over time

The exact wording of some of the questions analysed for this report have been subject to minor revisions over the years. For example, the wording of the question related to understanding the percentage of people who provide care to others, and the hours of care they provide, has undergone minor changes between the surveys to improve clarity, and to better align the wording with questions on unpaid carers in other Scottish Government surveys such as the Scottish Health Survey.

However, the nature of the questions has remained consistent across all the years. Therefore, we do not anticipate any impact on the comparability of the answers over time and have presented the results for these questions in a time series.

Other questions about people’s experiences of care have changed substantively over the years the HACE survey has been running. We did not analyse the responses to these questions for this report, as the answers can’t be compared over time.

Further information around the rationale for, and detail of, any changes in questions between years is outlined in the technical reports accompanying each HACE national report. These are accessible from the Health and Care Experience survey collection page.

Weightings

As with the main HACE publication, throughout this report the survey results are presented as weighted average percentages. Different proportions of people were selected to take part in HACE from each General Practice in Scotland, and the response rate varied by age and sex. Weighting is used to produce results from the survey which are more representative of the population of Scotland as a whole.

A new weighting methodology was introduced for HACE in 2017-18. Figures from previous surveys were backdated in HACE reports from 2017-18 onwards, where appropriate, to ensure valid comparisons over time. The analysis for 2015-16 presented in this report uses the new weighting.

As mentioned previously, some responses to questions about people’s experiences of care were filtered out of this time series analysis, to ensure comparability across years. As the total number of records being removed each year by this filtering was very small, we assessed that the likely impact on the results of retaining the initial weights applied at national level would be minimal. We therefore decided not to recalculate the weightings used for estimating national-level results to the questions about people’s experiences of care.

Confidence intervals

Confidence intervals provide a way of quantifying sampling uncertainty. A 95% confidence interval means that, over many repeats of a survey under the same conditions, we would expect that the confidence interval would contain the true population value 95 times out of 100.

For example, given a result of 80% and a confidence interval of +/-3%, there is 95% confidence that the true result is between 77% and 83%.

There is more information on confidence intervals (including how they are calculated) in the HACE 2023-24 technical report.

The confidence intervals associated with the results presented for this analysis are available in the supplementary tables.

Significance testing

All changes over time that are discussed in the report are statistically significant at the 95% level. Due to the large sample size for some questions, even small changes (of less than 1 percentage point) in the national results may be statistically significant.

The approach to significance testing in this report is consistent with the main HACE publication. Details of this can be found in the HACE 2023-24 technical report.

Open text analysis of responses to question on unmet need

This publication presents the results of further analysis conducted on open text responses to question 32 of the HACE survey 2023-24, which asked ‘If you are not receiving all the help and care services for everyday living that you feel you need, which options describe your situation?’. Respondents could select more than one option and had space to leave an open text response in a comment box accompanying the option “Other”.  

 Whilst some respondents interpreted the “other” option as a space to detail further reasons for experiencing an unmet need for health and care services, not all respondents did so, and many used this box to provide insights related to different parts of their experience. For example, many respondents used this box to clarify that they did not need support for everyday living, or that their needs were fully met. To provide an accurate percentage breakdown of question 32, wherein the percentage for “other” could be interpreted as expressing “other” reasons for unmet need, it was necessary to first manually recode these open responses based on whether they indicated an unmet need.  

 In total, 1,601 respondents provided an open text response to question 32. Each open text response was checked to ascertain whether it indicated an unmet need. If the response indicated unmet need, the “other” box was confirmed/ selected. If the open text response did not indicate unmet need the “other” box was deselected. This resulted in 643 open text responses being selected/confirmed as “other” and 958 responses deselected. Cases where respondents had selected the box “other”, but had not left an open text comment, remained selected as “other”, there were 387 such cases. In total, the recode resulted in 1,030 responses being confirmed/ selected as “other”.

We also conducted additional content analysis of the 643 open text responses which indicated an unmet need. We first sought to identify which service need was expressed, if any (e.g., hospital care, primary care, mental health etc.). We then developed key themes through a second stage of iterative content analysis. In total, 18 themes were identified, these are listed in Annex A.