Funeral Support Payment: evaluation

Methodology

This chapter provides an overview of the evaluation approach for Funeral Support Payment. It introduces the logic model and research questions driving the evaluation activities, and gives a summary of the different data sources used.

Overview of evaluation design and logic model

The Funeral Support Payment policy evaluation is based on a "theory of change" logic model. The model shows mechanisms whereby interventions (such as Funeral Support Payment) have a chain of immediate, short-term, and medium-term outcomes. If these are met, they can generate longer-term outcomes and contribute to wider Scottish Government policy impacts.

Funeral Support Payment's medium-term outcomes, and the Scottish Government's long-term policy impact (i.e. reduced poverty which stems from funeral costs), will take time to determine, and may require access to robust quantitative data that is not currently available. The long-term impact in particular will be affected by a range of factors in addition to Funeral Support Payment, making it difficult to measure and attribute changes specifically to that payment. However, the achievement of shorter-term policy outcomes (e.g. reduced need for borrowing to arrange a funeral) could reasonably be expected to contribute to this wider outcome.

The logic model for Funeral Support Payment is below at Figure 1.

The immediate outcomes of Funeral Support Payment relate to benefit delivery process i.e. promoting and administering the benefit. Some of the short-term outcomes also relate to delivery process. However, as shown in Figure 1, they overlap with policy outcomes, which are the intended aims of Funeral Support Payment. Other short-term and medium-term outcomes are specifically policy-related.

Immediate outcomes (Process Evaluation)

• Funeral Support Payment is well promoted
• Funeral Support Payment and its eligibility are well understood
• Funeral Support Payment is taken up
• Making an application is clear and easy
• Applications are processed in a timely manner
• Clients feel they have been treated with dignity, fairness and respect
• Reduced uncertainty about what Funeral Support Payment will provide

Short-term outcomes (Process and Policy Evaluation)

• Clients can pay for funerals within a reasonable timeframe
• Funeral Support Payment is a helpful contribution towards cost of a funeral

Short-term outcomes (Policy Evaluation)

• Reduced need for borrowing to arrange a funeral
• Reduced incidence of money-related stress and grief
• Increased timely payments to funeral director

Medium-term outcomes

• Reduced incidence of debt stemming from funerals
• Reduced business insecurity for funeral directors
• Clients have a positive experience of Scotland's Social Security System

Long-term outcome and impact

The long-term outcome in the logic model is "reduced cases of poverty stemming from funerals". This is when individuals and families experience a financial shock as a result of having to pay for a funeral which pushes them into unsustainable debt. This outcome will not only be impacted by Funeral Support Payment. It will also be influenced by all social security interventions, and actions set out by the Scottish Government to tackle funeral-related poverty^[1]. Therefore, Funeral Support Payment will play an important, but not exclusive, role in contributing to this long-term outcome.

Evaluation questions

Below are the key questions that informed the evaluation design:

1. To what extent did Funeral Support Payment achieve its immediate, short-term, and medium-term policy outcomes?

2. Is there any evidence of Funeral Support Payment contributing to the long-term government policy impact of reduced poverty stemming from funerals?

3. What are the implications of the evaluation findings for future policy development?

Summary of data sources

In accordance with the evaluation strategy^[4], the evidence used in this report was drawn from multiple data sources, described below:

Bespoke Commissioned Research

Ipsos MORI Scotland was commissioned by the Scottish Government to conduct qualitative research with Funeral Support Payment recipients, third sector representatives who provide support to recipients, and funeral directors. The research mainly focused on bereaved people's experience of receiving the benefit, and how this mapped on to Funeral Support Payment policy objectives (i.e. the immediate, short-term and medium-term outcomes outlined above). However, it also explored additional aims, such as potential barriers to applying for the benefit.

In-depth qualitative interviews, lasting around 45 minutes, were conducted with 26 Funeral Support Payment recipients across Scotland. Additionally, a third sector group interview and 6 interviews with funeral directors were held. Fieldwork took place via telephone or online video chat between August 2021 and January 2022^[5]. The key conclusions from the research findings are presented in this evaluation report to provide a fuller understanding of the implementation and impact of Funeral Support Payment at this early stage of delivery. The full bespoke research report from Ipsos MORI Scotland is available at Annex B.

Official Statistics

Social Security Scotland collects information on applications, payments, and clients in the process of delivering the benefits. Some of this information is published online as Official Statistics. The following Official Statistics publications are used as sources of evidence in this report:

• Funeral Support Payment – high level statistics to March 2022
• Social Security Scotland client diversity and equalities analysis to May 2020, November 2020, and May 2021
• Social Security Scotland feedback statistics to 30 September 2021

Of these three publications, the high level statistics data is used most frequently, and is hereafter referred to as "Official Statistics". The other publications are named in full when they are cited.

Please also note the following technical points about how Official Statistics are presented throughout this report:

• Figures are rounded for disclosure control and may not sum due to rounding
• Where stated, secondary analysis has been conducted on rounded figures from published Official Statistics
• Most results are presented to zero decimal places. '0%' should therefore be interpreted to mean less than 0.5%. If no responses were given then this is denoted by '-'.

More detailed figures and information about the Official Statistics used in this report are provided at Annex A.

Social Security Scotland Client Survey

The Social Security Scotland Client Survey ran August/September 2020 (round 1) and May/June 2021 (round 2). It was open to everyone who at that time had received either (a) a Social Security Scotland benefit, or (b) a decision on a benefit application from Social Security Scotland's inception in September 2018 to March 2021.

The Client Survey collected equalities and socio-economic information from applicants. It also asked about their experience of Social Security Scotland and receiving benefits. In total, the survey received 10,575 responses (around 4% of the total number of invites sent), of whom:

1. 337 had applied for Funeral Support Payment only (i.e. as opposed to having applied for Funeral Support Payment and other Social Security Scotland benefits)

2. 259 had received Funeral Support Payment only (i.e. as opposed to having received Funeral Support Payment and other Social Security Scotland benefits)

Throughout this report, Client Survey findings are based on respondents who only applied for or received Funeral Support Payment. This is because their views and experiences specifically relate to applying for Funeral Support Payment, as opposed to potentially applying for numerous benefits.

Please also note the following technical points about how Client Survey findings are presented throughout this report:

• The number of respondents providing a valid answer to each individual question/statement varied slightly, within the ranges shown.
• Most results to the closed questions are rounded to whole numbers. As such, results (e.g. those presented in tables) may not sum to 100% due to rounding.
• Most results are presented to zero decimal places. '0%' should therefore be interpreted to mean less than 0.5%. If no responses were given then this is denoted by '-'.

Limitations

This section explains what can and cannot be determined from the available data sources, and how this influences the extent to which conclusions can be drawn about the early impact of Funeral Support Payment.

Role of qualitative research: This evaluation is largely dependent on findings from qualitative research commissioned by the Scottish Government. This provides a rich and detailed insight into the impact of Funeral Support Payment based on recipients with a range of personal and demographic characteristics, and also the views of third sector stakeholders and funeral directors. However, the research has the following limitations:

1. The findings are not representative of all Funeral Support Payment recipients (or other stakeholders), because (a) while diverse, the overall sample of participants was small, and (b) participants were self-selecting, meaning that they actively chose to take part, as opposed to being randomly selected.

2. Findings are based on the participants' perceptions of impact, rather than objective measures of impact.

These are standard limitations of qualitative work. More detailed limitations have been provided at Annex B.

Role of Social Security Scotland research: The Client Survey statistics contained in this report are based on a range of 252-337 responses from people who applied for Funeral Support Payment (and the number of respondents providing a valid answer to each individual question/statement varies within the ranges shown throughout the report).

Although the survey results provide an insight into the views and experiences of a substantial number of Social Security Scotland clients, it should be cautioned that this represents a small fraction of the 21,375 Funeral Support Payment applicants up to 31 March 2022, and that views are drawn from a self-selecting sample of applicants. No weighting had been applied to counteract potential response bias. Therefore, it cannot be assumed that the results represent the views of Funeral Support Payment clients as a whole.

It should also be noted that the Client Survey was designed for all Social Security Scotland clients, not just Funeral Support Payment applicants. This means that questions are worded in a general way to make them applicable to all benefits. Therefore, while it is a useful source of supplementary evidence, it is not specifically designed to gather the views and experiences of Funeral Support Payment recipients about the benefit.

Medium-term and long-term impacts will take time and additional data to determine: Understanding the true impact of Funeral Support Payment would require a rounded evaluation of progress towards its medium-term outcomes, and also its longer-term contribution to the wider Scottish Government impact of reduced poverty resulting from funeral costs. Doing so requires suitable time to have passed, and for the latter in particular, it would involve isolating the influence of Funeral Support Payment from other contributing factors, such as wider social security benefits, or the actions outlined in the Funeral Costs Plan^[1]. A step in this direction would be to gain access to data with appropriate outcome variables e.g. from population surveys, or further bespoke research. However, at the time of writing there are no concrete proposals on future evaluations of Funeral Support Payment.

More details on options for extended policy evaluation are provided in the evaluation strategy report^[4].