Budget 2019 to 2020: feasibility of distributional analysis - study

1. Introduction

This report presents findings about the feasibility of undertaking distributional analysis for tax, benefits and public services expenditure of the Scottish Budget.

Developing distributional analysis tools within the Scottish Government could help inform future policy decisions and enable the government and others to understand the value of the Scottish offer.^[1] This type of analysis is produced by both HM Treasury^[2] and Office for National Statistics^[3] to show the impacts of the UK budgets in terms of tax, social security benefits (welfare) and expenditure on public services (benefits in kind) by income decile.

This report summarises the approaches and methods used in undertaking this challenging task. All distributional analysis requires a range of decisions and assumptions to be made about the benefits, taxes and services that should or could be included in the analysis and the modelling approaches used to define them.^[4] The report outlines the chosen approach taken for the feasibility work and sets out the decisions taken including the pros and cons of each. It also highlights some of the challenges and limitations of interpreting any form of distributional analysis.

In this report the feasibility of undertaking distributional analysis has been tested by income distribution and also by protected characteristics. Further details, including technical details, are included in Annexes A to G.

1.1 Background

Scottish Government has for some time undertaken distributional analysis of individual direct tax and social security benefit policies. They are an analytical part of the policy development process intended to provide information about the impact on different groups. However, there have been calls in the past few years for this budget analysis to be extended to produce a cumulative assessment in order to understand the full impact of both direct and indirect taxes alongside other budget decisions on individuals and households. In particular, the Scottish Parliament's Budget Process Review Group recommended that Scottish Government "consider the feasibility of distributional analysis for tax, benefit and expenditure by protected characteristic and income distribution".^[5] A similar call was made by the Parliament's Equality and Human Rights Committee and the Equality and Human Rights Commission (EHRC).

Distributional analysis is commonly used to estimate the effects of direct taxes and benefits on people with different levels of income. The call from the Scottish Parliament's Budget Process Review Group was to extend this to consider distributional analysis by protected characteristics, enabling investigation of how different groups – by protected characteristic – fare in terms of tax and benefit policy. It might also provide a better understanding of the intersection of different characteristics. The further expansion to include public expenditure allows for a better understanding of how the Scottish offer, in full, impacts on those in different income brackets and also by protected characteristic. The addition of analysis by equality characteristics and the inclusion of public spending alongside direct taxes and benefits, were the two major novel features of the project.

1.2 Challenges in extending distributional analysis

The calls to extend distributional analysis also recognised that this would present considerable challenges. A seminar of experts brought together in October 2018^[6] to discuss the feasibility of distributional analysis in Scotland identified a range of issues that would need to be addressed.

A key initial question was which components of government activity to include in the model. On the one hand, distributional analysis which incorporates indirect tax and public services is desirable because it provides a more complete picture of the relationship between individuals and the state than an analysis which focuses solely on direct taxes and benefits. On the other hand, while public services tend to be progressive, they are based on the principle of universality and as such are not intended to be redistributive in the manner of tax and benefits. For this reason, performing distributional analysis on public spending is conceptually problematic. This may explain why despite quite a few governments undertaking and presenting distributional analysis of changes to tax and welfare benefit policies – very few attempt to present analysis of the overall offer/package that people receive.^[7]

One of the main challenges of performing distributional analysis in Scotland is the relatively small size of the Scottish sample in UK-wide surveys such as the Family Resources Survey (FRS) and the Living Costs and Food Survey (LCF), the two main data sources for most distributional analysis in the UK. Although the Scottish FRS sample has been boosted, and the Scottish LCF sample is due to be boosted, it is still necessary to pool multiple years of data. Pooling, however, runs the risk of concealing distributional changes within rolling averages.

There are also issues with the quality of the data sources themselves. The FRS, for example, is known to underreport benefit take-up and mis-report very high and very low incomes. Gender also poses unique challenges for distributional analysis due to the lack of data on how income is shared within households, which obscures both distributive impacts and behavioural responses. Ethnicity and religion and belief data is affected by small sample size and other characteristics such as sexual orientation, gender reassignment and pregnancy and maternity are absent from the data.

Public spending is the most difficult component to model from a technical perspective. Generally speaking, the first step involves determining patterns of service use, either by gathering usage data or by estimating usage patterns based on household characteristics. However, the benefits of a given service cannot always be ascertained from usage of that service. Additionally, some services benefit everyone equally and are not exactly 'used' by anyone. This is the case with defence and environmental protection, for example, which are generally excluded from distributional analyses. Even when usage differs between groups it may present a misleading picture. For example, individuals who do not use a given service may still benefit from it, and one person's use may have knock-on benefits to others. The NHS, for example, provides insurance to all members of society, not just those who have existing health conditions, and also benefits employers by supporting a healthy workforce. Conversely, it is simplistic to assume that, because certain people use a public service more, they would necessarily benefit more from public spending on that service. For example, women use buses more than men, but it would be misleading to conclude that spending on buses is straightforwardly positive for gender equality, since asymmetric bus use is likely to reflect underlying inequalities.

The second step in incorporating public services in distributional analysis is to combine service use with some kind of 'price' in order to translate in-kind benefits into monetary values. Some aspects of public service expenditure are easier to model than others. Resource expenditure, for example, is generally more straightforward than capital, the benefits of which may not accrue immediately and may be more dispersed across the population. Even resource expenditure raises problems, however, since the benefits may spill over from the immediate beneficiaries. For example, if NHS pensions were increased, and morale and productivity improved as a result, it could be argued that the wider public would benefit in addition to NHS workers and their families through an improvement in the service.

In summary the technical challenges of distributional analysis can be broken down into three sets of issues:

• Data availability challenges include:
  • lack of sufficient Scottish sample size in some surveys – which provide the basis for analysis for protected characteristics
  • data gaps about income sharing in families
  • data gaps about income and benefits received (including under-reporting of benefit take-up and inaccuracies in reporting of income received).
• Modelling of public spending on services is difficult for several reasons, including:
  • determining patterns of usage
  • determining the value/price attached to usage
  • quantifying the benefit produced by, and who benefits from, usage.
• The interpretation of findings can be complex.

1.3 Feasibility project

In December 2018, it was agreed with Ministers that the Scottish Government would take forward work to consider the feasibility of distributional analysis as a specific element of budget analysis.

The project aimed to look at the feasibility of developing analysis of the cumulative impact of different taxes, cash benefits and spending on public services. Many individual policies make up the overall package of the 'Scottish offer' but the aim of this work is to understand how the different parts of the spend and revenue of the Scottish Budget combine and the impact of that combination on people and households in Scotland. It does not look at the impact of changes in individual policies, but only at the overall offer at a point in time for different individuals and households. The results from the project in effect produce a baseline of the current offer in total. This could help highlight areas where a separate 'what-if' analysis might be usefully undertaken to understand the effect of making a change to a particular aspect of tax or expenditure.

The aims of the project were:

• to undertake feasibility work, on the technical aspects of cumulative distributional analysis
• to undertake feasibility work on the potential outputs, applications and interpretation of the results from cumulative distributional analysis
• to advise Ministers and stakeholders on the technical feasibility and potential usefulness of distributional analysis of the Scottish budget for informing policy development.

To undertake the technical feasibility of a cumulative distributional analysis it was necessary to design, populate and test a model. To test the model, data was used from a previous budget (Scottish Budget 2019-20). The results allowed us to consider wider aspects of feasibility such as issues in interpretation and potential usefulness of the analysis.

The project was guided by a project board and internal quality assurance was provided by a group of Scottish Government analysts and policy advisers. External quality assurance was provided by IPPR.

1.4 Structure of report

Chapter 2 of this report sets out which budget items our analysis is based on. Annex A gives references to the individual budget lines.

Chapter 3 gives an overview of our approach to modelling and the model we have developed, which brings together taxes, benefits and public services. It shows what is included in the model and gives an estimate of coverage. Annexes B, C and D give technical overviews of how we have approached each aspect of the modelling, together with some descriptive output.

Chapter 4 shows the outputs of the model and gives a commentary on the charts produced. The charts and chart data can be found in Annex F and G. Annex E gives additional information about the composition of the different income deciles.

Finally, Chapter 5 presents our conclusions of the feasibility study.