Income supplement: analysis of options

Annex III: The Policy Simulation Model

The Policy Simulation Model (PSM) is a static micro-simulation model^[68] of the UK tax and benefit system. It is primarily used as a tool for policy appraisal^[69] – estimating the relative impacts of different policy options – and is also currently used by DWP as a part of Universal Credit forecasting at fiscal events.

It is actively maintained, developed, and quality assured by a dedicated professional resource within the DWP's Model Development Division. PSM methodology and assumptions are regularly reviewed, scrutinised, and quality assured by analysts in DWP as well as the Office for Budgetary Responsibility (OBR). The model and its outputs are used by analysts in DWP and across Government.

While DWP owns the PSM framework, the specific modelling approach, analysis, and assumptions outlined in this document (see Annex IV) are the responsibility of the Scottish Government.

DWP provide the Scottish Government with a version of PSM based on Scotland only Family Resources Survey (FRS) data under a Service Level Agreement. Scottish Government are grateful to DWP analysts for the provision of PSM and their support and scrutiny of the income supplement work.

Policy simulation in the PSM is based on two processes. The first is to create simulated future survey data to use for policy appraisal. The PSM works by modelling future versions of the FRS, a continuous household survey which collects information on a representative sample of private households in the United Kingdom. Each household in FRS represents a number of other households who were not surveyed, by being multiplied by a weight. The total of the weights of all the households in the survey adds up to the number of households in the country overall, and the total of the weights of all the households in certain groups adds up to the number of households in those groups. The FRS data is projected forward on a 'static' basis: all household attributes are held constant from the base year, but sample weights are adjusted to match forecasted changes in the population.^[70] For example, to simulate an ageing society, the sample weight of young people would gradually decrease and the weight of older people gradually rise. The PSM is calibrated,^[71] in this fashion, to demographic as well as benefit outturn forecasts.^[72] The resulting future projections of FRS datasets can then be used to model the impacts of a policy in future years.

The second process in the PSM is to apply a model of the tax and benefit system to these projected datasets. By modelling tax and benefit rules in each year, household attributes reported in the FRS can be used to estimate tax liability and benefit entitlement for each household. This allows simulation of a given policy for a given household or sub-group (e.g. by disability status) of the population. The micro impacts from these individual household records can also be aggregated up to the population level through using the survey grossing weights. Note that, some additional information which is necessary to calculate benefit entitlement/ receipt, but not collected in the FRS, is also included in the model.

The impacts of different prospective policies can be simulated by integrating them into the tax and benefits model, and comparing them to other versions of the PSM with baseline and alternative tax and benefit models (i.e. alternative scenarios). Annex IV outlines the assumptions used for the scenarios modelled in this document.

There are several caveats and interpretive notes which should be borne in mind when considering PSM outputs.

1. The PSM is largely based on the FRS and like any self-reported sample survey has certain limitations, e.g. it relies on claimants (and interviewers) providing accurate responses, and is subject to sampling variation and other forms of error associated with a sample survey. While the FRS is the best available source for modelling benefit entitlement, there are known issues such as the under-reporting of benefit receipt,^[73] and reporting of respondent's 'usual pay' being inaccurate for the purposes of reporting benefit receipt.

2. The PSM provides some correction for the under-reporting of income-related benefit receipt. It does not correct for under-reporting of non-income related benefits.

3. As any model, the PSM is based on a certain set of assumptions, which introduces uncertainty into the modelling results produced by the model. It uses data from the past, reweighted and with remodelled benefit entitlements and take-up, to predict the impact of policies in the future. The range of assumptions in this approach – the most fundamental being that past FRS data (suitably reweighted) gives a good representation of the future – all carry associated uncertainty. It should be noted that the reweighting regime also calibrates PSM outputs to official forecasts, which are themselves central estimates with their own underlying assumptions and hence are subject to uncertainty.

4. The PSM assumes no behavioural responses to any policy changes modelled. In some cases, this assumption is likely to be unrealistic, e.g. some policies or other external factors can have a significant and unexpected impact on behaviour, whilst certain policies are specifically designed to engender changes in behaviour. The PSM does not have the capacity to take such effects into account.

5. The version of the PSM we are provided is based on a Scotland-only subset of the FRS, which currently contains around 3,000 sample cases (benefit units), representing the Scottish population. Specific policies may only affect a subset of the population (e.g. certain age-groups), and further sub-setting can lead to small sample sizes. In such cases, a small number of sample observations can account for a large absolute number of individuals through their sample weights.^[74] Conclusions drawn at lower levels are therefore highly sensitive and should be treated with care.