Scottish housing market: tax revenue forecasting models – review

2. Methodology and evaluation criteria

To identify candidate approaches, we searched peer-reviewed journals and the research of public and private sector forecasters. We also interviewed experts on housing market forecasting and budget preparation in several OECD governments, independent fiscal scrutiny bodies, and central banks. We narrowed our evaluation to the model classes in Section 3 based on these initial findings. We then selected the literature that was most relevant to Scotland to support a systematic assessment of these model classes.

The criteria against which we evaluated each model class are listed in Table 1. We first assessed whether the model was likely to be suitable for the Scottish housing market. We divided the application criteria into two components: 1) forecasting, and 2) policy analysis. The distinction between these two objectives is important for model selection and is discussed in detail in Box 1.

Assessing accuracy is difficult without developing the models themselves and comparing their out-of-sample forecasting properties to the current approach. As will be shown, research suggests that the accuracy of different model classes depends on the unique circumstances under which the model is asked to perform (that is, the forecast period, geographical region, and position within the business cycle). We were able, however, to provide indicative evidence from comparative studies. Because models tend to have different forecasting properties over the short and medium term, we attempted to find evidence on performance in the first eight quarters and the last three years of the intended five-year forecasting horizon. This breakdown could be useful for combining models.

We then assessed several elements of the model's ease of communication, included whether the model can tell a convincing story for both its current path and revisions since the last forecast round, and whether it is likely to be transparent to stakeholders.

We also ranked the model on whether its data requirements are likely to exist and be met in a timely matter, and the likely use of resources the model will require to develop, run, and maintain over time.

Each criterion was assigned a summary score of good, fair, or poor, based on the most likely scenario we can foresee without carrying through the model development itself. Results may vary in practice.

Box 1: forecasting models versus policy models

A first question to ask is: what will the model be expected to do? If it will be required only to forecast baseline revenues, a wide variety of forecasting approaches are possible. However, forecasting models in a public budget setting are often asked to do other analysis, broadly defined as policy analysis. Policy analysis includes:

• Fiscal impact costing. The model may be asked to assess the fiscal cost (that is, the increase or decrease in government revenues) of changes in housing market policy or changes in LBTT policy, such as rates and thresholds.
• Scenario analysis. The model may be asked to assess the impact on housing demand of changes to other economic and fiscal assumptions (for example, stricter financial regulation or higher social assistance for lower income households).
• Risk analysis. Budget forecasters often publish a table of fiscal sensitivities or risks to the outlook-that is, how a change in GDP, inflation, or other inputs may affect revenues. This is useful, for example, if a minister is interested in how LBTT revenues would change if the outlook for economic growth were to fall below the budget's assumption, or if the central bank were to miss its inflation target.

Policy and risk analysis requires models that have been developed to capture the underlying process that generates the data. This largely rules out, for example, univariate forecasting models that make no attempt to describe how the housing market is influenced by other economic and fiscal variables of interest (and therefore cannot use alternative assumptions to compute costings, scenarios, and fiscal sensitivities).

Forecasting and policy models place different emphasis on the intertemporal dynamics of the data (the way that past values influence current and future values). Forecasting models place the importance of observed intertemporal properties and dynamics above all else. Policy models, on the other hand, place importance on economic and statistical theory to drive the equation specification. By emphasizing theory, policy models sometimes miss out on potentially useful forecasting information in the dynamics of the data.

We did not attempt to assign weights of relative importance to the criteria, and their interaction is not straightforward. For example, a model that is specified to easily attribute forecast errors to economic determinants (that is, it scores highly on the communication criterion) is likely to require forecasts of exogenous economic variables. Forecasting exogenous variables may require hiring more analysts (reducing the model's resources score) and may introduce additional uncertainty (reducing the model's accuracy score). Scores should therefore only be taken as indicative and should not be considered in isolation.

Table 1: Assessment criteria