Scottish housing market: tax revenue forecasting models – review

5. Suite modelling, combining forecasts, and averaging forecasts

As discussed in Box 1, it is unlikely that any one model can fulfil both the forecasting and policy requirements of public budget forecasters. Even if the model is intended for forecasting alone, the literature on forecast performance in Section 3 and the techniques to improve upon standard forecasts in Section 4 suggest that it is unlikely that one forecasting approach will suit all circumstances.

In response to these issues, many practitioners employ several models to improve forecasting performance or provide additional detail for policy development. Multiple models are general employed in three main manners: suite modelling, forecast combinations, and forecast averages.

Suite modelling. The demands placed on a housing market forecast could require a broader macroeconomic forecasting framework and extensive use of auxiliary models. If an objective of the housing market forecast is to include influences of the wider economy, such as employment and real income growth, then out-of-sample forecasting may require exogenous forecasts of these variables. Further, these variables could in turn depend on the housing market, and will need to be a part of an iterative macroeconomic forecasting framework. Not having this broader framework would restrict the modelling approaches that could be used.

Macroeconometric models may lack the detail required to model certain tax bases, or national accounting concepts may not be directly comparable with tax bases. In this case, multivariate econometric models estimated on their own can be combined with large-scale aggregate macroeconometric models to provide the detail necessary for bottom-up fiscal forecasting.

A suite approach may be particularly important in turbulent periods, when the economy is subjected to significant structural changes. OBR (2013) describes their approach to modelling during times of uncertainty:

[…] it makes sense to use a range of approaches and models to inform the forecast, rather than to rely solely on the behavioural relationship implied by the model, which will reflect the behaviour of the economy over the specific time period over which the relationships were estimated. (p. 3)

The Bank of England makes extensive use of suite modelling, and applies several dozen models to many of its monetary policy decision frameworks. According to Burgess et al. (2013), the Bank uses 15 or more statistical forecasting models running from univariate time series to Bayesian and factor-augmented VARs to produce a wide range of results for GDP and inflation.

In their words:

These more traditional models have undoubted strengths: they are usually simple to understand, can quickly identify potential inconsistencies in COMPASS-based forecasts, and in many cases have an established track record in the Bank's forecast process. However, they also have limitations when compared with more structural models. They are not designed to produce joint forecast densities for the complete set of COMPASS observable variables, which makes direct comparison problematic. Moreover, in some cases, they can only produce conditional forecasts, taking some variables from COMPASS and other suite models as inputs. As a result, their forecasts may not be fully independent of all the judgements captured in the central organising model. (p. 47)

The typical use of a suite of models to ensure new policy measures are reflected in the fiscal and macroeconomic outlook was described in a practitioner interview as follows (using a change in mortgage insurance requirements as an example): 1) estimate the direct impact on GDP of changes in new and used home purchases using a micro-simulation approach based on borrower-level data from the public mortgage insurance provider, 2) estimate the broader economic implications by imposing the direct GDP shock onto the finance department's macroeconometric model, 3) update the fiscal model with both the detailed housing sector data and the new set of economic determinants from the macroeconomic model, and finally 4) iterate the fiscal model results between the macroeconometric model and fiscal model until a steady-state solution is achieved.

Simple univariate models that only consider seasonality are frequently used by practitioners to build up in-year estimates from monthly receipts. For example, if LBTT receipts exhibit a predictable monthly profile, it could be exploited to arrive at an estimate for the current year that is likely to be better than a forecasting model's raw output, depending on how many months are available. This profiling can be adjusted based on expected economic developments using the series' historically estimated sensitivity to the economy and any abnormal transactions can be removed. This careful monthly monitoring may be particularly important for Scotland's non-residential market, where a small number of high-value commercial property transactions can cause a significant spike in a month's receipts and may be carried through to future months by the model. ^[43]

Combining forecasting approaches. Aspects of the different model classes assessed in Section 3 can be combined-either directly in the model's specification, or by relying on different modelling approaches for different time periods.

For example, error-correction model techniques are often applied in a systems VAR approach to give vector error-correction models. Large-scale macroeconometric models can be combined with DSGE modelling approaches to create a more compact hybrid approach, as in OBR (2012b).

Forecasting models could be joined to forecast different time horizons. For example, a model that excels at the near-term could be combined with a model that excels at the medium- to longer-term. Or the end of the forecast horizon could be anchored with a technical assumption and then a model result used for the near-term, interpolating between. The Scottish Government previously used this approach for prices, using an ARIMA model for the first two years of the outlook and then anchoring the end of the forecast with the historical average growth rate of prices.

Combinations could be used to fill data gaps, such as the combination of a private sector average for the first two years followed by a rule of thumb (prices grow in line with average earnings) for the third year to the fifth, as the OBR used before introducing their own model. The OBR continues to use a combination approach, using their auxiliary error-correction model of the housing sector for outer years and leading indicators and judgment for earlier quarters of the outlook (Auterson, 2014).

Carnot (2014) suggests that the econometric approach to forecasting the housing sector is generally less useful during the first year of a forecast than monitoring building permits and other high frequency data for residential construction.

Averaging forecasts. There is considerable evidence that a combination of forecasts can often have better forecasts than any one approach on its own. For example, An de Meulen, Schmidt, and Micheli (2011) find that combining an AR, multivariate regression, and VAR substantially reduces the MSFE for forecast of a German house price dataset. More generally, Granziera et al. (2013) look at combining forecasts for a wide range of macroeconomic variables and find it generally improves forecasts relative to a benchmark.

Averaging internal forecasts is not done formally among any practitioners with whom we spoke. However, the same effect is achieved via judgment, challenges from other internal models, or comparisons to other non-government sector forecasts.