Ukraine Sponsorship Scheme in Scotland: modelling report

4. Methodology: Welcome accommodation forecast

4.1 Monte Carlo simulation

This model uses Monte Carlo simulation, a methodology for estimating the possible outcomes of uncertain events which uses randomness to capture the uncertainty in the process.

In this model, each uncertain input is represented by a probability distribution that defines the possible values of the input and the relative likelihood of each value occurring. The probability distributions for the inputs are derived predominantly from historical data and, where necessary, uncertainty ranges are chosen to represent the level of confidence in the input data and assumptions. Where historical data is unavailable e.g. for ended matches (the modelled number of host matches ending in Scotland each day) we use survey data. The inputs are regularly monitored and updated as new data becomes available. By reviewing the model performance as outlined in Section 4.2 we can verify the effectiveness of the inputs used. A significant deviation in the actual observed data compared to the model forecasts could indicate issues with the methodology, the input data being used or a change in behaviour that is not captured in the model.

Each model run consists of 5,000 iterations, and on each iteration a single value for every input is randomly selected from its probability distribution. These values are combined to give one possible forecast of welcome accommodation occupancy:

The model therefore uses the following inputs, all estimated from the available data:

• Daily departures – the modelled number of departures from welcome accommodation each day
• New arrivals – the modelled number of arrivals into Scotland each day
• New arrival entry percentage – the modelled proportion of new arrivals into Scotland that will require welcome accommodation
• Ended matches – the modelled number of host matches ending in Scotland each day
• Re-entry percentage – the modelled proportion of host matches ending that may seek to re-enter welcome accommodation

When all the iterations are combined, there are possible forecasts which give a range of possible outcomes for each forecast date. From this a central forecast is calculated by taking the mean value for each date, and upper and lower limits are calculated by taking the 5th and 95th percentiles, respectively, for each date.

4.2 Performance of Monte Carlo model compared to historical data

Figure 2 shows five previous forecasts produced using the Monte Carlo model, covering the period from 29th March 2023 to 14th August 2023. The forecasts (5th and 95th percentiles) are represented by shaded regions in various colours, while the actual numbers of rooms and ship cabins reported as occupied are shown as black dots. If the data and assumptions underlying each of the model runs are correct then there is a 90% chance that the number of rooms required in the future will lie within the shaded region.

Figure 2: Historical welcome accommodation occupancy (actual data shown by solid, black dots) compared to forecasts of the number of rooms required (overlapping fans) created between 29th March and 4th July 2023.

The chart shows the combined total of rooms and ship cabins occupied. However, note that the contract for the M/S Victoria ended on 11 July 2023 and all guests disembarked by the end of the M/S Victoria contract therefore there are no ship cabins occupied from this date.

Figure 2 shows that the vast majority of the actual data points for welcome accommodation occupancy are within the modelled prediction interval. This provides confidence in the suitability of the overall methodology and forecast.

Note that over time the (vertical) width of the shaded regions of the forecast increases. This represents increasing uncertainty in the forecast the further away we move from the initial starting point. For example, the forecast based on data up to 10th May 2023 predicted that the number of rooms required on 5th June 2023 would be between 1,960 and 2,400. The actual number of rooms reported occupied on that date was 2,180, very close to the midpoint between these two extremes.

In light of the rate at which the prediction interval increases over time it would be most appropriate to use this model in isolation to produce medium-term forecasts. The prediction intervals would be too wide in longer-term forecasts to be of practical use. In addition, there are a number of unknown factors that are beyond the scope of this model but which could, in principle, have a substantive impact on the trajectory of the number of rooms occupied. For this reason the Monte Carlo model is used to give medium-term projections which can then be regularly updated in light of new data.

Changes in the input data for each forecast include not only the latest number of occupied rooms and cabins, but also additional data not shown in the chart (such as the number of arrivals, the proportion of arrivals entering welcome accommodation and the rate at which people are leaving welcome accommodation). Where exact figures are not available a probability distribution representing a range of possible values is used instead.

4.3 Model input distributions

The modelling approach uses historic data where available to define the input distributions. The inputs are regularly monitored and updated as new data becomes available. By reviewing the model performance as outlined in Section 4.2 we can verify the effectiveness of the inputs used. A significant deviation in the actual observed data compared to the model forecasts could indicate issues with the methodology, the input data being used or a change in behaviour that is not captured in the model.