Air departure tax in Scotland: an economic assessment

7. Monitoring and Evaluation Framework

The introduction of ADT represents a significant investment of public funds. As such, there is a requirement to establish a framework for monitoring and latterly evaluating the impact of the policy once implemented.

The implementation of ADT will represent a significant change to the aviation industry in Scotland. However, as explained at various points throughout this report, its implementation will only be one of a large number of factors which influence the structure of the aviation industry and the demand for flights. Outwith general considerations, such as background economic performance, specific issues which could impact on the industry include the emergence of low cost long haul, Brexit and movements in the oil price (at least in relation to Aberdeen Airport). Even with a robust monitoring and evaluation (M&E) framework, isolating and quantifying the impact of the ADT policy will be challenging. This set of monitoring and evaluation difficulties applies to many other taxes and is not unique to ADT.

It is also important to note that, given booking systems are currently open for 2018 and aircraft rotations are already scheduled, there will likely be a short lag before the impact of ADT fully emerges (particularly on the supply side).

Despite the challenges, it will be possible to develop a monitoring framework which will allow for the subsequent evaluation of ADT (albeit taking cognisance of the above points).

Monitoring

The monitoring regime for ADT should be focussed on collecting comparable data on the demand for and supply of air travel from Scottish airports. Any monitoring regime should seek to provide data which will allow the impact of ADT to be determined on (i) the level of connectivity from Scotland’s airports provided by airline type, (ii) passenger numbers by type; and (iii) the cost of air travel.

Monitoring of Demand

There are various levels at which demand for air services can be monitored:

• At the national level, in terms of total passenger throughput from Scottish airports each year
• At the airport level, in terms of passenger throughput at each airport – the CAA publishes monthly terminal passenger numbers through each Scottish airport
• At the route level – data are not readily available at this level but could potentially be collected through a periodic data request to the airlines.

Recording the above information over a period of up to three years should be the foundation of the monitoring exercise.

In addition, RDC Aviation Economics records airline fares at various levels of aggregation. This data should also be collated on a monthly basis, with a profile of average fares at different levels of aggregation maintained over the monitoring period.

The most recent CAA survey covering the Scottish airports (Aberdeen, Edinburgh, Glasgow and Inverness) was published in 2013. We understand that the next equivalent survey will be undertaken in 2018. As part of the monitoring process, the survey coverage should be reviewed to ensure it records travel by the following sub-categories:

• airline type (low-cost carriers etc.);
• UK / foreign passengers;
• destination region;
• price of travel;
• ADT band; and
• business / leisure split.

In terms of recording potential displacement impacts, the above data should also be collated for Inverness, Newcastle, Liverpool John Lennon, Manchester, Leeds / Bradford (although note that it will not be possible to identify causal impacts).

It is not recommended that any data on rail passengers is recorded. Whilst there may be a degree of displacement from rail, particularly in terms of business travel to and from London, the level of displacement in absolute terms is likely to be too small to effectively isolate and monitor.

The level and type of ADT paid to Revenue Scotland should, of course, also be closely monitored through statistical publications or specific, detailed data requests to Revenue Scotland.

Monitoring of Supply

The monitoring of supply will involve annually tracking changes in:

• Airlines – listing of the airlines by market segment operating from each Scottish airport.
• Routes – record of all routes operated from Scotland by market segment, with new routes flagged accordingly.
• Service frequencies – annotation of the record of routes with a record of service frequencies, with increased / decreased service frequencies flagged accordingly.
• Aircraft type – annotation of the record of routes with a record of aircraft type, with increased / decreased seating capacity flagged accordingly.
• Based aircraft operating from Scotland – list of the number of aircraft based in Scotland by airline / airline type, with increases / decreases annotated accordingly.

Some of the supply side data will largely only become available through consultation with the industry. We would recommend that data is collected twice a year over a three-year period, once in winter and once in summer so as to capture the seasonal variation on the supply side.

The demand and supply side data should be recorded on an annual basis in a short working paper. Comment should not be made on the data until the evaluation commences. We would recommend that data is collected for three to five years after the implementation of ADT.

Evaluation

The Scottish Transport Appraisal Guidance ( STAG) ^[50] defines two discrete stages of the evaluation process.

• Process Evaluation – this is conducted at an early stage in the existence of a project and which is primarily concerned with how well the project has been implemented, this is also known as Formative Evaluation; and
• Outcome Evaluation – this is conducted once the project has been in existence for a sufficient period to enable an examination to be undertaken of actual performance against identified targets.

Process Evaluation

As the introduction of ADT is a policy borne out of the current Scottish Government’s manifesto rather than an infrastructure measure, we do not consider a process evaluation to be relevant in this context.

Outcome Evaluation

The introduction of ADT is likely to have an immediate impact on the aviation industry (after a short initial lag), with a more settled position emerging over time. To this end, we would recommend carrying out a relatively light touch evaluation a year (say summer 2019) after the policy is announced, with a full outcome evaluation being carried out around 3 years after its introduction (by which time a more settled position on e.g. Brexit, low-cost long haul etc. is likely to have emerged).

Year 1 Outcome Evaluation

The Year 1 Outcome Evaluation should be focussed, drawing out changes in demand and supply from published datasets. No primary research should be undertaken at this stage.

The evaluation should involve analysis of each of the datasets outlined above and preparation of a short working-paper setting out key trends in demand and supply.

Outline conclusions should be drawn at this stage but it will also be important to note that these will be revisited / firmed-up in the subsequent Year 3 Outcome Evaluation.

Year 3-5 Outcome Evaluation

The Year 3-5 Outcome Evaluation should be a more involved study. The first step in the analysis would be revisiting and extending the Year 1 Outcome Evaluation. Commentary on long-term trends in terms of e.g. terminal passenger numbers, average fares etc would be provided.

If an updated version of the CAA passenger survey including Scottish airports is available by the time of the evaluation, this should be reviewed to determine changes in a number of factors by market segment (e.g. airline type, destination region, business / leisure split etc).

The Year 3-5 evaluation would also include consultation with:

• Airlines, to determine the extent to which different types of airlines have responded to the introduction of ADT (quantified where possible).
  • Specific information on additional based aircraft and the employment associated with these planes should be identified. This should be compared against the 43 jobs per based aircraft figure used in this report.
• Airports, to understand any investments they have made, the number of additional staff employed and the impact on revenue as a result of introducing ADT.
  • It would be particularly beneficial to compare the ratio of direct employment to additional terminal passengers cited in this report
• Virgin East Coast and West Coast to collect any qualitative views on displacement from Anglo-Scottish rail services.
• Tourism bodies to determine the outturn impacts of ADT on this key sector assuming they are in a position to accurately report these.

With the ADT scenario confirmed and up to three years of outturn passenger data available, the appropriate scenario economic impact calculations above should also be re-run to determine an ‘outturn’ economic impact, assuming that a causal impact can be determined. This analysis should take account of the revised employment ratio cited above plus any evidence collected on displacement. A comparison should be made with the forecasts included in this study and reported on.

The final element of the Year 3 Outcome Evaluation should involve a review of the trend in ADT-related revenue accruing to Revenue Scotland. This should be compared with the pre-intervention APD position and the subsequent years of ADT.

Monitoring and Evaluation Framework

The process outlined in this Chapter is formalised in tabular form in Appendix C.