Small Business Bonus Scheme: evaluation

3. Non-domestic rates data

In this section we describe the data that were made available to undertake the evaluation, and highlight some of the issues the data present when attempting to evaluate the effectiveness of the SBBS.

Data on non-domestic properties in Scotland are housed in three main databases:

1. Valuation Roll (VR);

2. Proprietor, Tenant and Occupier Address (PTOA) database; and

3. Billing Snapshot.^[14]

The first two of these are maintained by the Scottish Assessors Association (SAA) and contain information on every non-domestic property in Scotland that is not exempt from entry on the valuation roll. The SAA is made up of 14 Assessors spread across Scotland's 32 local authorities who are responsible for collecting data on non-domestic properties and maintaining the Valuation Roll.

The Billing Snapshot is constructed by the Scottish Government annually by combining billing information from Local Authorities on properties that receive some form of NDR relief. It is therefore a billing snapshot in that it provides an accurate description of the relief awarded to properties on the snapshot date, but not over the course of a whole year.

The VR contains information on each property's location, RV, and the type of business operating from the property. The PTOA contains information on the proprietor, tenant and/or occupier of the property as well as the address of each. The Billing Snapshot contains, for each property that receives NDR relief, the type and level of that relief, as well as the name and address of the recipient.

3.1 How we combined the datasets for our analysis

The Scottish Government provided us with a database of all non-domestic properties in Scotland containing combined VR and PTOA information for each quarter from April 2009 to January 2020.

In addition, for properties in receipt of NDR relief, the Scottish Government provided the Billing Snapshot data for 2011, 2012, 2013, 2015, 2016, 2018 and 2019, combined with the VR and PTOA information for those properties. For 2011-16 the Billing Snapshot data were recorded in October, and in 2018 and 2019 they were recorded in June and May respectively.^[15]

The data in each of the original databases are property-level data. They can be matched and combined at property level by virtue of properties in each database having a Unique Assessor's Reference Number – a number assigned to the property by the responsible assessor that uniquely identifies it from all others within the local authority/ies the assessor covers, which might be only one. Given that this variable is comprised of a string of numbers and letters which included portions of the address, it also uniquely identifies the property across the whole of Scotland.

We combined the VR/PTOA databases with the Billing Snapshot databases in the years in which the Billing Snapshot data are available.^[16] This gives us a database of all properties that contains both the VR and PTOA data and also indicates whether that property enjoyed mandatory or discretionary NDR relief (and if so which, and how much), allowing us to identify which properties received SBBS relief and which did not.

3.2 Properties versus businesses in the data

Whilst some of this evaluation is suited to the property level data, comprehensive analysis of the impacts of the SBBS requires us to consider businesses. However, the available datasets we describe above are unsuited to an analysis of businesses as it is difficult to construct business entities from the information they contain.

The Scottish Government provided some additional information on the presumed business name for each property along with the subsequent site count and cumulative RV for that business (i.e. the number and total RV of properties associated with that business name in Scotland). This was imputed from information recorded in the PTOA dataset.

For example, if Stacey Smith is the proprietor (owner) of the one non-domestic property from which she operates her business, Stacey Smith Ltd., its status would be proprietor. This status would then be used to extract the business name from the first line of the proprietor address in the PTOA dataset.

If Stacey owned two non-domestic properties, say an office and a shop, then the same process would be used and they could then be grouped by their common business name, Stacey Smith Ltd..

If Stacey owned the shop but rented her office space, then the status of her office would be recorded as tenant, and the name of the business using the office would be extracted from the address information of the tenant as recorded in the PTOA database. Again, this could then be combined with her shop by their common business name.

From the inclusion of these presumed business names, the Scottish Government were able to collect together properties in Scotland associated with the same business name to construct a database of businesses. In principle, this also allowed them to identify businesses that occupy a single property (single-site businesses) and businesses that occupy multiple properties (multi-site businesses), and for the latter the number of properties associated with that business and the cumulative RV of those properties. This is important because the eligibility requirements for SBBS relief differ for these two types of businesses.

However, the methodology used to derive business-level information involved a level of uncertainty and the resulting business groupings were imperfect. Unfortunately, the shortcomings of the data sources used to collect information on non-domestic properties in Scotland on which this methodology is used further compounded this uncertainty.

A business entity will only be correctly constructed if every property associated with that business has exactly the same business name, and a business name different to all other businesses. As such, we might deduce that a business has a single property entity when it is actually part of a larger business chain because other properties have been associated with a different business name; and multi-site business entities may have more or fewer properties than should be attributed to the business for the same reason.

For instance, in the above example, correctly identifying the name of a business operating at a property relies on information being recorded accurately at the business level. It is conceivable that the address of Stacey Smith's shop might be recorded as Stacey's residential address and the address of the office as its physical location. This would lead to two different business names being selected despite both properties being used by the same business.

Whilst this simple example may be easily solved by removing the "Ltd." from address fields, it is straightforward to imagine any number of scenarios in which two addresses recorded for different properties in the same business might not match up. It is also possible that removing suffixes like "Ltd." could worsen an already existing issue with any attempt to use this methodology – the properties of a second Stacey Smith operating a business in Scotland, say Stacey Smith & Co., will then be combined with those of Stacey Smith Ltd..

Another important drawback of the business data in Scotland comes as a consequence of its collection. 14 assessors maintain the VR and PTOA data across Scotland's 32 local authorities meaning there are potentially subtle differences in how they are collected, recorded and maintained. For example, there could be differences across assessors in how, when and why key information in the VR is updated. Other than for those under appeal, RVs stay constant until revaluation, the last of which was in 2017; however businesses open, close and move between properties more frequently.

Whilst the structure of the data collected is consistent across assessors, it is unclear why and when address information is updated on the PTOA database – the database used to construct business names. If, for example, the process for changing the PTOA data for a property when the business operating there changes varies across assessors, it would not be possible to account for this in any analysis.

Additionally, local authorities do not coordinate to record properties in different regions that belong to the same business. When applying for rates relief, ratepayers are asked whether their business operates from other non-domestic properties in Scotland. However, this information is self-reported and is not necessarily cross-checked between local authorities. This potentially creates an incentive to misreport as the SBBS has a cumulative RV cap.

Any such incentive is in principle weaker for highly visible businesses such as, for example, large supermarket chains, as the local authority is more likely to be aware of the multi-site nature of the business; and stronger for smaller low-profile businesses which the local authority may not be familiar with. As a result, relief levels recorded at the property level might not align with the eligibility criteria which are laid out by the Scottish Government and which we must use to classify businesses.

3.3 Challenges in evaluating impacts on businesses posed by the data

From the perspective of an evaluation, the largest drawback of the available data is the fact that it is at the property level and not at the business level. As we have noted, the construction of businesses from the data has a high degree of uncertainty. As such, the database of businesses that we have been asked to use, and that forms the basis of the evaluation, is unreliable.

This means it is difficult to identify the entities that should be evaluated, and how they should be treated in terms of SBBS eligibility, the rules for which depend on total property RV within a business. At the same time, the data available does not contain the information to effectively evaluate the impacts of NDR relief on, for example, business activity, creation, survival, relocation and growth – all of which are key outcomes that the Scottish Government are interested in when evaluating NDR relief.

The Scottish Government did attempt to obtain data on business outcomes by merging in some additional administrative data for our econometric analysis (Section 7). We ourselves also attempted to merge the NDR data with survey information from business owners (Section 5 and 6). However, the fundamental basis on which these exercises were carried out is the property level address information. As a result, the uncertainty around identifying the appropriate business entities affects all components of our analysis.

For example, merging our survey data with the NDR data still relied on the estimates of businesses' site count and total RV, as did the sample design and strategy on which it was based. Similarly, the Scottish Government's attempt to merge the NDR data with administrative data on businesses outcomes relied on matching properties in the former database to those in the latter using their address information – a process that shares all the uncertainty of the method by which business groupings were estimated.

Having discovered the extent of these uncertainties during the process of our research, despite the best efforts of the Scottish Government to provide all possible data available held for the purposes of NDR, we are ex ante limited in the confidence with which we can draw any conclusions from this evaluation, primarily due to the inaccuracy of information on the basic entities that should be evaluated.

This problem exists irrespective of our approach to the evaluation or the methodology it employs. It is a fundamental barrier to comprehensive evaluation of the SBBS – or any NDR relief scheme – which stems from the way in which data are collected and maintained. In all of the analysis that follows, we bear this in mind, and indeed make recommendations to rectify the situation.