Scottish Employer Skills Survey 2020: Technical Report

Sampling

Sampling population and survey sampling unit

The sampling unit was at an establishment level, rather than at an organisation level. This is in recognition of the influence that local labour markets have on skills issues and the fact that skills issues are felt most acutely at the site level. This mirrored the establishment-based approach adopted in previous UK-wide Employer Skills Surveys and the Scottish Employer Perspectives Survey, as well as the legacy skills surveys in Scotland.

The respondent sought at each establishment was the person who had most responsibility for staff issues such as training, recruitment or resourcing. For smaller establishments this was most often the general manager or owner, and for larger establishments this was most often the HR manager.^[2]

In line with the approach adopted in the 2013, 2015, 2017 UK ESS and the 2019 survey undertaken in England, Northern Ireland and Wales, the survey population for Scottish ESS 2020 was establishments with 2 or more employees: establishments were eligible if they had two or more people on the payroll at the site, regardless of whether or not these individuals were working proprietors or not, and excluding the self-employed, outside contractors and agency staff.

The 2011 survey was the first year of transitioning to a UK-wide Employer Skills Survey and so it had 1+ employment coverage to allow comparisons with the preceding national skills surveys. Note that in the current Scottish ESS report and all ESS reports going back to 2013, where comparisons are made with 2011, this is based on 2011 data that has been re-weighted on a 2+ employment population. This means that results from the 2011 survey that are presented in the Scottish ESS 2020 report will not necessarily match those published in the 2011 report. The rationale for the change in survey population and the 2011 re-weighting process is detailed in the UK ESS 2013 technical report.^[3]

Sampling approach and setting quotas

Matching the previous iterations of the survey, the Scottish ESS 2020 adopted a disproportionate stratified random sampling strategy such that the quota targets set intentionally oversampled some groups and undersampled others, rather than setting targets in direct proportion to the business population. In practice this means that some smaller sub-groups of employers (such as large establishments) are oversampled to ensure that a sufficiently large number of interviews are achieved to allow for robust sub-group analyses. Oversampling of larger establishments also meant that a larger proportion of the workforce were covered by the survey. Some of the initial quota targets were adjusted towards the end of fieldwork due to the available sample being exhausted, but sample 'substitutions' (i.e. the introduction of new sample outside of the initial sample drawn) were not made.

Quota targets were set by size and sector using interlocked size and sector targets within Scotland.

The first step in setting quotas involved establishing employer size targets, which needed to strike a balance between over-sampling larger employers (relative to the population), whilst not skewing the size profile too far away from smaller establishments. After setting targets by employer size, targets were set by sector within each size band, in proportion to the establishment population within each size band.

Population statistics used to stratify the business population were established through the March 2019 Inter-Departmental Business Register (IDBR), which was the latest available at the time. The IDBR is administered by the Office for National Statistics (ONS), it holds records of all businesses registered for VAT and all businesses operating a pay as you earn (PAYE) income tax scheme. The IDBR is widely regarded as being the most accurate and comprehensive 'official' source of business population data available, and was used for sampling and weighting in all previous editions of the Employer Skills Surveys.

Sizeband quotas

In line with the approach utilised in the Scottish EPS 2019 survey, quota targets based on establishment size were set within each region by distributing interviews in each sector into seven sizebands (see Table 1). Interviews were distributed across sizebands utilising a set of selected ratios that struck a balance between over-sampling larger employers (relative to the unit population) whilst not skewing the size profile too far away from smaller establishments.

Larger establishments were oversampled in order to maximise the proportion of the workforce covered by the survey and because interviews in the largest sizebands have historically proven more difficult to complete. This oversampling of larger establishments was corrected when weighting the survey results (as detailed later in this technical report).

When setting quotas by employer size (for each sector within each region), the sample-to-target ratios were set at an average sample-to-target ratio of 8:1 for each sizeband. Due to the availability of sample (i.e. in certain sizebands it was not possible to order at the desired ratio) this resulted in sample being ordered at a ratio of 8:1 for the 2-4, 5-9, 10-24, 25-49 and 100-249 sizebands,11:1 for the 50-99 sizeband and 7:1 for the 250+ sizeband.

Table 1: Interview quotas by sizeband

Sizeband (employees)	Quota	Sample drawn	Ratio	Proportion of the overall target
2 to 4	930	7,647	8:1	27%
5 to 9	795	6,470	8:1	23%
10 to 24	760	6,144	8:1	22%
25 to 49	447	3,606	8:1	13%
50 to 99	211	2,356	11:1	6%
100 to 249	240	2,038	8:1	7%
250 or more	117	821	7:1	3%

Sector quotas

Sector quotas were defined after setting interview targets by sizeband. Interview targets within each sizeband were allocated by sector according to the proportion of the employer population within each sizeband.

In terms of the definition of sector categories, the quota approach in Scottish ESS 2020 matched that of the UK-wide ESS 2017, with sector definitions utilising 13 categories (as opposed to the 12 sectors used in Scottish EPS 2019). These sectors, defined using Standard Industrial Classifications (SIC), were:

• Primary Sector and Utilities (SIC 01-03, 05-09, 35-39)
• Manufacturing (SIC 10-33)
• Construction (SIC 41-43)
• Wholesale and Retail (SIC 45-47)
• Hotels and Restaurants (SIC 55-56)
• Transport and Storage (SIC 49-53)
• Information and Communications (SIC 58-63)
• Financial Services (SIC 64-66)
• Business Services (68-82)
• Public Administration (SIC 84)
• Education (SIC 85)
• Health and Social Work (SIC 86-88)
• Arts and other service activities (SIC 90-96)

Further information on the SIC definitions for these sectors can be found in Appendix A.

Adopting such an approach ensured the maximum Standard Error associated with findings by sector were minimised, in order to report with a greater level of confidence in the results. Critically, this also helped to ensure that within each broad sizeband and sector, key cuts of the data (such as the nature of skills gaps relating to individual occupations or the causes and implications of specific types of skill-shortage vacancies) were associated with sufficiently robust base sizes.

In some sectors (particularly Public Administration), using this approach produced a regional and sizeband sector target that was greater than the number of interviews realistically achievable given the population in that sector and region. In these cases, targets were revised down to the maximum possible, with the difference redistributed as evenly as possible across the remaining sectors.

Regional targets

Mirroring the approach of previous ESS surveys, interviews were allowed to fall out 'naturally' by geography within Scotland. For the analysis and reporting Regional Outcome Agreement (ROA) areas have been used (how the ROAs are defined in terms of Local Authority is presented in Appendix C). The expected distribution by ROA is shown in Table 2. The 'expected 'fall out' of interviews' column is calculated by multiplying the ROA distribution within the population by the target overall sample size. This expected distribution was felt to provide relatively robust base sizes for regional analyses. The final column shows the actual (unweighted) distribution of interviews by ROA region, which is reasonably close to the expected distribution.

Table 2: Expected (and actual) interview distribution by ROA region ^[4]

Region	Population	Proportional Distribution	Expected 'fall out' of interviews	Actual achieved interviews (unweighted)
Aberdeen and Aberdeenshire	17,200	11%	396	410
Ayrshire	9,100	6%	211	230
Borders	4,300	3%	99	109
Dumfries and Galloway	5,900	4%	137	178
Edinburgh and Lothians	20,200	13%	465	474
Fife	8,600	6%	198	201
Forth Valley	7,500	5%	173	166
Glasgow	21,200	14%	489	375
Highlands and Islands	19,300	13%	444	500
Lanarkshire	16,800	11%	387	328
Tayside	12,100	8%	279	314
West	9,400	6%	217	197
West Lothian	4,100	3%	95	87

Population data is taken from the 2019 Inter-Department Business Register (IDBR) - the latest available business population statistics published by ONS at the time of sampling. Population figures are rounded to the nearest 100.

Sample sources

As in 2017, Market Location was used as the principal sample source of the Scottish ESS 2020. The IDBR was not used as the sample source for the Scottish ESS 2020 (nor any of the previous iterations of the survey) as the majority of records in the IDBR do not come with a telephone number.

Sample was ordered from Market Location at an average ratio of around 7:1 against target interviews required.

A total of 28,884 records were ordered from Market Location for fieldwork. This supplemented 200 records which were initially drawn for piloting purposes from the Scottish EPS 2019 survey (a sample of respondents that had given permission to be re-contacted). In total, 29,082 records were loaded for fieldwork. All sample records were postcode-validated to ensure that geographical regions had been correctly assigned. Checks were also undertaken in instances where duplicate telephone numbers existed within the sample. In certain sectors, such as Retail and Finance, it is common for different establishments to appear under the same centralised telephone number. Such establishments were marked up on the sample ‒ with the address of the sampled establishment displayed on-screen ‒ so that interviewers would be aware that the telephone number they were calling was a centralised switchboard and thus they would need to request to be transferred to a particular site.