Local Area Labour Markets in Scotland - Statistics from the Annual Population Survey 2013

Summary publication of results from the Annual Population Survey 2013, presenting analysis on the labour market, education and training. Results are provided for Scotland and local authority areas in Scotland.


Annex F: Technical note on APS sample size

Response rates for many government sponsored social surveys have been decreasing steadily for several years. Typical response rates for ONS survey in the 1980/1990's was around 80%, but these have declined over the years to their current levels in the region of 60%. These are being driven by increasing refusals and non-contact (where the interviewer fails to make contact with anyone at the target address after many repeated attempts). A multitude of factors have been used to explain the downward trend in response rates. Some of these include:

  • falling contact rates attributed to: rise in single person households; rise in households where all adults are in work; rise in controlled access to properties
  • increasing interview lengths
  • survey overload: the salience of a survey topic has become a more important determinant of response
  • external shocks such as data losses, which have impacted on trust in statistics;
  • reduced budgets which result in less appetite to administer costly re-issue exercises.

In additional the Annual Population Survey shows higher levels of attrition than in the Labour Force Survey (where respondents who had completed the survey in the first wave, either refuse to take part or are no longer contactable in subsequent waves). This is because respondents in the APS are re-interviewed at annual intervals over four years, compared to the five quarterly interviews over the course of one year for the LFS.

Previously, one of the responses to falling response rates was to increase the overall sample pool, hence ensuring that the final sample of responding households remained constant. However, as government budgets have reduced, this option has become increasingly difficult to support. In the fiscal year (April 2011 to March 2012), this meant that ONS held the sample pool for Scotland at a fixed level instead of increasing it, as it had in previous years, resulting in the actual sample size for the APS for Jan-Dec 2011 in Scotland being reduced by around 90 households each quarter. For the last fiscal year (April 2012- March 2013), a further reduction of 90 households per quarter was made as financial constraints within government departments continued. For the coming financial year (April 2014 - March 2015) the Scottish Government have agreed a settlement with ONS to hold and guarantee the sample size at the level set in April 2012- March 2013, which will maintain the quality of survey estimates.

The impact of these sample size reductions will be seen in both the variability of estimates over time and in the reliability of estimates. Comparing confidence intervals (CI's) for 2011 to 2012's data for employment rates, for example, an increase of around 3% was seen at Scotland level, from +0.54pp to +0.56pp. However, this is not consistent across local authority areas; some areas see decreases (e.g. Moray and Aberdeenshire where CI's decreased by 6% and 11% respectively), while most (22 out of 32) have seen increases, with the highest increases in Highland and Shetland Islands (where CI's have increased by 17% and 43% respectively - from +2.5pp to +2.9pp and from +4.3pp to +6.2pp respectively). It is worth noting that the high increase in Shetland Islands may be more related to poor response rates than any decrease in sample size, as the ONS target for the required number of economically active interviews had not been cut in this area due to its already small level. Table 12 below shows how these targets have been changed. It should be noted, however, that these figures are target levels and that the reductions shown are in part due to sample reductions, but also in part due to rebasing of levels to acknowledge the issues ONS has with lower response rates and achieving target levels which were originally set when response rates were much higher.

ONS have an on-going program to address the declines in response rates tackling a wide range of issues including; interviewer training, incentives, survey materials, questionnaire reviews, interviewer performance, IT infrastructure and interviewer contracts. In addition, ONS has a programme of research looking at the impact of declining response on survey outputs, including:

  • review of reasons for non-response and refusal
  • an analysis of attrition bias between waves in the LFS
  • refusal follow-up study funded by EUROSTAT
  • review of use of incentives
  • Census non-response link study
  • evaluation of internet data collection for the LFS

Table 12: Change in ONS target for economically active adults interviewed by Scottish local authority

Local authority Previous target for 2011-12 Target for 2012-13 on
Aberdeen City 875 675
Aberdeenshire 875 675
Angus 875 675
Argyll & Bute 700 675
Clackmannanshire 300 250
Dumfries and Galloway 875 700
Dundee City 875 675
East Ayrshire 875 675
East Dunbartonshire 875 700
East Lothian 800 600
East Renfrewshire 800 600
Edinburgh, City of 875 675
Falkirk 875 675
Fife 875 675
Glasgow City 875 675
Highland 875 675
Inverclyde 700 600
Midlothian 700 600
Moray 875 600
North Ayrshire 875 675
North Lanarkshire 875 675
Orkney Isles 200 200
Perth and Kinross 875 675
Renfrewshire 875 675
Scot Borders, The 875 675
Shetland Isles 200 200
South Ayrshire 875 675
South Lanarkshire 875 675
Stirling 600 600
West Dunbartonshire 875 700
West Lothian 875 675
Western Isles 200 200

Contact

Email: Alan Winetrobe

Back to top