Cereal and oilseed rape harvest: final estimates - 2016

8. Methodology and Quality Note

This section provides a summary of information on these statistics against five dimensions of quality, based on the European Statistical System (ESS) quality framework: Relevance; Accuracy; Timeliness and Punctuality; Accessibility and Clarity; and Comparability. The Scottish Government adheres to the Code of Practice for Official Statistics and the National Statistician's guidance on quality. In addition the Scottish Government provides its own guidance on quality, which is available to view at the Scottish Government's Statistics internet pages.

Further information on quality:

• Code of Practice for Official Statistics
• National Statistician's Guidance on Quality
• Scottish Government's Corporate Policy Statement
• Scottish Government Guide to basic quality assurance
• European Statistics Code of Practice (including quality framework)

Methodology

The 2016 final estimates of production are based mainly on final yield results from the 2016 Cereal Production Survey (CPS) and final crop areas from the 2016 June Census. The CPS is a disproportionate stratified random sample of around 630 farms in Scotland, stratified by region. The construction of the sample is based on the Neyman Allocation which apportions larger sample sizes to the strata with the most variation in yields.

In 2016, the number of holdings submitting a return for Spring Barley was 275, Winter Barley was 100, Wheat was 148, Oats was 93 and Oilseed Rape was 63. For some regions relatively few returns were received for some crops.

Totals of sample production and sample crop area for each stratum (i.e. crop and region combination) are used to derive a sample estimate of yield. These yield values are applied to national crop areas from the June Agricultural Census to provide national estimates of production. Where sample sizes for strata are insufficient to calculate production results national average yield estimates for the crop are used to calculate estimates of production.

2016 regional results for winter oats, triticale and spring oilseed rape were generally based on national averages.

The Cereal Production Survey is carried out by Rural Payments & Inspections Division (RPID) and Rural and Environment Science & Analytical Services (RESAS) within the Scottish Government (SG). The survey is carried out by telephone with forms mailed to farms on request. Completed returns are analysed by RESAS.

The data undergo several validation processes as follows; (i) checking for any obvious errors on the paper survey forms upon receipt, (ii) cross checking against Census area data and internal validation within survey forms to ensure totals match, (iii) results are standardised to 14.5 per cent moisture content for cereals and nine per cent moisture content for oilseed rape (iv) assessing data for any extreme yield values and removing if necessary, (v) if required, area offices are contacted to ensure that data is correct.

Additional quality assurance is provided at the later stages by utilising expert knowledge within the Scottish Government.

Data quality and assurance measures used for June Census area data are contained in Final Results from the 2016 June Agricultural Census.

Provisional Estimates - published on 6th October 2016

The provisional estimates were derived from yield values of individual growers collated by several industry bodies. More information on the methodology and results of the 2016 first estimates of the cereal and oilseed rape harvest can be found in the first estimates of the cereal and oilseed rape harvest release.

Relevance

The degree to which the statistical product meets user needs for both coverage and content.

The cereal estimates are produced for a wide range of purposes. The statistics help the government to form, monitor and evaluate policy, and to assess the economic well-being of the cereal sector. They are also required by law by the Statistical Office of the European Communities, as the information is essential for management of the EU markets. These early provisional estimates are timed to enable provision of data for an EU regulatory deadline. Specific regulations are listed on pages 3 to 5 of our 2014/16 annual statistics plan.

The production estimates also feed into the UK cereals balance sheet, which provides an independent, unbiased, timely and comprehensive picture of the supply and demand position of the UK cereal market. The balance sheet is also the prime tool for tracking new developments in the UK cereals industry and determining their impact on the market. The balance sheet is widely used by policy makers, the EU Commission and the wider cereals industry. The balance sheets are published by the Home Grown Cereals Authority (HGCA).

User Feedback

Though we are not aware of any unmet user needs in relation to these statistics, the Scottish Government is always interested to here from users about what is most relevant to them and welcomes feedback from users of these statistics. Contact details are available from the Agriculture Statistics contacts webpage.

Details of both current and past user consultations are available on the Agriculture Statistics consultations webpage.

Accuracy

The closeness between an estimated result and the (unknown) true value.

The number of agricultural holdings surveyed in the CPS was 628 in 2016. Usable returns were received for 342 of these; a response rate of 54 per cent. Although 342 holdings participated, many holdings grow more than one crop. The total number of returns received for all crops combined was 691, this equates to five per cent of cereal crop numbers, and nine per cent of the relevant planted areas in Scotland.

The results from the CPS have a margin of error associated with them, reflecting the error resulting from sampling. Sampling error is the difference between the estimate derived from a sample survey and the true value that would result if a census of the whole population were taken under the same conditions.

The sampling error can be estimated and used to produce confidence intervals around the survey results. These intervals tell us the range of values within which the true value lies, with a given degree of confidence. The intervals below are 95 per cent confidence intervals; this means that if the sample survey was repeated a large number of times, 95 per cent of the resulting estimates would lie within the intervals around our sample estimates. For example, there is a 95% chance that the true production value for all cereals in Scotland will lie within the range of 2.752 million tonnes ±120,000 tonnes. Charts A and B, below, show the main production estimates marked with the upper and lower bounds of the associated confidence intervals. This is shown on two charts with different scales to allow results to be viewed clearly.

Table A - 95% Confidence Intervals for 2015 CPS Estimates

Crop	Number of Holdings (June Census)	Sample Size	Sampling %	Production ('000 tonnes)	Confidence Limits ('000 tonnes)	Confidence Limits (%)
Total Cereals1	18,614	628	7.29	2,752	±120	±4.36
Spring Barley	7,154	275	3.84	1,296	±39	±2.99
Winter Barley	2,009	100	4.98	329	±20	±6
Wheat	2,846	148	5.20	926	±41	±4.43
Oats	1,783	93	5.22	201	±20	±10.18
Oilseed Rape	1,058	63	5.95	102	±9	±9.27

Charts A and B - Production Estimates with Confidence Intervals: 2016

Area data are sourced from the June Agricultural Census and are assumed to be accurate as farmers face financial penalties for supplying incorrect information.

Comparison of provisional and final results

This section compares past provisional estimates of the harvest to the final estimates of the harvest. Provisional estimates are derived from averaged yield estimates of growers, collated through the cooperation of several organisations within the agricultural sector, applied to crop area estimates from the June Agricultural Census. Final estimates are derived from average yields from the Cereal Production Survey (CPS). The purpose of this section is to quantify the size and direction of the differences between the two estimates in order to give an indication of the robustness of these provisional estimates.

The results from the CPS have a margin of error associated with them, reflecting the error resulting from sampling. Sampling error is the difference between the estimate derived from a sample survey and the true value that would result if a census of the whole population were taken under the same conditions. The intervals were calculated as 95 per cent confidence intervals, meaning that there was a 95 per cent chance that the true population value was within the resulting interval.

The 2016 first estimates of overall production, and for each of the crops, were within these 95 per cent confidence intervals. This suggests that initial 2016 figures from the Crop Report Meeting provided a reasonable estimate of Scottish cereal production.

It can be seen from Chart C that in the last ten years the provisional estimate of the total cereal harvest has been within five per cent of the final estimate. In 2016 the difference between total cereal estimates from the two sources was 0.8 per cent. In most years, the largest differences between provisional and final production estimates are for oats, with the largest difference being 17 per cent in 2012. This year the largest difference was 4.7 per cent for winter barley.

Chart C: Cereal Production, Comparison of Provisional v Final Estimates, 2007 to 2016 (final estimates minus first estimates, as percentage difference)

Revisions

There are no revisions to previous years' data this year

Timeliness and Punctuality

Timeliness refers to the lapse of time between publication and the period to which the data refer.

To provide reliable estimates of the year on year changes in production, the CPS is carried out at the same time each year. The reference date for the CPS, the date at which respondents are asked for production information, is the 31^st October each year. However, respondents are asked to make estimates for any crop still to be harvested. Typically, at the end of October the vast majority of the Scottish cereal and oilseed rape harvest is complete, allowing for reliable estimates to be made.

The release of results is completed within six weeks of this date, to allow sufficient time for data collection, processing, quality assurance and compilation and dissemination of final results.

Punctuality refers to the time lag between the actual and planned dates of publication.

The results of the 2016 CPS were released on the scheduled date of 14^th December 2016.

Accessibility and Clarity

Accessibility is the ease with which users are able to access the data. It also relates to the format(s) in which the data are available and the availability of supporting information.
Clarity refers to the quality and sufficiency of the metadata, illustrations and accompanying advice.

These statistics are made available online at the Scottish Government's statistics website in accessible formats (html and pdf versions are available)
www.gov.scot/agricstats
All data tables are made available in excel format to allow users to carry out further analysis. Methodological notes and additional notes to tables, identifying specific quality issues, are included in this document, which is available online and linked to from all National Statistics outputs containing cereal production estimates. Links to other UK Agriculture Statistics outputs are available at
www.gov.uk/government/organisations/department-for-environment-food-rural-affairs/about/statistics

Comparability

The degree to which data can be compared over time and domain.

Results for England, Wales and Northern Ireland are compiled on a comparable basis with Scottish estimates.

The EC regularly produces estimates of cereal and oilseed production both EU-27 countries and individual countries. Further information on EC cereal statistics is available at the Eurostat website.

Typically EC results are published later than Scottish or UK results due to the additional time required to collate, validate and analyse data from several countries. Users interested in comparing results between countries should evaluate the relevant methodologies of sources used.

Respondent Burden (the estimated overall cost to respondents)

The estimated respondent burden is calculated as the total number of survey responses (A), multiplied by the median time taken to respond to the survey (B), multiplied by the median average hourly wage of typical respondents (C).
(A x B x C)
(A) The 2016 Cereal Production Survey (CPS) surveyed 342 holdings.
The time taken to respond to the survey varies with each respondent. Scottish Government (SG) Rural Payments and Inspections Directorate (RPID) staff conducting the 2014 survey were asked to provide estimates of the average time taken to administer the telephone survey. The median time to respond in hours was calculated from these responses.
(B) The median time taken to respond to the survey is 0.083 hours.
Respondents to the CPS are usually farm owners themselves or farm managers. Both are usually full-time workers.
(C) The estimated median hourly pay rate for full-time employees in Scotland in 2016 was £9.53 (source: Scottish Agriculture Hours and Earnings Survey)
The respondent burden for CPS data collection in 2015 was
342 X 0.083 X £9.53
= £271

Related publications

First estimates of the Cereal and Oilseed Rape Harvest

Economic Report on Scottish Agriculture (ERSA) contains Cereal usage figures derived from the CPS survey. These were last published in June 2014.

Agriculture statistics publications contains all published results from Scottish Government agricultural surveys.