Cereal and oilseed rape harvest - first estimates: 2017

7. Methodology and Quality Note

This section provides a summary of information on these statistics against five dimensions of quality, based on the European Statistical System ( ESS) quality framework: Relevance; Accuracy; Timeliness and Punctuality; Accessibility and Clarity; and Comparability. The Scottish Government adheres to the Code of Practice for Official Statistics and the National Statistician's guidance on quality. In addition the Scottish Government provides its own guidance on quality, which is available to view at the Scottish Government's Statistics internet pages.

Further information on quality:

• Code of Practice for Official Statistics
• National Statistician's Guidance on Quality
• Scottish Government's Corporate Policy Statement
• Scottish Government Guide to basic quality assurance
• European Statistics Code of Practice (including quality framework)

Data Providers

The Scottish Government relies on the data collected by industry bodies to produce these statistics and therefore the quality of the data available from these sources impacts significantly on Scottish Government analysis.

The provisional estimates are derived from yield values of individual growers collated by several industry bodies. These industry bodies meet to discuss and quality assure these estimates at the annual Crop Report Meeting, which in 2017 was supported by representatives from:

• Scottish Government, Rural and Environment Science and Analytical Services
• Rural Payments Agency
• Bairds Malt
• Scottish Agricultural Organisation Society
• Scotland's Rural College
• Agricultural Industries Confederation
• National Farmers Union Scotland
• The Agriculture and Horticulture Development Board
• Science and Advice for Scottish Agriculture

First estimates from growers are collected by several means, by: area offices of the Scottish Government ( SG) Rural Payments and Inspections Directorate ( RPID); area offices of Scotland's Rural College ( SRUC); agronomists working for commercial bodies; farming co-operatives; the National Farmers Union Scotland ( NFUS), using electronic, paper based or telephone surveys.

Once all the yields have been collated, the industry bodies at the Crop Report Meeting carry out additional quality assurance by comparing resulting yields between different crops and regions within Scotland. This results in an agreed set of yield estimates which are then combined with June Agricultural Census area results to derive the harvest production estimates.

Relevance

The degree to which the statistical product meets user needs for both coverage and content.

The cereal estimates are produced for a wide range of purposes. The statistics help the government to form, monitor and evaluate policy, and to assess the economic well-being of the cereal sector. They are also required by law by the Statistical Office of the European Communities, as the information is essential for management of the EU markets. These early provisional estimates are timed to enable provision of data for an EU regulatory deadline.

The production estimates also feed into the UK cereals balance sheet, which provides an independent, unbiased, timely and comprehensive picture of the supply and demand position of the UK cereal market. The balance sheet is also the prime tool for tracking new developments in the UK cereals industry and determining their impact on the market. The balance sheet is widely used by policy makers, the EU Commission and the wider cereals industry.

User Feedback

Though we are not aware of any unmet user needs in relation to these statistics, the Scottish Government is always interested to here from users about what is most relevant to them and welcomes feedback from users of these statistics. Contact details are available from the Agriculture Statistics contacts webpage.

Details of both current and past user consultations are available on the Agriculture Statistics consultations webpage.

Accuracy

The closeness between an estimated result and the (unknown) true value.

When considering the accuracy of these statistics it is important to note that the collection of estimates prior to the completion of the harvest will lead to a level of inaccuracy. Given the timing of the Crop Report Meeting and the unpredictability of the weather and the knock-on effects this has on production ( e.g. poorer yields in later harvested crops which have been subjected to longer periods in unfavourable growing conditions), providing early estimates of cereal yields and production is a challenging task. These estimates are based on the most up-to-date industry reports available at the time of the Crop Report Meeting (this year the meeting was held on the 25 th September). As harvest progress continues it is inevitable that these reports will become more reliable. These statistics are provisional estimates and will be followed by final estimates in December 2017.

In the last 10 years the provisional estimates of the total cereal harvest has been within five per cent of the final estimate – see chart 17.

The nature of the industry bodies involved in the production of provisional estimates means that the results are likely to be more representative of commercial cropping farms, and less representative of farms growing crops for on farm uses, this is most likely why the provisional estimates tend to overestimate production and yields compared to the final Cereal Production Survey ( CPS) estimates.

Comparison of provisional and final results

This section compares past provisional estimates of the harvest to the final estimates of the harvest. Provisional estimates are derived from averaged yield estimates of growers, collated through the cooperation of several organisations within the agricultural sector, applied to crop area estimates from the June Agricultural Census. Final estimates are derived from average yields from the CPS. The purpose of this section is to quantify the size and direction of the differences between the two estimates in order to give an indication of the robustness of these provisional estimates.

The Cereal Production Survey is based on a sample of around 400 to 550 farms in Scotland, stratified by region. In 2016, 630 agricultural holdings were surveyed, with many holdings growing more than one crop. Usable returns were received from 342. The total number of returns received for all crops combined was 691, equating to a sampling rate of five per cent of holdings and nine per cent of the relevant planted area.

The results from the CPS have a margin of error associated with them, reflecting the error resulting from sampling. Sampling error is the difference between the estimate derived from a sample survey and the true value that would result if a census of the whole population were taken under the same conditions. The intervals were calculated as 95 per cent confidence intervals, meaning that there was a 95 per cent chance that the true population value was within the resulting interval. The 2016 first estimates of overall production were within these limits – suggesting that the provisional estimates were an accurate assessment of the 2016 harvest. More information on the quality of the final estimates and the differences between first and final estimates are contained in the Final Estimates of the Cereal and Oilseed Rape Harvest 2016 release.

Final estimates undergo several validation processes as follows; (i) checking for any obvious errors on the paper survey forms upon receipt, (ii) cross checking against June Agricultural Census area data and internal validation within survey forms to ensure totals match, (iii) results are standardised to 14.5 per cent moisture content for cereals and nine per cent for oilseed rape (iv) assessing data for any extreme yield values and removing if necessary, (v) if required, area offices are contacted to ensure that data is correct. Additional quality assurance is provided at the later stages by using expert knowledge within the Scottish Government.

Data quality and assurance measures used for June Census area data are contained in Final Results from 2016 June Agricultural Census.

In previous years, the provisional June Agricultural Census area figures used to calculate the provisional production estimates have been slightly different from the final June Agricultural Census areas used to calculate the final production estimates. However, these differences have generally been small (less than one per cent) and are not a main contributor to differences in the production estimates.

The main reason for differences in the provisional and final production estimates are differences in provisional and final yield estimates. Chart 17 shows these differences for the last ten years. A full breakdown of the differences between the estimates is provided in Table 4.

It can be seen from Chart 17 that in the last ten years the provisional estimate of the total cereal harvest has been within five per cent of the final estimate. In most years, the largest differences between provisional and final production estimates are for oats, with the largest difference being 17 per cent in 2012.

Chart 17: Cereal Production, percentage comparison of Provisional v Final Estimates, 2007 to 2016
(positive numbers mean the final estimate was higher than the provisional)

Timeliness and Punctuality

Timeliness refers to the lapse of time between publication and the period to which the data refer.

In order to provide timely estimates of cereal and oilseed rape production the Crop Report Meeting ( CRM) is held as early as is practical following data collection by the industry representatives attending the meeting. The progress of the harvest limits how early this meeting can be held. Yield estimates collected before the completion of the harvest are inevitably less reliable than estimates completed after the harvest. The 2017 CRM was held on the 25 th September. The results were published under National Statistics protocols on the 4 th October.

Punctuality refers to the time lag between the actual and planned dates of publication.

When reliable results can be expected, the planned publication date of the first estimates of the cereal and oilseed rape harvest is scheduled no later than the release date of UK harvest estimates, as both feed into the UK cereals balance sheet.

Accessibility and Clarity

Accessibility is the ease with which users are able to access the data. It also relates to the format(s) in which the data are available and the availability of supporting information.
Clarity refers to the quality and sufficiency of the metadata, illustrations and accompanying advice.

These statistics are made available online at the Scottish Government's statistics website in accessible formats (html and pdf versions are available). Data tables are made available in excel format to allow users to carry out further analysis. Methodological notes and additional notes to tables, identifying specific quality issues, are included in this document, which is available online and linked to from all National Statistics outputs containing cereal production estimates. Links to the Agriculture Statistics series of outputs are available from the Gov.uk website, www.gov.uk.

Comparability

The degree to which data can be compared over time and domain.

The first estimates of the cereal and oilseed rape harvest (from the Crop Report Meeting) contained in this document are compared to final estimates (from the Cereal Production Survey) for previous years.

Due to the typically later harvest period in Scotland compared to the rest of the EC, it is not practical to survey farmers for production and yield estimates at this point in the year. EC regulations governing the collection of cereal and oilseed estimates account for this by allowing early estimates to be collected by other means than a survey. Results for England, Wales and Northern Ireland, also released in October, are based on provisional results from surveys similar to that of the Scottish Cereal Production Survey. The latest results of the UK cereal and oilseed rape harvest, including Scottish estimates, are available from the Gov.uk website, www.gov.uk.

The EC regularly produces estimates of cereal and oilseed production as both totals of EU-27 countries and individual countries. Further information on EC cereal statistics is available at the following website:
http://epp.eurostat.ec.europa.eu/statistics_explained/index.php/Agricultural_products . Typically EC results are published later than Scottish or UK results due to the additional time required to collate, validate and analyse data from several countries. Users interested in comparing results between countries should evaluate the relevant methodologies of sources used.

Related publications

The final estimates of the 2017 Scottish cereal and oilseed rape harvest are due to be published in December and will be accessed here: Agriculture and Fisheries - Publications

Cereal usage figures have been published in the Economic Report on Scottish Agriculture ( ERSA). These were last published in June 2017, and can be accessed here: Agriculture and Fisheries - Publications

Results from all Scottish Government agricultural surveys can be accessed here: Agriculture, Fisheries and Rural - Publications