Scotland's Census 2021 Topic Consultation

National Records of Scotland is planning for Scotland’s Census 2021. A lot can change in the 10 years between censuses. To help inform our planning, this consultation will seek information from users about their needs. This will help determine the topics to be included in the next census.


5. Evaluation criteria

The evaluation criteria to be used for Scotland's Census 2021 topic consultation are broadly the same as those used for 2011. However, some changes have been made in order to make the evaluation criteria more robust and transparent and to take into account the move to a primarily online census.

The criteria relating to user requirements remain the key criteria for evaluation. Operational requirements strengthen the case for inclusion of topics, especially if a topic is thought to improve coverage.

The considerations criteria will predominantly be used in conjunction with the strength of user requirements score to steer the development of the Census questionnaire. However, issues identified using the considerations criteria may mean a topic is not included in Scotland's Census 2021 despite a known user or operational need (for instance if we cannot develop a question that respondents can accurately answer).

User requirements

✓ Strength of user need

✓ Need for information for small geographies or populations

✓ Suitability of alternative sources

✓ Need for multivariate analysis

✓ Need for UK comparability

✓ Need for continuity with previous Censuses

Considerations

✓ Data quality

✓ Public acceptability

✓ Respondent burden

✓ Financial concerns

✓ Questionnaire design

Operational requirements

✓ Improving coverage

✓ Coding of derived variables

5.1 User requirements

Topics must carry a strong and clearly defined user need. A robust case must be made, or exist, for topics to be included in the 2021 Census. Responses from users on the importance of the information to them will help inform the final list of topics to be included in the Scotland's Census 2021.

Criteria

Description

Strength of user need

Strength of user need, identified primarily by the purpose it serves, is central to the evaluation process. User need could be justified in terms of, for example: significant resource allocation, improved service provision, major policy development or policy monitoring.

Additionally, requirements arising from national or international legislation strengthen the case for inclusion. For example, requirements of the UN, UN Economic Commission for Europe and Eurostat will be taken into account.

Need for information for small geographies or populations

There must be a need for information for small population groups and/or at detailed geographical levels. Information that is only required for broad geographic areas or large population groups might be better obtained from other statistical sources.

All small geography data must easily be aggregated to higher levels of geography, enabling outputs to be consistent and comparable regionally and nationally.

Suitability of alternative sources

The 2021 Census will only seek to collect information that there is no other sufficient means of obtaining. Consideration should be given to whether or not suitable information is available from other sources; for example administrative records or sample surveys.

In addition, the Census should only collect information that will remain relevant for a significant time after collection due to the time lag between census night and the publication of results.

Need for multivariate analysis

A key benefit of census data is the ability to analyse particular variables in conjunction with one or more others. A requirement to undertake multivariate analysis will affect whether suitable alternative sources for the information exist and can strengthen the case for inclusion in the Census.

Need for UK comparability

A requirement to obtain comparable data for a topic or sub-topic at a UK level is considered. Where possible, if there is a user need, the three UK Census Offices will release Census outputs that are consistent across the UK.

Need for continuity with previous Censuses

Comparison with previous censuses is an important aspect of census analysis and, wherever appropriate, the 2021 Census questionnaire will collect the same information as the 2011 Census questionnaire.

However, other aspects of user needs may override the need for comparability.

5.2 Considerations

The census is a compulsory survey carried out on a self-enumeration basis; each respondent is required to complete all relevant questions on the questionnaire. The following considerations are therefore important.

Criteria

Description

Data quality

The data collected in the 2021 Census should be expected to be of sufficient quality for outputs to be robust and useful. Hence information should not be collected that is not reliable or replicable, or causes difficulty for people to answer accurately.

Public acceptability

The census needs to carefully consider whether sensitive or potentially intrusive questions should be asked that may have a negative impact on response rate, or lead to respondents giving socially acceptable rather than accurate answers. It should also not enquire about opinions or attitudes.

Respondent burden

The inclusion of questions on a topic should not impose an excessive burden on respondents. Burden could, for example, result from lengthy instruction or explanation, large numbers of response categories, or large numbers of questions on a single topic.

Financial concerns

The cost of including a topic or question that presents major difficulties for coding the information, or requires extensive processing, must be considered carefully. The collection of information on a topic and/or question should not significantly add to the overall cost of the Census.

Questionnaire design

The move to a predominantly online data collection creates new opportunities as well as challenges. These will be taken into account when considering which topics to include in the 2021 Census.

Although the primary mode of data collection will be online there will also be a paper questionnaire, therefore space and design considerations on the paper questionnaire and comparability between the two modes will also be considered.

5.3 Operational requirements

A number of operational factors could also affect the content of the 2021 Census. The most important of these is improving coverage of the population, as the primary aim of the census is to provide a robust count of the population of Scotland.

Criteria

Description

Improving coverage

The 2021 Census questionnaire may include some questions that aid respondents in identifying who should be included in the response; for example questions on visitors and defining the types of usual resident that live in the household. Such questions may be included despite a lack of strong user need for outputs related to these questions, as they are key to delivering high quality outputs.

Coding of derived variables

It is expected that some key outputs will be based on derived variables (for example age derived from date of birth). Variables may be included in the 2021 Census questionnaire if a user need is identified for data that is derived from a variable, even if there is no user need for outputs relating directly to the question.

Contact

Email: Cecilia MacIntyre

Back to top