The Anholt Nation Brands Index®: 2024 Report For Scotland Methodology Report

Quality considerations

Fieldwork dates and fieldwork quality control

Across all countries, 42,107 interviews were conducted online with at least 2,000 interviews per country for the 2024 NBI Survey. Of these, 10,162 interviews about Scotland's reputation were conducted. Fieldwork was conducted in July and August 2024 for all countries, excluding 3,004 interviews (500 about Scotland) with Russian participants from the 2023 NBI due to international sanctions that prevented data collection with the Russian panel in 2024.

Survey data validation and cleaning was an essential part of the quality control procedures. Unlike telephone or face-to-face surveys which are administered by trained interviewers, online surveys are self-administered and are therefore at a higher risk of respondent error or falsification. The quality principles and documentation that were applied with global consistency throughout NBI 2024 were set by experts. Quality assurance measures began at the panel level and include IP address validation and risk assessments, digital fingerprinting, bot detection, and checks for duplicate respondents. Further measures were taken within surveys, such as the detection and removal of "speeders" (respondents who complete the survey in an unreasonably fast amount of time) and "straight-liners" (respondents who click through the survey giving the same answers for each item). These measures were supplemented by repeated quality checks and data cleaning throughout the project lifecycle.

General panel maintenance and panel quality control

For online panel quality control in general, ongoing strict verification and cleaning rules were in place including the removal of lapsed panellists and undeliverable emails. The ongoing monitoring system of the panels also identified potential fraudulent behaviour based on survey-taking activities and patterns; offending panellists were then removed from the panel. At the same time, the panel supplier continually recruited new members to ensure that the panel was dynamic in nature and reflected the changes in online population composition over time.

Panellists were only contacted for the purposes of conducting market or survey research. They were not exposed to third party advertising or direct marketing campaigns, nor was their personal data sold to third parties. Also, a real-time support help desk was provided to panellists in their native language to solve any technical problems that might have occurred and to ensure the experience in survey participation was satisfactory to panellists.

Panel operators employed a variety of incentive approaches, while obeying rules and laws of the panel countries, and based on what works best within a given country. Such incentives included donations to charities on behalf of panellists, sweepstakes, points, gift cards, prizes, music downloads and cash payments.

The survey was conducted in full compliance with GDPR as well as all other relevant global regulations regarding data privacy and security.

Sample balancing and weighting

For NBI 2024, multiple outgoing sample replicates were carefully spaced across the field period so that the achieved total sample in each country included early and late responders, as well as weekday and weekend responders. This procedure ensured sufficient fielding periods for each launched replicate to ensure responses from people who had different time availability (reflecting age, lifestyle and working status differences) and to minimise early responder biases.

It is important to highlight that the NBI assessed the perceptions of the online population of each of the panel nations. As such, samples were recruited to reflect the composition of the online population rather than that of the overall population of the country. Using the most up-to-date online population parameters, the achieved sample in each country was weighted to reflect key demographic characteristics such as age, gender and education of the online population in that country. Additionally, in the United States, the United Kingdom, South Africa, India and Brazil, race/ethnicity was used for sample balancing. The report reflects the views and opinions of online populations in these 20 countries. Weights for each country were calculated by Anholt & Co.

As the achieved sample sizes may vary slightly across survey countries, when computing the aggregated 20-country average, "one-country-one-vote" weighting was applied so each survey country was treated equally in the global average.

Analyses

Data processing and statistical analyses were performed by Anholt & Co. The NBI score is an average of the scores from the six NBI indices, and each of the six indices is an average of the scores of the ratings questions from the corresponding area. There are between three and five ratings questions for each of the indices, as outlined above.

Scottish Government's quality assurance processes

The Scottish Government received the analysed data from Anholt & Co. in October 2024, and upon receipt of the data Scottish Government analysts performed rigorous quality assurance checks to make sure that they were as accurate, reliable and coherent as possible.

Voluntary compliance with the Code of Practice for Official Statistics

Scotland's NBIreport is not published as Official Statistics, however for the fourth time, Scottish Government analysts have decided to voluntarily comply with some aspects of the Code of Practice for Official Statistics.

The approach is flexible and entirely voluntary. Scottish Government analysts have reviewed the approach to producing and publishing the Nation Brands Index^® report in relation to the three pillars (Trustworthiness, Quality and Value) and considered what is already being achieved and whether there are ways of improving practice. Compliance with the code relates to the handling of data once it has been received by the Scottish Government.

A statement has been published in Annex A within this methodology document to outline why users can be reassured that what the Scottish Government has produced achieves Trustworthiness, Quality and Value.

Limitations

The Scottish Government purchases this data from Anholt & Co. Therefore the Scottish Government has no input in to how the survey is designed or carried out and also has limited insight in to how the data is analysed at Anholt & Co.

Reporting

Report writing was carried out by Scottish Government analysts and reviewed by Anholt & Co before publication.^[7]