Reducing health harms of foods high in fat, sugar, or salt: consultation analysis

Annex 3: Data Cleaning

Cleaning the data

Prior to analysis, missing answers to closed questions were reviewed and, where written comments were provided, these were used to interpret the respondent’s answer to the closed question. Where the wording was explicit (i.e. matched the wording of an answer option), the appropriate answer was inserted. Where the meaning was implied (i.e. was not explicit but meant the same as an answer option), the response ‘inferred – yes’ or ‘inferred – no’ (as appropriate to the answer wording) was inserted. 

When analysing the data it was apparent that few respondents found some of the wording for the closed questions difficult to understand and they, therefore, struggled to select a response that best represented their views. In short, their written explanation did not match the answer option selected. Again, their written answers were used to interpret their intended response. 

The rationale for amending responses demonstrating misunderstanding of the questions was:

• To ensure that the quantitative results accurately reflect the views of respondents;
• To ensure that where misunderstanding within the quantitative results could not be corrected, it was flagged for consideration;
• To enable the analysis to describe trends more accurately; and
• To ensure the views submitted could be included in the analysis.

Where misunderstanding was clear, the original answer was changed to ‘inferred – yes’ or ‘inferred – no’ as appropriate to the answer option. Where the selected answer option did not match the written comments but the written comments were not sufficiently clear to confidently correct the response, their original answer was changed to ‘conflicting answer – yes’ or ‘conflicting answer – no’ as appropriate to their original answer option. Where the numbers of conflicting answers for a given question are small, these have not been expressly commented upon within the findings. Where the numbers of conflicting answers for a given question are relatively high, they have been commented on in more detail.

In a small number of cases where there was no written comment but the selected answer option did not match the overall flow of a respondent’s answers/opinions, their original answer was changed to ‘possible conflicting answer – yes’ or ‘possible conflicting answer – no’ as appropriate to their answer option. The purpose of this was to enable these issues to be flagged when describing the data. 

The overall impact of cleaning the data in the manner described above was reviewed. The impact varied from question to question - sometimes increasing the number of responses in favour of proposals; sometimes increasing the number of responses against proposals. However, data cleaning did not result in an overall shift in stance among respondents (i.e. the overall proportion for and against restrictions), nor did it alter the trends in the data (i.e. the relative popularity of each answer option within a question remained the same). The greatest impact was the better reflection of the views of those who are against mandatory restrictions. Due to misinterpretation of the question wording, in some cases their original selection indicated agreement with restrictions, when the written explanation demonstrated disagreement with restrictions.

A record was made of all changes made to the data set ensuring a robust audit trail.

Issues with the questions

This section provides further detail on the issues arising with interpretation of the questions, as well as the solutions applied in order to analyse the responses.

Question 2

Should this policy only target discretionary foods? [confectionery, sweet biscuits, crisps, savoury snacks, cakes, pastries, puddings and soft drinks with added sugar]   

Issue: While there were no specific issues with interpretation of the wording of the question, in error, via the survey software, more than one answer option could have been selected by respondents. Some did select more than one answer option and as a consequence the quantitative results of this question were not possible to interpret without cleaning the data. 

Solution: Responses were compiled and corrected:

• Where the options for ‘No – fewer’ and ‘No – no foods’ were both selected (but no other answer options were selected) the response was amended to ‘No – no foods’ on the basis that this answer option supersedes the concept of ‘fewer’. Where an explanation was submitted this was reviewed to ensure the change to the answer was compatible with their explanation.
• Where the answers conflicted i.e. the respondent selected ‘yes’ or ‘no – additional’ AND ‘No – fewer’ or ‘no – no foods’ any comments made were reviewed and the answer corrected to the appropriate option. Where there was no comment or the comments were not sufficiently clear to enable correction, the answer was recoded as 'conflicting answers'.
• The open comments were also reviewed where only one answer option was selected. Where the explanation did not match the answer selected, the answer was corrected to indicate the response that reflected their explanation. Where this was explicit, for example the respondent selected ‘no – fewer’ but stated ‘no foods should be targeted’, the answer option was corrected. Where it was less explicit but the meaning was the same, the response was recoded to ‘inferred’ (with the wording reflecting the appropriate answer option).

Questions 5, 7, 9 and 10

Overview: It was apparent that while the wording of the questions and answer options closely mirrored the wording used within the consultation paper, references to the concepts of applying restrictions and exemptions and also changing language to refer to ‘not’ applying restrictions created some confusion for few respondents. These areas of confusion are explored for each question in more detail below, followed by a description of the solution adopted (which was consistent across these questions).

Question 5: In relation to the foods being targeted, should this policy seek to: 
Restrict multi-buys
Restrict sales of unlimited amounts for a fixed charge
Not restrict temporary price reductions  
Not restrict multi-packs?

Issue: The wording of the question created confusion among some respondents particularly in relation to the last two items (i.e. to not restrict). For example there were instances where ‘yes’ or ‘no’ was selected for all four items and then an explanation was given such as ‘none of these should be restricted’.

Question 7: Should the restrictions apply to any place where targeted foods are sold to the public, except where they are not sold in the course of business (e.g. charity bake sales)?

Issue: There was a small amount of confusion in terms of how to answer this question when the viewpoint held on the first part (i.e. restrictions applying to any place) was different to the viewpoint held on the second part (i.e. except where they are not sold in the course of business). For example some respondents selected ‘no’ and then stated ‘exempt charity events only’. Some of those who did not agree with any restrictions being implemented also struggled to interpret how best to answer this question.

Question 9: Should restrictions to displaying targeted foods at end of aisle, checkouts etc., not apply where there is no reasonable alternative to displaying them elsewhere? 

Issue: There was some confusion in regard to how to answer with respect to an exemption - i.e. for this question selecting ‘yes’ meant the restrictions should not apply and selecting ‘no’ meant the restrictions should apply. This was the opposite of the meaning in previous questions - i.e. typically selecting ‘yes’ meant restrictions should apply and selecting ‘no’ meant the restrictions should not apply. For example, some respondents selected ‘no’ and then commented that no restrictions should apply.

Question 10: Should food marked as discounted because it is close to expiry be exempt from:
Positioning Restrictions (end of aisle, checkouts etc.)
‘Promotion of value’ restrictions?

Issue: The same issue arose as per question 9 with there being some confusion in regard to how to answer with respect to an exemption.

Solution (for Q5, 7, 9 and 10): The written explanations for each question were reviewed for misunderstanding:

• Where misunderstanding was clear, the answer was recoded to ‘inferred - yes’ or ‘inferred - no’ as appropriate to the comment made.
• Where misunderstanding/conflict was sufficiently clear to suggest that the answer was wrong but the explanation was not sufficiently clear to confidently ‘correct’ the answer it was recoded as ‘conflicting answer’ with the exact wording reflecting the original answer e.g. ‘conflicting answer - yes’ or ‘conflicting answer - no’.
• Where there was no explanation submitted for a given question but the answer option selected clearly conflicted with the respondent’s overriding view (to other questions) - for example the respondent was attempting to disagree with everything but has not accounted for the negative phrasing used in some of the wording of the questions – the answer was recoded as ‘possible conflicting answer’ with the exact wording reflecting the original response wording i.e. ‘possible conflicting answer – yes’ or ‘possible conflicting answer – no’.

Question 11

Question 11: Please list any other exemptions we should consider

Issue: Some respondents submitted responses indicating any other ‘restrictions’ instead of any other ‘exemptions’.

Solution: As the responses that indicated restrictions covered types of foods or types of promotions to restrict, the points had already been covered by a previous question and the response was therefore included at the relevant question and excluded from consideration in the write up for question 11.