Scotland's Census 2022: equality impact assessment - version 3

Annex B - Online Collection Instrument: Research

Accessibility Testing Research Summary

The Online Collection Instrument (OCI) delivers the core question set for the census. There was a need to conduct some initial accessibility testing on the early version of the site to identify any early issues. The User Centred Design UCD team conducted a round of accessibility testing with 10 participants with a variety of disabilities.

Around half of the participants were visited in their home in order to allow for the use of any specific assistive technology such as screen readers and magnifiers and to help make them more comfortable.

From the OCI, we tested:

• On-boarding – entering Internet Access Code (IAC), setting up a password and password recovery.
• Questions H1-5 – details of who is in the household

Participants ranged from 20 to 61 years of age and had a range of disabilities and assistive technology needs, including dyslexia, dyspraxia, autism, blindness, arthritis, detached retina, macular degeneration, Friedreich's ataxia.

Key findings

The majority of participants found it relatively straightforward to get through the on-boarding and H1-5 sections of the prototype. They were very positive about the idea of completing the census online as it meant they would be able to complete it in a format best suited to them, save progress and complete it in their own time.

However, there were a number of overarching issues that affected all types of users. These were predominantly usability issues, but some assistive technology specific issues were encountered as well. Usability issues will cause problems for all users, but they are heightened for disabled people and users of assistive technology. Causing confusion, frustration and being slowed down can have a significant impact on these users. In some situations it can cause stress/anxiety and lead them to seek support with completion or make them feel like they don't wish to continue. Many of the participants had to stop to ask the moderator what was meant or required and some needed to be told how to progress to the next step.

Additionally, while the questions in this version of the prototype were not the final 2022 question set, a number of usability issues were uncovered that will be important to consider independent of the question content and can be worked on and resolved for subsequent rounds of usability testing.

The key issues identified included:

• unclear error messaging when creating a password
• 'set up password recovery' usability and error messaging
• users having difficulty understanding what's being asked for in the 'temporarily away' question
• users having difficulty understanding what is being asked for in the 'visitors' question
• confusion caused by the 'dashboard' when users land on it for the first time
• unclear or missing instructions or supporting text – e.g. needing to use capitals and hyphens in the IAC code
• the IAC code was challenging for many

Findings specific to assistive technology were identified:

• Zoomtext issues: participants who used screen magnifiers appreciated the amount of space on each page and having single questions per page.
• Screen reader issues: participants who used screen readers had very different experiences. This could be for a number of reasons such as version of software, device used or level of users' experience with assistive technology.

Additional findings:

• Some users may not be able to read the letter, the service needs to consider how blind people living on their own can be supported.
• Some users will require a large print letter to be able to read the IAC number, consider how they can be easily identified and provided with the right information. Is there any way of knowing about this need before sending the letters out?
• When the initial letter will be sent out to give citizens enough time to get right version of information that they need, and also how any reminder services may come through to them (i.e. reminder letters are inappropriate for a blind person).
• Some users will misplace or lose the letter and so how they can still access online without the IAC code will need to be considered and made clear to users.

Information Needs User Research Summary

This research, to understand whether the digital version of the 2022 Census meets the needs of citizens, comprised of multiple rounds of usability and accessibility testing from December 2018 – March 2019, conducted by the Scottish Government Digital Transformation Division's User Centred Design team. While these sessions predominantly focussed on the use of the digital components (i.e. the website portal and the 'online collection instrument' (OCI)), we also discussed what information participants expected or felt they would need in order to take part in the census at various points, including showing a version of the letter to set context.

The main findings from these sessions are regarding usability and accessibility, which have been reported via the 'OCI User Testing' strand of work, but broader insight around information needs have also been uncovered. In order to further understand information needs for citizens we utilised a number of sessions to explore the following objectives:

• Understand what information is required by a user to support them in their census experience.
• Understand what channels users expect to be able to access information to support them in their census experience
• Review of the current language and terminology to determine whether it supports the user's understanding of the information they need in order to complete the census
• Understand how the user's need for information changes over their census collect experience.

Key findings:

• Standard patterns for question pages: reduces cognitive load which may quicken response
• Explanation and narrative to set expectations and orientate users within the form would benefit some users
• In a small number of cases there was misunderstanding of the questions
• Type ahead functionality for industry and occupation questions caused particular problems in choosing an answer
• Hard validation meant users got stuck in a loop where they could not answer a question and so could not submit their census response
• Problems accessing Help and Support while in the questionnaire
• Age related question routing meant some questions are asked of children within the household that are not relevant for a child
• Misunderstanding of questions or uncertainty about how to answer.

Findings are informing further development of OCI and the question engine to maximise respondent ease and minimise respondent burden.

Audience Discovery Research - Qualitative research among 'seldom heard' audiences

• To investigate their needs and inform the design of the process and website for Scotland's Census 2022.
• Research to provide a deep and robust understanding of user needs – with a specific focus on those users who have specific situations for Census collection or who require Assisted Digital support.
• The purpose is to ensure that these users' needs are accurately represented in the design of the OCI: -
  • Flat / House Sharers
  • Communal Establishments
  • Halls of Residence
  • Care Homes
  • Communities with Reduced Links
  • Ethnic Communities (sample included Somali, Roma, African, Romanian, Kurdish)
  • Religious Communities (sample included Sikh, Muslim)
  • Other Communities (Camphill)
  • Skill Limitations
  • Digital Disengagement
  • Low Literacy Skills
  • Reading Impairment
  • Supported Applications
  • English Language Limitations
  • Gaelic Speakers

With a focus on exploration and discovery, the detailed research objectives were:

• Develop a deep knowledge of who the service users are in terms of their circumstances, situations, attitudes, skills, abilities (as appropriate)
• In relation to officialdom generally; in relation to the Census specifically
• To what extent information and support are/are not accessed
• Understand motivators and barriers to completing the Census
• Comprehension of the Census; its (perceived) importance
• Personal obligations
• Identify the support and interventions that would facilitate participation in the online Census
• Information needed, support needed, enumeration needs
• Explore perceptions of and reactions to the OCI design
• Aspects that help and hinder completion
• Identify how the above should be reflected in the OCI, and in general, to ensure a successful Census

Situations and skills

• Broad spectrum of situations: some people have thrived in Scotland, some have struggled.
  • Positive experiences can inspire appreciation of Scotland / its government, and willingness to comply with officialdom.
  • But some feel let down by the system and less willing to comply.
  • Others fear the authorities and sharing personal information.
• Broad spectrum of skills and skill levels across Digital, English Language, Reading Impairment, Literacy.
  • Low skills don't seem to necessarily correlate with a less positive life experience.
  • Key factor seems to be whether the family or household unit perceives it's 'doing well'.
  • Possible exception: low Literacy Skills might have a stronger correlation with feeling let down by the system, that life has been impoverished (unnecessarily).

Support

• People with lower skills tend to have an established support network
  • Family, friends, Community Leaders and Organisations etc they trust and turn to for help / advice.
  • Many need 'hands-on' support with officialdom in general, English translation, reading and writing, or digital activity.
• Hands-on support is likely one of the most important success factors for the Census
  • Providing the skills that are missing…and also overcoming low motivation / mistrust / scepticism.
• Reaching out to Community Leaders and Organisations is vital
  • They are close to their communities, strong advocates of the Census, and very keen to help.
• Like the general population, some people have stronger skills and will do well with 'self-serve' support available from their network or on the website.

Attitudes to the Census

• A spectrum of engagement with the Census.
  • A few 'evangelists', e.g. Community Leaders and Managers of Community Establishments.
  • Some people are mistrustful or resistant, likely due to personal experience.
  • Most seem to be around the mid-point: willing to carry out their legal obligation, but might not perceive the value of the Census.
• Opportunity to improve engagement for the greater success of the Census – quality of experience for the public as well as quality and accuracy of information.
  • Many simply don't know much about it – what it's for, what it stands for, the difference it can make.
  • Improving understanding tends to improve engagement.

The letter

• The letter presents significant difficulties for people with lower skills
  • What's it about? What does it mean? What do I need to do? Do I have to?
• Indeed, for many, the letter is 'daunting' to read – many won't see it through to the second page and will seek help.
• It conditions expectations that participating in the Census will be difficult – too difficult to attempt.

The website

• By contrast, when people see the design for the website, confidence builds.
• Some now feel they will 'have a go' under their own steam, seeking help if needed, and look forward to a sense of achievement.
  • So, it's important to make the process (seem) easy in order to optimise autonomous participation.
• The website design creates a pleasing and reassuring atmosphere.
  • Excellent use of colour, space, imagery (people), and nuggets of text to increase engagement and reduce cognitive effort – and suggest it's easy.
  • Wears its official credentials lightly.
  • Surprisingly and positively different from the letter and much official communication; should play a central role in shaping engagement.

Overall, there is enough evidence to suggest that, with sufficient and appropriate support and communication, these 'seldom heard' audiences are very likely to engage with Scotland's Census 2022.