Statistics.gov.scot improvement project: discovery user research report

Workshops

Four workshops were held from 30/04/24 to 21/05/24, with 18 SG staff taking part. Two workshops were held in-person at Victoria Quay in Edinburgh, and two were held remotely using Microsoft Teams. Two workshops were tailored towards SG data publishers and two for other SG users. The workshops were facilitated by Tom Farrington (User Researcher) from Storm ID, with three members of the SG Open Data Team joining to help facilitate, observe, and note-take as needed.

The overall aim of the workshops was to understand how a range of largely experienced internal SG users are using the site for providing data and working with data. The emphasis here was on smaller group work involving fairly in-depth user scenarios and tasks.

The workshop outlines and user journey mapping scenarios were agreed in consultation with the Open Data Team, and are included in Appendix A.

Anonymous data was generated in the form of notes typed by project team members into a secure Confluence Whiteboard, and (where in person) pen and paper sketches and notes by participants. This data was subject to a basic qualitative content analysis by the facilitator, being a version of template analysis as described in the Methodology section.

Preliminary analysis

The following analysis is preliminary in the sense of being drafted immediately after the workshops, and before the remaining programme of research. This led to a set of preliminary themes, each accompanied by illustrative quotations from the workshops. This analysis was presented to the wider Open Data Team for discussion on 22/05/24.

Preliminary theme 1: Statistics.gov.scot is jargon-heavy

Participants expressed confusion over terms like ‘data cart’, ‘ATLAS’, ‘data cube’, ‘pipelines’ and ‘RDF’, suggesting that these are not easily understood by even relatively experienced users.

There was a general sentiment that the technical language used throughout the site could alienate users who are not familiar with data-specific and site-specific jargon…and even those who are.

• “Unless you are data literate, the overall language is quite intimidating - data cube, data slice, API - there’s a lot of very technical language that would be enough to put people off.”
• “I spend my life working with data and I don’t understand that [dimension locking] instruction.”
• “I feel like I’ve been failing as a statistician not understanding this site.”
• “…written by experts, for experts.”

This indicates a need for clear explanations, descriptions, and guidance at the point of use, with more complex descriptions progressively uncovered as required by expert users.

Preliminary theme 2: The site does not offer an intuitive user experience

The site is not intuitive, even for more experienced users, leading to confusion about how to find and work with specific datasets, provide/upload datasets, and use specific features such as dimension locking.

• “[output of dimension locking is] always a surprise.”
• “I’ve never figured out how to use this site.”
• “This is so hard to interpret.”

Participants who provide data found the steps for uploading and updating datasets complex and time-consuming, with data providers referring to personal step-by-step guidance notes or needing/asking for help during the process.

• “The process is counter-intuitive. People forget what to do when uploading annual publications.”
• “[I] need to have multiple screens open to update.”

Navigation issues were prevalent, with participants having difficulty deciding upon the best way to find datasets, amongst multiple search and browse options presented on the first page.

• “If this was a shopping site then I’d have gone to your rivals!”
• “I send people screenshots of how to use the site - it’s painful!”

Preliminary theme 3: The site is slow and often unresponsive

Site performance issues were a recurrent complaint. Participants discussed common experiences of the site loading slowly and taking a long time to process actions like locking dimensions, which doesn’t meet current user expectations:

• “It takes forever to lock the variables, even when you understand how the lock works.”
• “Dimension locking is very slow and painful.”
• “When you click on something you expect it immediately.”

Participants who provide data talked about scheduling their uploads during off-peak times to avoid site lag, which was not always convenient. One participant mentioned using their personal computer for this as their work computer could not handle the large file sizes.

• “It takes days to upload some large datasets.”
• “It’s an ordeal, rather than just another job to tick off.”

Preliminary theme 4: Mixed reviews on user interface and design

There were mixed feelings on the visual design, with a newer user finding it clean and intuitive while others struggled with its layout and navigation.

• “Very clean the way it’s laid out” (a new user)
• Site requires “not just a refresh but a rethink” (a more experienced user)

Specific UI elements were mentioned as problematic, such as the placement and labelling of the full dataset download button and the dataset view options.

• “I hate trying to get data out of the site.”

Users suggested improvements like better labelling, more intuitive placement of features, and clearer visual indicators of ongoing processes (e.g., a “spinny wheel” to show dimensions are being locked).

• “We shouldn’t make our users do the heavy work.”

Preliminary theme 5: Offline support is excellent, but guidance can be improved

Participants seemed generally aware of how to get in touch for support, but contact details were noted as easy to miss, being buried in the footer. Participants noted that support responses were very quick and very helpful, although the out-of-office response suggests a 5-10 day response time.

Participants wanted more documentation and guidance readily available on the site, including visual/video guides and tooltips for better understanding. There was some frustration over the lack of guidance or instructions on the site:

• “Finding out how to upload data is not intuitive”
• “[It’s] not easy to find help or guidance.”

There are opportunities for collaboration and knowledge sharing. Participants seemed keen to learn from each other about how to use the site, suggesting that informal peer support may be valuable.

There appears to be an appetite for more collaborative tools or forums where users could share knowledge and experiences.

Preliminary theme 6: There is uncertainty around dataset formatting and processing

Again, issues with data download options were highlighted, where participants didn’t understand the ‘data slice’ and ‘data cart’ features, so felt they needed to download entire datasets rather than selecting specific parts. The option to download the entire dataset was not immediately apparent to a newer user.

The process for correcting errors or revising datasets was seen as cumbersome, sometimes requiring users to clear contents and start again.

Some participants were unsure about the practicalities of translating data from source formats to the site’s required formats, and how well external users might be equipped to handle certain formats if required to download datasets. There was some confusion from two participants about why data needs to be provided in specific (perceived as non-standard) formats, and why it is processed in certain ways:

• “Would it be better to have this data in a different format - or a different way of accessing it?”
• “Column names are changed between us providing data and data getting published - e.g., FeatureName - we don’t know why.”
• “Standards being inconsistent impacts upon the site.”

Preliminary theme 7: User management and access control could be more specific

Concerns were raised about the implications of having only one person in a team with data publisher/provider level access, both in terms of control and potential single points of failure.

• “A single point of failure if only one person has access…but also quality could suffer if there is no control…”

Participants discussed the need for more flexible access control, such as conferring different levels of authentication and temporarily granting ‘edit’ authority to others, as in MS365:

• “You could just validate or you could have editing permissions.”

Preliminary theme 8: Participants are enthusiastic about the potential of the site, and want to know more about who is using the data

Participants understood the potential role of the service in providing simple access to a comprehensive set of official statistics. They’d like to be able to publish more datasets, and bigger datasets (which isn’t currently possible). Participants were also aware of the potential for the site to pre-empt or respond to FOI requests, which it may currently be prompting.

• “FOI is standard now as [the public’s] first port of call.”
• “Loads of data isn’t on there…it should be all the data…it implies it’s everything…it’s quite misleading.”
• “This is why we get ‘FOI’d so much - I can’t even find my own data on here.”
• “[The site should be] a place to publish nice and clean Scottish Government data for users to access.”

There was a genuine appetite for a feedback loop communicating who is accessing the site, what they are downloading, and how they are using this. This would be all the more useful and encouraging if the site offered an intuitive experience for all users.

• “I don’t know who uses it…”
• “If we had a platform that worked well…you’ve got that engagement and you’d have people helping themselves.”