Scotland’s Equality Evidence Strategy 2023-2025

6. Challenges

There are several challenges involved in improving Scotland's equality evidence base, including those raised by stakeholders in the EDIP consultation and stakeholder engagement events. We also share a number of challenges which are similarly experienced by the Welsh Government and the ONS in filling equality evidence gaps. These challenges provide important context against which improvement activity has been developed and will be implemented throughout the lifespan of this strategy.

Cost and feasibility challenges

The Equality Analysis team within the Scottish Government has a prominent role to play in promoting and coordinating robust analysis to build a strong evidence base. The team also plays a leading role in enabling others to lead effectively and develop good practice. However, most of the analysis has to be carried out within relevant policy areas across the Scottish Government, in line with our mainstreamed approach. As such, these policy areas determine priorities for equality evidence in their own area. In general, analysts work to support policy officials to ensure policies are developed in an inclusive way. Thus, priorities for filling equality evidence gaps will depend to some extent on policy priorities within each area. However, not every item of data that policymakers would like to know can be collected, often because of technical, ethical or cost barriers. There is always a need be clear on what data are required and why to ensure that the right data are collected to meet the specified purpose, and to balance costs and benefits.

Trust

Trust is often a barrier to participation in research, especially among the most marginalised groups. Research by the Inclusive Data Taskforce, which was established in October 2020 by the UK Statistics Authority to improve the UK's inclusive data holdings, found that there was a general sense of distrust in government and in government statistics among participants, particularly among under-represented groups (in the Inclusive Data Taskforce research these groups included those from Gypsy, Roma and Traveller communities, other minority ethnic groups and documented and undocumented migrants).

Equality questions can involve people divulging sensitive personal information so understandably some people are reluctant to provide answers. For example, people may be reluctant or feel particularly sensitive about sharing information about gender reassignment, sexual orientation, or religion or belief due to fears of discrimination or harassment or feelings that the information is irrelevant to the service. Many people will have a distrust of public bodies' ability to store their data securely. While there may be public distrust of large data linkage systems, the lack of join-up nationally and locally between data sources and collections means that the same questions need to be asked and answered again and again, increasing respondent burden further.

It may be that if public bodies are sensitive to these concerns, seek to link collections safely and manage the response issues in a safe and supportive way, that sample sizes will increase along with the quality and validity of the data. Seeing data collections leading to specific improvement (i.e. policies) is also likely to increase response.

Over the term of this strategy, work will be undertaken with under-represented groups and relevant stakeholder organisations to increase trust in and understanding of government research and statistics. Collaboration with the ONS will be sought on relevant projects in this space.

Under-representation of certain groups

The previously discussed issues around trust suggest that certain groups are more likely to be under-represented in government statistics and research. This means that policy decisions may not adequately reflect their interests and experiences. Furthermore, where underrepresented groups do take part in research, they may make up a proportionately small percentage of the sample, limiting options for disaggregated data.

If sample sizes are small, alternative sources of data will be sought, where possible, utilising methods such as data pooling or through qualitative approaches.

Accessibility and usability of data collection tools

Data collectors should design appropriate methodologies and instruments for data collection, including question wording and setting (face-to-face, digital, self-completed or staff-completed). Respondents should be able to recognise themselves and their circumstances within categorisations. The Scottish Government has produced guidance on how to collect equality data. The recommended questions aim to be simple to understand and should be used consistently across data collection tools.

Data quality and analysis

While administrative data sources with a good response rate may be able to provide service-level information disaggregated by a variety of equality variables and, in some instances, intersections between characteristics, it will always be more difficult with population surveys which by nature have smaller samples sizes. There are, however, various analytical statistical techniques (for example combining years, imputing data, data linkage) that might allow data to be mined better than it is at present. It is also important to quality assure data and data should be checked for consistency and to ensure that individual cases are not disclosive.

Reporting and communication of equality evidence

Once data are clean, quality assured, analysed and ready for reporting, there can still be some issues to overcome. Included in this would be the categories that can or should be reported; the labelling used; acceptable confidence intervals, disclosure issues and presentation in a range of easy-to-understand accessible formats. The reporting and communication of equality evidence should always be done in ways that do not further stigmatise or blame individuals or groups.

The Equality Evidence Finder provides a central repository for equality data, but it is important to consider a range of accessible formats and presenting data and charts in a variety of formats.

Impacts of the COVID-19 pandemic

Population surveys are important sources of equality evidence and, as such, it is important to note when thinking about change over time. In March 2020, to help prevent the spread of COVID-19, fieldwork for the Scottish Government's core surveys - the Scottish Household Survey (SHS), the Scottish Health Survey (SHeS), and the Scottish Crime and Justice Survey (SCJS) - was suspended. A revised approach, which involved no interviewer travel and surveys conducted remotely by telephone or by video, was piloted and adopted for the remainder of the 2020 sample. This change in method has the potential to change the accuracy of estimates and introduce discontinuity into data series'. As such, it has been recommended that when using core survey data 2020 data should not be compared with data that came before it.

Stakeholder differences

Different stakeholders have varying, and sometimes conflicting, views on which equality evidence gaps should be prioritised and how data should be gathered. This came through clearly in our consultation. Where time and resources are limited and it is not possible to meet all demands and views, there will need to be prioritisation of data development work and decisions made that do not meet all stakeholder and user needs. However, a range of stakeholder views will often be sought in order to guide our decision making.