Equality evidence strategy 2023-2025: interim review

3. The Actions

While it is important to understand the high-level overview of progress towards the Strategy’s principles, it is also worth setting some context relating to specific progress in terms of the actions set out in the Equality Evidence Strategy 2023-2025.

As of September 2024, of the 45 actions, 14 were already ‘complete’ and 23 were ‘on course’, demonstrating strong commitment to advancing equality evidence across many policy areas. Seven actions are marked as ‘delayed’, and one as ‘not yet started’. The following provides an update on progress across the action categories, including specific examples of progress from the Strategy’s actions.

3.1 The Action Categories

Establishing processes (e.g. with stakeholders) to enable more equality data collection

Before equality data collection can progress in some areas, new processes or approaches need to be agreed with internal and external stakeholders. There are currently 16 actions where this is their primary objective. As of the September 2024 update, of these actions one is complete, 11 are on course, and four are delayed. Examples include:

• Action 11 (action on course) from Justice research states, “We will promote best practice and support justice organisations with their data collection and reporting, including workforce data, through mutual support and sharing of good practice.” Activity has focused on ongoing support and promotion of best practice through the work of the Cross Justice Working Group on Race Data and Evidence. Relevant activities, to date, under this Action include:
  • Discussions with Justice Organisations to find out about the ethnicity data they currently collect, and understand any barriers to improvement;
  • Inviting the Equality and Human Rights Commission to speak to the group to promote better understand of reporting requirements under the Public Sector Equality Duty;
  • Agreeing a best practice approach to collecting staff equality data across justice organisations in Scotland;
  • Facilitating opportunities for peer learning across justice organisations including sharing ideas and experiences.
• Action 25 (action on course) from Housing and Homelessness analysis states, "By 2025, work to consider the appropriateness and feasibility of gathering equality data via the HL1 and PREVENT1 [specific datasets] collections will have concluded. Agreement will be reached with data providers (i.e. local authorities) on the changes to be made, and there may be some progress made as to implementing these." To date, engagement with the public and data providers has helped shape what the new content in these systems will look like and will highlight key groups facing inequalities in relation to homelessness.

Enhancing equality data collection / collection of new equality data in existing datasets

A subset of actions relate to enhancing data collection directly. This comprises: improving how equality variables are captured in a pre-existing data collection; or adding one or more new equality variables into a pre-existing data collection (e.g. new questions into an annual survey). There are currently eight actions where this is their primary objective. As of the September 2024 update, of these actions four are complete, three are delayed, and one is marked as not yet started. Delays are largely related to changes in methodology and lead analysts are considering alternative ways to approach their action. Examples of progress include:

• Action 39 (action complete) from Labour Market and Employability, set out to analyse and publish equality data collected through a new ‘No One Left Behind’ data template (including on trans status and disability). After completion, the team also went on to:
  • Collect pregnancy data from April 2024.
  • Collect data on trans status for Fair Start Scotland
  • Publish additional equality breakdowns at Local Authority level, by sex, ethnicity and disability as well as long-term health condition type and number, barriers to employment and progression information.
  • Support a PhD intern to complete a project on intersectionality analysis using an extract of the Fair Start Scotland dataset.

Action 16 (action complete) from Poverty research planned to publish further analysis of the food security data collected in the Family Resources Survey. Household food security questions were newly added to the Family Resources Survey in 2019/20. They ask about whether people were worried about running out of food, had to reduce meal sizes or skipped meals. Equalities breakdowns were first published in the Poverty and Income Inequality in Scotland publication in March 2023 (Tables 41 to 47). The team plan to publish time series information once there are enough years of data to draw together.

Enhancing analysis of equality data

In developing the Equality Evidence Strategy, it was found that there were some instances where equality variables were being collected but not analysed. Reasons for this included a policy focus on high level equality findings, a lack of clear direction on which intersectional combinations should be examined, and a need to wait for data collated across a number of years to produce meaningful statistical results. There are currently 10 actions where improving analysis of existing data is their primary objective. As of the September 2024 update, of these actions four are complete and six are on course. Examples include:

• Action 27 (action complete) from Health and Social Care analysis area, set out to, “Produce cross-tabulations of headline survey results by equalities variables and publish them as additional analysis where disclosure rules allow.” This was achieved and a full breakdown of all Scottish Health and Care Experience (HACE) Survey questions by equalities characteristic was published along with a "Variations in Experience" report.
• Action 22 (on course) in Equality Analysis sets out a plan to undertake work to strengthen the evidence base on the experiences of non-binary people in Scotland. The team have recently published an Evidence Review of Non-Binary People’s Experiences in Scotland and plan to produce additional analysis of Census 2022 data in the new year.

Enhancing reporting or publication of equality data collected

In the majority of cases, where new equality data is being collected, or where equality data is being newly analysed, this is also being published. There may be a few exceptions to this, for instance where sample sizes are too small. There are currently seven actions where this is their primary objective. As of the September report, of these actions four are complete and three are on course. Examples include:

• Action 35 (action complete) from Transport Scotland & Public Health Scotland have now published ethnicity information collected by hospitals for those receiving hospital care as a result of road traffic accidents.
• Action 4 (action complete) from Education sought to investigate the addition of a supplementary table covering the attainment of school leavers who have been assessed or declared as disabled. This was introduced in the 2021/22 school leaver publication, which brings the data tables in line with other associated education statistics^[3].

Additional Categories

There are three actions related to developing new datasets that are all marked as ‘on course’ and one action that commits to publishing equality data similar to previous years which is marked as ‘complete’.

3.2 Interim Action Insights

As part of this interim review, 1:1 discussions were carried out with lead analysts working on each action to understand what opportunities and challenges have emerged so far and whether further lessons learned can be identified at this stage.

It has been noted where opportunities and challenges are similar to or deviate from the expected challenges outlined in the Strategy.

3.2.1 Opportunities

Factors which have helped to facilitate EDIP actions are presented below.

• Actions that were specific and less complex to achieve were straightforward to complete. As would be expected, actions that involved completing additional analysis on existing data were more achievable than actions which involved a whole new programme of work or collecting new data. Connected to this, analysts who understood their action as having a clear impact reflected that this made it easier to work towards completing the action.
• Another factor promoting delivery, identified by analysts, was the support of EDIP. Reminders and tracking processes built into EDIP’s quarterly monitoring helped to keep many actions a priority within workplans.
• Equally, action progress was easier to drive forward once internal and external buy-in had been gained. Internal buy-in was usually where actions were connected to wider policy strategies. External data providers and collectors who were willing to improve equality data and engage with processes around this was noted as an important contributing factor to completed and on course actions.
• Updates to data collection methods improved the quality of data that could be collected and analysed for a number of actions. Examples of this include the change from SAS and other statistical packages to R within Scottish Government, as well as updated IT systems collecting ethnicity data from external data providers. While some analysts reflected that this took more time to complete, these improvements overall were beneficial for working towards actions more effectively. The ability for data collectors to design appropriate methodologies and instruments for data collection counteracts one of the expected challenges, accessibility and usability of data collection tools, outlined in the original Strategy.
• Finally, being able to draw on the expertise of analysts was identified as a key facilitating factor by lead analysts. A breadth of skills and expertise within the team responsible for the EDIP action allowed some of the issues that arose with completing the action to be resolved quickly. Further, analysts reflected that being able to access support and advice from analysts in other teams in the Scottish Government helped them to improve the expertise within the team responsible for the action and progress work towards completion. For example, knowledge on how to aggregate data across multiple years to address sample size issues or how to use new statistical tools and techniques efficiently to analyse and present data. This countered one of the expected challenges, data quality and analysis, outlined in the original Strategy.

3.2.2 Challenges

As with the above facilitating conditions, reasons for challenges and delays were identified through discussions with lead analysts. Some lead analysts identified counter points to areas highlighted as an opportunity or benefit.

• Actions which weren’t specific and/or lacked a clear route to achieving its goal made it more challenging for analysts to progress or complete actions. For example, actions that tried to incorporate lots of goals which relied on multiple strands of work or stakeholder and data collector input tended to be more complex to progress. In addition, where specific focused tasks were missing in the actions this also caused issues for analysts who found it difficult to know when to mark the action as complete. For example, some actions are worded in a way that means the work could be continually ongoing even after the life of the Strategy, such as continuing to produce annual publications. Equally, if actions contained multiple goals within the one action, as mentioned above, there may be confusion around status updates as aspects of the action may be completed, delayed, or on course all at once.
• Similarly, actions in which the purpose and intended impact was unclear sometimes led to its de-prioritisation in comparison to competing demands within a team. Analysts also highlighted challenges related to buy-in around this, where data providers, collectors, policy colleagues and other stakeholders question the purpose or importance of collecting or improving equality data based on unclear purpose or impact.
• Handover of information regarding the Equality Evidence Strategy, Actions and EDIP from previous action leads to new staff taking over responsibility of actions also caused challenges and delays. Staff and team changes have occurred throughout the Strategy which has led to a loss of knowledge, especially where the lead analyst who completed the data audit and agreed their action in 2021, is no longer in post. In some cases, analysts were not aware of actions they were taking responsibility for until the EDIP quarterly progress update was requested. Improvements to communication, such as team handovers and better introductions to the Strategy and EDIP, are required.
• Finite resources were a consistent prohibiting factor among teams responsible for completing their actions. This ties into one of the expected challenges - cost and feasibility - outlined originally in the Strategy document. Analysts highlighted that constraints on staff and capacity within the team posed challenges to carrying out the work, particularly if the action was complex or time consuming.
• Methodology issues were also a common reflection from analysts facing challenges to completing their EDIP action. It was also noted that there were challenges and delays caused by the methodology or dataset stated in the original data audit, especially where an alternative approach was not possible. The most occurring methodological challenge for analysts was small sample sizes. This made it difficult to analyse and capture statistically significant equality data breakdowns and provide intersectional data. This was often connected to low completion rates for both in-person and online data collection methods; analysts reflected that data collectors and respondents to surveys may choose not to submit equality information due to the potentially sensitive nature of the data. This may be linked two of the expected challenges, trust as a barrier to participation and under-representation of certain groups, which were outlined in the original Strategy document. However, as noted in the Strategy document and in opportunities outlined above, if sample sizes are small, alternative sources of data will be sought where possible, utilising methods such as data pooling or through qualitative approaches.
• Another reflection from analysts was the challenge of carrying out intersectional analysis, particularly around the feasibility and utility of this type of analysis due to small sample sizes. It was suggested that guidance on this, as well as other peer learning opportunities with other analysts examining intersectional equality breakdowns would be useful.

The original Equality Evidence Strategy document, had outlined a series of additional expected challenges that, during discussions with lead analysts most, were not seen as a major challenge.

Reporting and communication of equality evidence: This has not been a reported issue for the majority of actions so far, however a few action leads did mention that they do not see a wide engagement in the equality statistics they produced.

Impacts of the COVID-19 pandemic: The COVID-19 pandemic impacted data collection across a number of Scottish Government commissioned surveys and research, for example changes from in-person surveys to online or pausing of a survey entirely during this time. This in turn impacted the availability and comparability of results across years. The impacts of COVID-19 on the data has been taken into consideration across various actions.

Stakeholder differences: Action leads continue to engage with a range of stakeholder views in order to guide their decision making. However, where actions are linked to wider policy work, this may lead to prioritisation of data development work that may not meet all stakeholder and user needs.

At this stage, no trends have been observed in terms of areas that a progressing well or not.

3.3 Action Impacts

As outlined in the Strategy’s vision, a stronger evidence base will allow for tackling structural and intersectional inequality of outcomes and development and delivery of sound, inclusive policies and services for people living in Scotland.

Lead analysts across a number of the actions fed back that the impact of producing data on policymaking and improvements to people’s lives has been hard to measure at this stage. This is in part because the Strategy is at the half way point and also that many completed actions have long-term intended impacts which is difficult to measure within the lifetime of the Strategy.

We will continue to encourage and support action leads to focus on policy and other impacts of their equality data improvement work over the course of the Strategy’s final year. A more detailed review of impact will be conducted for the wider evaluation after the end of the Strategy in 2025.