Scotland's equality evidence strategy 2017-2021

9. Strategic Approach to Filling Gaps

As noted above, there is a substantial amount of equality evidence available for Scotland, and much of this can be disaggregated by a number of equality characteristics. This strategy will seek to ensure that best use is made of existing evidence before recommending new data collection or new bespoke research. However we do recognise that gaps exist and that the evidence base can be improved. But none of us have the capacity or the resources to do everything we need to do immediately, meaning that we will need to identify a strategy by which to prioritise gaps and identify lead responsibilities.

Approach to Prioritising Gaps

In trying to establish priorities we will first aim to establish how important the gap is and what major policies, programmes and frameworks new evidence would feed in to. For example, the National Performance Framework, the Race Equality Framework, A Fairer Scotland for Disabled People and the British Sign Language ( BSL) National Plan would all be examples of high priority work. Work for the independent Race Equality Framework Adviser and the Poverty and Inequality Commission would also be key priorities.

Once priority evidence gaps have been established, the next step will be to consider the most cost effective way to fill these. There could be several options, and we will ensure an assessment of costs in terms of resource, finance and time is undertaken. We recommend that priority is placed on making better use of data that already exists but is not yet fully analysed or effectively disseminated. Some policy areas may well be evidenced with equality data that's out of date, and this should be weighed up against areas where there is no data at all. More expensive ways to fill gaps would be a new survey or qualitative research, and in order to justify expenditure on these, the gaps would have to be of the highest priority.

It should be made clear, however, that this strategy is aimed at improving Scotland's evidence base - the Scottish Government do not intend to take responsibility for attending to every evidence gap identified in this document.

In addition to projects that Scottish Government lead, we will also encourage partner organisations in the public and third sectors to develop projects aimed at strengthening Scotland's equality evidence base. We will seek to collaborate with external analysts who would be interested in using their own data and research to fill evidence gaps to inform policy, as well as encouraging other organisations to fund and carry out research. We will seek to improve channels of communication and encourage collaborative working, if appropriate setting up themed working groups or round table events. Where appropriate we will seek to work with partners to improve the quality of research to ensure that it is carried out in line with current Government Social Research best practice guidelines.

The following diagram describes the broad approach we intend to take when considering evidence gaps. Note that the policy and frameworks shown are illustrative only. The '£' loosely represent costs, from '£' relatively inexpensive to '£££' relatively expensive. Where we've used the word 'fill' this could also mean enhancing evidence to cover part of a gap.

Diagram 1 - Approach to Filling Evidence Gaps

Identifying Options to Fill Gaps

As Diagram 1 shows, there are a range of options that can be used to fill a gap depending on the nature of the gap. These include:

• Published Statistics - the Scottish Government regularly publishes National and Official Statistics across a wide range of policy areas and makes equality data available on its websites ^{[87 88]} . These statistics have been produced using the Code of Practice for Official Statistics ^[89] , which ensures that statistics informing decision making have been produced and explained to high standards. However sometimes users find this information difficult to find, so we will continue to work to improve the accessibility of this, making it available through the Equality Evidence Finder and the evidence toolkit. We will seek to make it easier for users to find data that's already in the public domain.

• Anticipate future releases e.g. of Surveys or Census - if a gap could be filled or updated with new or emerging data e.g. from the SSCQ multi-year dataset or the Census 2021, then this could influence decisions on whether or not to commission a new survey.

• Desk Reviews - before commissioning new work it is good practice to carry out a desk review to ensure that the evidence sought, or something similar, has not already been done by others.

• Unpublished Survey and Admin Data - data on equality characteristics has often been collected in a large survey or administrative source but hasn't been released due to constraints on analytical resources. In this instance a conversation with the data owner could help raise this work as a priority. It may also be possible for others e.g. academics, to set up a process in a safe environment where they can access the data directly for analysis. The new Digital Economy Act 2017 ^[90] may impact on data access and sharing of administrative sources.

• Add new question to existing survey ^[91] - major household surveys are regularly reviewed by survey managers and data gaps can be filled by requesting that new questions are amended or added to better reflect current policy making and priorities.

• Data Linkage - the linking of administrative and/or survey data can offer new insights into social issues and maximises the public policy benefits of the information we already collect. Data Linkage in a safe and secure environment under nationally agreed Guiding Principles ^[92] could provide opportunities for researchers and policy makers to acquire robust data for meaningful, evidence-based policy development. To support this approach, the Scottish Government's Office of the Chief Statistician and Strategic Analysis ( OCSSA) has invested in the Scottish Informatics and Linkage Collaboration ( SILC), ring-fencing funding for up to 30 data linkage projects. An example of ongoing data linkage project is the Scottish Health and Ethnicity Linkage Study ^[93] ( SHELS) which aims to examine the relationship or link between a person's ethnic group and a number of important health issues which affect the Scottish population. A person's ethnic group is often based on characteristics shared within the group. Usually it is things like a common language, culture, tradition and history.

• New Bespoke survey - new surveys can be designed and administered to fill information gaps. The costs of these can vary depending on the sampling design and the methods used to ask respondents questions. Often a fairly large sample is required to gain reliable results and it can be costly and time consuming to gather the data. If interviewers are required this can add significantly to the costs. Online surveys can be a quicker and cheaper way of achieving results but it is difficult to gain representative responses using this medium. However if all other alternatives have been exhausted it might be that a new survey is the only way to fill a data gap.

• Qualitative research - gathers information that reveals a target audience's range of behavior, perceptions, views and experiences. Data is not usually in numerical form, and questions tend to be open-ended rather than with pre-determined response categories. It produces 'rich', in-depth data that is particularly useful for understanding how and why things have happened. It can help to develop ideas for potential quantitative research e.g. when you are exploring a subject about which you do not know much in advance. It can also help to explain quantitative data e.g. when you want to grasp the underlying meanings, motives and reasons behind the numbers. At a very simple and straightforward level, qualitative quotes can be used to 'add life' to official statistics, give them more meaning and engage the reader. These were included in the Scottish Government and Poverty Truth's 'Poverty in Scotland' ^[94] report, for example.

Some common qualitative data collection methods include focus groups, individual in-depth interviews, and participation/observations. The sample size is typically small, and respondents are selected to provide a range of different experiences and perspectives, rather than to be 'representative' in a statistical sense.