Tackling child poverty delivery plan - progress report 2022-23: annex c - priority family types - approach to reporting evidence

Reporting approach for priority family types

Available evidence

The Scottish Government conducts a range of research and analysis on child poverty and in relation to the six priority household groups.

The Poverty and Income Inequality report is a statistical release that includes the overall child poverty rates in Scotland, as well as breakdowns for the priority household groups. Further to this, government statisticians publish ad hoc additional child poverty analysis in which further, and more detailed, statistical data are provided.

Since the first tackling child poverty delivery plan was published in 2018, the Scottish Government has produced a series of 'focus reports' that provide in-depth analysis of each of the priority household groups. The reports make recommendations, based upon the evidence presented, of the kind of support that would be most beneficial, and how this support can best accommodate the needs of the priority group. The focus reports look at the experiences of people in the priority groups in relation to the 'drivers' of child poverty (income from employment, income from social security/benefits in kind, and the cost of living). The following are all available to access online:

• Focus report - Minority ethnic families - 2019
• Focus report - Larger families (3+ children) - 2019^[1]
• Focus report - Lone parent families - 2020
• Focus report - Families with a disabled adult or child - 2021
• Focus report - Mothers under 25 - 2022
• Focus report - Families with a baby - 2022

Having produced a focus report for each of the priority household groups, and in recognition of recent inflationary pressures, the 2023 focus report is concentrating on the effects of the cost of living crisis for all groups, and this will be published alongside the main progress report in June 2023. Going forwards, the focus reports will resume the cycle of assessing any new evidence available and update data for each of the six priority family types.

In addition to the focus reports, a slide pack providing an overview of updated evidence relating to the priority family groups has been produced which consolidates existing information. The current version online will be updated following publication of the progress report in June 2023. Tackling child poverty priority families overview - gov.scot (www.gov.scot).

Furthermore, to support the development of the second tackling child poverty delivery plan, 'Best Start, Bright Futures', Scottish Government analysts produced a what works report highlighting policy options for each of the six priority family groups.

This review allowed Scottish Government analysts to assess the current level of available data at priority family level. Overall, we believed that the approach taken through the latest evaluation strategy provides a good compromise between availability and usability of the data at hand.

For transparency and completeness, the following sections explore the current approaches to quantitative data and the issues to consider. Specifically around statistics for the child poverty targets and the detailed update on the measurement framework indicators.

Child poverty targets

Progress towards the four child poverty targets are reported from data available on the Family Resources Survey (for relative poverty, absolute poverty, and low income and material deprivation) and on Understanding Society (for persistent poverty).

Progress towards the targets is reported annually. Scottish Government has funded a 100% boost to the Scottish Family Resources Survey sample since the early 2000s, which before the pandemic resulted in an achieved sample size of just over 3,000. It should be noted that response rates to surveys have been falling for quite some time now, exacerbated by restrictions imposed during the pandemic, which highlights the importance for an enhanced and effective use of the full range of available sources of evidence to properly assess the impact of actions to reduce levels of child poverty.

Analysts will take stock of survey response rates and the full range of evidence as we look forward to reporting on the interim and final child poverty targets. We will also take account of long-term developments in survey methodologies being explored by a range of survey commissioners adapting to declining participation.

Measurement framework review

As part of the review, analysts explore the range of data sources that report on the indicators of the measurement framework. There are generic challenges that impact on the ability to provide indicator level data by priority family type. Therefore, instead of providing a breakdown of reasons for each specific indicator, the section below explains the common challenges faced, with some examples of where this would be relevant.

The main challenges are: sample size, applicability and contextual circumstances. Taking each one of those in turn.

a) Sample sizes

The most significant technical barrier to providing reliable, regular data updates for each of the priority household groups is achieving satisfactory sample sizes. We will continue to publish data as part of the focus reports on each of the priority family types. Sometimes, base sizes do not allow for detailed analysis. For over half of the indicators where breakdowns are possible, at least three years of survey data would be required to reach a minimally viable sample size. Even with multiple years combined, sample sizes may still be relatively small which means there is a greater margin for error and we can have less confidence in how representative the data is, and in differences observed over time.

Generally, combining annual data is helpful to minimise seasonal factors that may affect the data and provide a more accurate view of long-term trends. However, combining multiple years of data means that it is more difficult to monitor year-on-year change, which the measurement framework aims to do, as the most recent available data is mediated by previous years. For example, the effects of the recent spike in inflation would be underestimated when looking at data on the costs of living.

Furthermore, sample sizes can fluctuate over the years. Weighting techniques help smooth these sample size differences. But still, confidence intervals of smaller sample sizes are wider, meaning that less confidence that an observed change in an indicator is real.

Even with combining multiple years of data, some of the priority groups still do not constitute an adequate sample size which means most indicators in the framework could only ever be partially updated and reporting would be inconsistent. Households with a mother under 25 or with a child under 1 represent smaller proportions of indicator data than, for instance, households with a disabled person. This means that we might be able to achieve a sufficient sample size of households with a disabled person by combining 3 years of data, but the same methods would not provide the requisite number of households with a mother under 25. Analysis of these groups could not be included in the most recent Poverty and Income Inequality report because sample sizes were too small.

b) Applicability

Some of the indicators in the framework are simply not amenable to breakdowns by demographic characteristics. For example, the proportion of childcare services that are open in the holidays or outside of school hours, and the real-terms value of specific benefits. These indicators measure resources; they do not show proportions of people affected and so cannot be broken down by specific characteristics.

c) Contextual circumstances

There are also important differences within each of the 6 groups identified, particularly for the 'minority ethnic' priority group which comprises all groups outside of the majority white British population. Variations in outcomes between ethnic groups are hidden when all groups are treated together, masking a wide range of experiences. For instance, the likelihood of being in a lone parent family is higher for those from an African or Caribbean background than it is for Indian families^[2].