Tackling child poverty: first year progress report - Annex A

Background, Purpose and Scope

1 This evaluation strategy sets out how we intend to monitor and evaluate action taken by Scottish Government and its partners to meet, by 2030, the four ambitious targets set out in the Child Poverty (Scotland) Act 2017:

• Fewer than 10% of children living in families in relative poverty. This means fewer than one in ten children living in households on low incomes, compared to the average UK household.
• Fewer than 5% of children living in families in absolute poverty. This means fewer than one in twenty children living in low income households where living standards are not increasing.
• Fewer than 5% of children living in families living in combined low income and material deprivation. This means fewer than one in twenty children living in low income households who can't afford basic essential goods and services.
• Fewer than 5% of children living in families in persistent poverty. This means fewer than one in twenty children living in households in poverty for three years out of four.

2 In order to meet these targets, the Scottish Government is required to produce a series of delivery plans, setting out how we propose to meet the targets; and annual progress reports, describing the measures taken and the effect of those measures on progress towards meeting the child poverty targets. Throughout the period to 2030, there will be 3 delivery plans. The first delivery plan, Every Child, Every Chance, covers the period 2018-2022.

3 Every Child, Every Chance sets out a range of actions, from policy areas across the Scottish Government, that aim to impact on the three key drivers of the child poverty targets: income from employment, costs of living, and income from social security and benefits in kind.

Infographic:

4 Income from employment, income from social security and housing costs influence all four targets. Other costs of living influence the combined low income and material deprivation target, but not the remaining three targets.

5 Every Child, Every Chance also sets out some actions that aim to mitigate the negative impacts of poverty for those children who have already experienced it. These important actions are intended to improve the quality of life of children in low income families. For example, we know that poverty increases the risk of mental health problems, and Every Child, Every Chance includes a commitment to increase funding to support better access to Child and Adolescent Mental Health Services. Whilst health policy makers and their delivery partners in the NHS may well wish to evaluate the impact this has on children, such evaluation is not within the scope of this tackling child poverty evaluation strategy. Only actions that are intended either to lift children out of poverty (as defined by the four child poverty target measures), or to prevent children from falling in to poverty, are in scope.

6 The child poverty targets are not an end in themselves. Ultimately, they are about improving children's quality of life and life chances, as summarised in the diagram below. Reducing child poverty, through increasing incomes and reducing costs of living, is one important mechanism for doing so - and it is the focus of Every Child, Every Chance, and of this evaluation strategy. But it is not the only mechanism. We recognise that there are many other important actions being taken forward by the Scottish Government and its partners that will improve children's quality of life and life chances, in ways that are not about increasing incomes or reducing costs of living.

Children's Quality of Life and Life Chances

• Child wellbeing and happiness
• Child physical and social development
• Educational attainment

Child Poverty Targets

• Relative poverty
• Absolute poverty
• Low income + material deprivation
• Persistent poverty

Child Poverty Drivers

• Income from employment
• Costs of living
• Income from social security and benefits in kind

7 Every Child, Every Chance identifies the following priority groups, which are at a higher than average risk of poverty:

• Lone Parents 41% of children are in relative poverty
• Disabled 30% of children are in relative poverty
• 3+ Children 32% of children in relative poverty
• Minority Ethnic 40% of children in relative poverty
• Youngest Child Aged <1 32% of children in relative poverty
• Mothers Aged <25 56% of children in relative poverty

8 We need to make sure that our actions are reaching these groups, and are appropriately tailored to meet their needs.

The Assessment Approach

9 Every Child, Every Chance set out the three key elements of the assessment approach.

Infographic:

Infographic text:

• Monitoring child poverty
• Monitoring the drivers of child poverty
• Assessing the impact of policies and external factors on poverty and its drivers

Monitoring child poverty

10 Data for the four child poverty targets are published annually, usually in March of each year. The data for the relative poverty, absolute poverty, and low income and material deprivation indicators come from the Family Resources Survey (FRS), which is administered by the UK Government's Department for Work and Pensions (DWP). The Scottish Government provides funding to make sure that enough Scottish households are surveyed to give us robust data at the Scotland level. Statistics produced in March 2018 were based on data from 2014-17, and provided annual updates on a three year rolling average. The data for the fourth target, relating to persistent poverty, come from the longitudinal Understanding Society survey. Both sets of statistics (from FRS and Understanding Society) are usually published together, and the date for publication is set by DWP.

Monitoring the drivers of child poverty

11 A new Child Poverty Measurement Framework has been developed to monitor how the drivers of poverty (e.g. parental employment levels) are changing over time. It will be reported on annually, though it will be important to also look at trend data over a longer time period. It is intended to help us understand why performance against the targets is improving, worsening or remaining static, which drivers are moving in the right direction, and which are not - potentially signalling the need for additional action or a change in approach.

Assessing the impact of policies and external factors on poverty and its drivers

12 Poverty and its drivers are influenced by many different factors, only some of which can be influenced by public policy in Scotland. We will ensure that the interpretation of how child poverty and its drivers change over time is supplemented as needed with consideration of key external factors - for example, demographic change, and UK-wide policy on employment law, migration, social security, foreign policy and trade deals.

13 Monitoring the drivers will tell us how they have changed over time, but not necessarily why they have changed. In order to understand the contribution of particular policies to changes in poverty and its drivers, we need to consider external factors, as described above, and evaluate the relative contribution of different policy actions.

Evaluating implementation and impact of individual policies

14 There are over 50 actions in Every Child, Every Chance. There will be a requirement to check whether all actions are being delivered. But given the size and scope of the delivery plan, it is not feasible or justifiable to attempt a comprehensive evaluation of the impact of every action on child poverty. As with the evaluation of Scotland's alcohol strategy, decisions on where to focus evaluation efforts have taken into account a range of criteria, including: likely impact of the intervention on the 4 child poverty target measures; size of the investment from Government; and feasibility, cost and value of a robust evaluation of the intervention's impact on child poverty. This has resulted in the following shortlist of actions:

Infographic:

Infographic text:

Income from Employment / Costs of Living / Income from Social Security

1. Fair Start Scotland 2. Expanded Early Learning and Childcare 3. Private Residential Tenancy 4. Financial Health Check 5. Best Start Grant

15 The choice of actions for inclusion in the shortlist will be kept under review, based on policy developments and stakeholder feedback. Some of the actions in Every Child, Every Chance do not set out a specific policy intervention. Rather, they relate to further exploration of an issue, in order that new policies may be developed. Examples of such actions include: exploring options for a new income supplement for low income families; working with the social housing sector to agree the best ways to keep rents affordable; designing energy efficiency policy and programmes in a way that ensures reduction of child poverty is prioritised; and developing plans with stakeholders for how all areas of transport policy can best support the delivery of the child poverty targets. In advance of any new policies being implemented, they will be considered for addition to the evaluation shortlist. For example, we anticipate that Parental Employment Support and the Income Supplement will be added to the evaluation shortlist in due course.

16 The shortlisted policies will be evaluated by the relevant policy and analytical teams in the way that is most useful for taking forward their policies. In many cases, reducing child poverty is not the primary objective of the policy. The role of the tackling child poverty evaluation strategy is to provide a co-ordination point, to ensure that evidence is collected that will allow an assessment of the contribution of each of the shortlisted actions on progress towards meeting the child poverty targets.

17 This will include information on how many children/parents each action has reached, as well as the characteristics of those reached (e.g. how many are from low income households and/or from each of the priority groups we know have a higher than average risk of poverty).

18 It will also include information to enable an assessment of the impact the action has had on the drivers of poverty (household income from employment and social security, and costs of living). Ideally, this will include a consideration of what would have happened in the absence of the action, in order to estimate the contribution of the action to any observed changes.

19 Qualitative evaluation evidence will be important to help unpack the reasons for any observed changes (or lack of change), identify any unintended side-effects, and understand how the actions could be refined and improved. Qualitative evidence includes detailed information about how each action is being implemented (which is likely to differ across service providers and locations), and the views of those the actions are aiming to support about what has and hasn't been helpful from their perspective.

Evaluating implementation and impact of the package of policies

20 Many of the individual actions will not be sufficient on their own to lift families with children out of poverty - rather, a package of policies will be required. It will therefore be important to evaluate how policies are working together. There are two elements to this:

• System evaluation (qualitative) to examine how well the policies have been designed, and are being implemented, as a coherent package. For example, is suitable child care available for Fair Start participants? Do the policies, and their implementation at a local level, seem joined-up from the perspective of the service user? Are there gaps in the coverage of the package of policies e.g. particular groups who are not being reached?
• Cumulative impact evaluation (quantitative) to assess the impact of the package of policies on the targets, as required by the Act. This will differ from simply adding up the individual impacts, but rather will aim to examine the interrelationships between policies in Every Child Every Chance, as well as wider tax and welfare policies, for different groups of people. This requires complex microsimulation modelling, and there is currently no single model available that successfully does everything that we require. There are also restrictions on the types of policies that lend themselves to being included in a microsimulation model. There will need to be a period of scoping and feasibility, taking on advice from key experts in the field, to help us identify the best route forward.

Governance Structure

21 The structure set out below is intended to provide good accountability, whilst also allowing us to benefit from the expertise of stakeholders and academics, without precluding them from being able to tender for any commissioned work. It ensures that the evaluation work is tightly linked to strategic national policy delivery and development through the existing Directors Delivery Group. It also ensures that links are made to the work by local partners, so that their evidence can both inform national policy and allow sharing of best practice.

Infographic: