Child poverty - monitoring and evaluation: policy evaluation framework

Theme 2 – Impact on targets

As with many evaluations, measuring policy impacts can be quite complex and full of caveats. The approach taken for child poverty is no exception. We aim to continue to learn and improve as policy evolves and policies progress.

At a national level, we can track progress towards meeting the child poverty targets with data from national statistics. Details on target definitions and data can be found in Annex A.

However, it is necessary to understand how individual policies can support this national effort. From an individual policy point of view, you can consider the following steps which will help you ascertain the likely impact of your policy on child poverty targets. Figure 5 shows how this can be done.

Figure 5. Estimating impact of policies on child poverty targets

• Driver of poverty
• Type of impact
• Target audience
• Potential size of impact
• Certainty of impact
• Immediacy of impact

Identify which driver of poverty the policy is likely to impact. Will it increase income from employment, reduce cost of living or increase household income via social security or benefits in-kind? Depending on what child poverty driver your policy is tackling, the impact on the child poverty targets is likely to be different. For example, a programme that raises a family's income has the potential to improve on three income related targets (relative poverty, absolute poverty, and, depending on time scale, persistent poverty). However, a policy which minimises the costs of essential goods will impact on absolute poverty and material deprivation. Consider whether child poverty is a primary objective for your policy or whether it is a secondary objective.

What type of impact is expected? Will the policy have a direct impact on households for example by directly increasing their income or reducing cost of living? Or is it more an indirect impact by for example improving living conditions which could in turn reduce cost of living? Indirect impacts are harder to quantify. However, through qualitative assessment it is still possible to establish an expected contribution of the policy on people's lives. Further detail on how to further investigate indirect impacts can be found in the 'enhanced life chances' section.

Who will benefit? Think about the characteristics of the policy's target group. The more that a policy is targeted specifically at children in poverty, the greater the potential impact of the policy on child poverty targets. The funnel approach to children in poverty is as follows.

All households > All households with children > All low-income households with children

Further, we know that there are certain household types most at risk of being in poverty. So, consider if you have more detail about your target group – do you know their characteristics. If your policy targets these groups, the likelihood of the policy's impact on child poverty is greater. Examples of these groups are: households with a baby under one, households with three or more children, lone parent households, minority ethnic households, households with a disabled adult or child, households where the mother is 25 or under. More detail on Theme 4 – Reach / Targeting.

Think about the potential size of the impact. That is, the anticipated scale of the policy. This can take the form of anticipated number of people reached or a monetary amount (be it through increase in household incomes or reduction in cost of living).

Targeted policies have more direct action but large scale policies with broader reach may still have beneficial impacts. For example, an employability programme aiming to get 100 parents into work will be more effective in reducing child poverty than an employability programme aiming to get 100 adults into work. However, an employability programme aiming to get 1,000 adults into work might be more effective than a smaller programme targeted at parents, especially if it is well linked to other services such as childcare.

Set out data collection from the start so that you can measure how many people are reached and, of those, how many have children and are in poverty or on a low income. Be as specific as possible to provide the best estimate of the anticipated scale of impact on one of the following metrics:

• Number of children in poverty / low income expected to benefit
• Number of parents who are in poverty / low income expected to benefit
• Number of households who are in poverty / low income expected to benefit

Where possible, we will help you model the child poverty impacts of policies, and include these into cumulative impact assessments. However, this will not always be possible. The larger the scale of the policy – and the more detail you provide around the anticipated size and characteristics of the target group, and the impact on the child poverty drivers – the more likely we will be able to model the policy.

How certain is the impact on child poverty? This looks at how confident you are that the policy, on its own, will have an impact on one or more of the target measures and the certainty that the scale will be.

When is the policy likely to have an impact? Consider the immediacy of your policy. That is, how soon will low income families be able to benefit from the policy, but also how long do you estimate it will take for any benefit to be translated in potential improvements to child poverty targets. For example, while the Scottish Child Payment puts money into families' pockets straightaway, the additional income provided will take time to be reflected in annual household income.

Individual policy actions will probably not be sufficient on their own to lift families with children out of poverty. Rather, it is a package of policies that will be required. In order to understand how policies work together, cumulative impact work is critical.

In this space, quantitative cumulative impact assessment estimates the impact of a package of policies on child poverty targets. The quantitative modelling can tell us the theoretical impact of policies. However, how they are realised depends on how much friction there is in the system. As such, we are currently developing a qualitative systems evaluation that examines how well the system is working for families and whether current policies in this space are improving life outcomes for families in poverty.