Child poverty - monitoring and evaluation: policy evaluation framework

Annex B: Evaluating the effectiveness of system-change initiatives

Evidence has shown the wide range of interacting factors that influence child poverty, and that a number of different forms of support are likely to be needed to help people move out of poverty - in different combinations, for different family circumstances. Child poverty support systems (i.e. the range of services that play a role in addressing child poverty) are often complex and can be hard for individuals to navigate, due to the opacity of the system - with complex eligibility requirements and entry routes.

The evaluation strategy for the second Tackling Child Poverty Delivery Plan aims to support greater understanding of how different policies are linked up together by undertaking a 'system evaluation' of the implementation and impact of the package of policies. Rather than understanding the impact of individual policies, the aim of the system evaluation is to understand the impact of policies in combination and how well they work together in practice.

The evaluation is currently under development. The work consists of two phases. Phase one consists of developmental and set-up work to identify and understand the implementation of system change initiatives. Phase two consists of collating and synthesising evidence and learning from across the range of initiatives focused on system change.

The research questions are:

Implementation and delivery questions

1. How successfully have actions to achieve system change been implemented?

2. What are the enablers that have facilitated system change and the challenges that have been experienced, and how have challenges been overcome?

3. What have been effective and ineffective ways of reaching intended beneficiaries?

4. What contextual factors have influenced the implementation and how?

5. What lessons for successful implementation can be drawn for other areas and for wider policy development?

System change outcomes

1. To what extent have system-level outcomes been achieved?

2. How have system-level outcomes been achieved?

3. What are the range of contextual factors that influence this? What has prevented system outcomes from being achieved?

4. How have service users, service providers and wider stakeholders experienced the systems change?

Impact on child poverty

5. Has the initiative resulted in a reduction in child poverty?

6. Has the initiative resulted in an impact on the child poverty drivers (i.e. increased income from employment or from benefits or reduced cost of living for families in poverty)?

7. To what extent have impacts varied for different groups of beneficiaries (e.g. priority family groups, depth of poverty, equality groups)?

8. What are the pathways by which system change contributes to a reduction in child poverty?

Costs and benefits

1. What have initiatives cost to deliver? What are the unit costs?

2. What are the monetisable benefits of the initiative and has there been (or is there a potential for) a return on investment?