Tackling child poverty pathfinders: evaluability assessment

What factors need to be considered before evaluating the Pathfinders?

Before assessing how best to approach an evaluation of the Child Poverty Pathfinders in Glasgow and Dundee, we need to consider whether there is value in investing in any evaluation at all. There are a number of factors that need to be considered by Scottish Government prior to deciding to progress with an evaluation of the two Pathfinders. Without further clarity on the issues outlined below, any evaluation is likely to:

• Spend valuable evaluation resource attempting to answer these questions before the evaluation can draw firm conclusions.
• Limit the usefulness of the insights and evidence gained through the evaluation to inform future Pathfinder activity.

Defining Child Poverty Pathfinders

As explained above, there is no consensus on how to define the Pathfinders and their scope. Further clarity on this would help ensure that the evaluation is targeted and useful in helping inform future decisions on the use of Pathfinders in Scotland.

Currently, Child Poverty Pathfinders appear to be understood by various stakeholders as meeting a range of key criteria:

Target those in poverty with children, rather than support at a population level – e.g. by supporting those who are unemployed

Using a joined-up service model that brings together services and organisations to provide holistic support to families

Using a place-based approach to target families in need.

Dundee's Pathfinder approach matches with all three criteria. Glasgow's approach looks to provide a single entry point to the collection of support that is relevant for those calling. It doesn't limit eligibility to families, those experiencing poverty, or particular locations, other than a Glasgow postcode. In addition, beyond the overarching aim of addressing child poverty by working closely with families in need, there is little in common between the two existing Pathfinder models.

As a result, it is likely that an evaluation of Glasgow and Dundee Pathfinders would yield insights into the approaches used by the two cities, but it will be difficult to translate evaluation findings into informing the development of the Pathfinder programme unless there is a clearer sense of what is meant by "a Pathfinder".

A suggested definition, based on the information set out above, is as follows:

"A Child Poverty Pathfinder is a service which aims to reduce child poverty by providing holistic support which directly addresses individual-level needs for each person or family in a way that was not previously offered. It is co-designed and delivered in partnership with those with lived experience, meaning the design of each Pathfinder is based on the place in which it operates. Ongoing delivery of the service is enhanced through learning and flexibility, making the Pathfinder a developmental process which evolves and continually improves."

Establishing what is used as the evaluand

Closely related to the above, understanding and being clear about the evaluand – that is, the subject of the evaluation – is critical for framing this evaluability assessment. There are two main possibilities for the evaluand of Child Poverty Pathfinders: the two local Pathfinder models, or the overall programme as a whole. The terms of reference for this research specify that the evaluand(s) will be the individual Child Poverty Pathfinders (as opposed to the programme as a whole) in order to gather learning and evidence of how and whether effective systems change is occurring and whether the Pathfinders are delivering holistic support that meets the needs of families.

However, as seen in the programme-level ToC, learning from the Pathfinders is critical to informing national policy, influencing national systems change and enabling the Scottish Government to support the Pathfinders in resolving barriers, and to provide evidence for scaling up or replicating successful approaches. Placing the local Pathfinders in the context of the wider programme-level ToC, allows us to consider scale. The Pathfinders are operating at a local level and commenced activities only in the first quarter of 2022. Given the intended aim of the Pathfinder is to explore and trial new approaches, they are as yet relatively small in scale. To deliver child poverty reductions nationally, at scale, is dependent on the implementation of the Child Poverty Pathfinder programme, as a whole, which may usefully be the focus of a future evaluation. The lack of cohesion on understanding and defining what a Pathfinder is – as explained above – means that at present it may not be feasible to evaluate through a programme-level lens. Therefore, for the purposes of this evaluability assessment, the two Local Child Poverty Pathfinders are the evaluand and a key aim of evaluating the Pathfinders is to gather learning and evidence of what is working to support further national efforts to reduce child poverty at scale.

Defining Scottish Government's role in a Pathfinder

Scottish Government is a key delivery partner within the two Pathfinders. In Glasgow, Scottish Government is currently supporting the development of some of the national level barriers to joint-working – for example data sharing between the various public sector partners. In Dundee the Scottish Government has a staff member who is part of the team that is shaping and designing the service.

It isn't currently clear what Scottish Government's role within Pathfinders should be now and in the future. Establishing that clarity will help with both the operational delivery as well as with evaluating the performance of the Pathfinders, for several reasons. First, Scottish Government is partly what ties the two Pathfinders together into a single overall programme. If there was a need for a future evaluation to evaluate the programme as a whole (which we do not currently recommend), then it will be necessary to solidify Scottish Government's role in order to help define 'the programme'. Second, it may be that the input that Scottish Government has in the Pathfinders is a contributing factor to why the Pathfinders do or do not have an impact, and so clarifying exactly what their role is may help to understand more about what does and doesn't work. Third, without clarity on the role of Scottish Government, the evaluation might miss out on key people within government who should be involved in the evaluation, or overlook data held by Scottish Government.

We understand that the Pathfinders programme is included as part of the Scottish Government's commitment to addressing child poverty, but that the approach requires action at the local level by local partners. Further consideration is needed by Scottish Government in deciding how they can continue to contribute to Pathfinders in two important areas:

The way it participates as a partner to the development of the approach and direction of the Pathfinder, including the way it provides constructive challenges as an equal partner and contributes insight to help the Pathfinders make evidence-based decisions.

The ways in which it can support the successful implementation of the Pathfinder – which may include working with senior stakeholders to create the buy-in required to see process and cultural change, which in turn facilitates joined-up services across organisations, as well as providing financial and in-kind resources for the Pathfinder.

Addressing the complexity of other Scottish Government funding and activity streams with families

It isn't currently clear how the Child Poverty Pathfinders sits within the broader work being conducted, particularly in Glasgow, around supporting families. The Whole Family Wellbeing Fund (WFWF) is providing funding to Glasgow and Dundee to develop whole family approaches to supporting families. The WFWF is being progressed through different Directorates in the Scottish Government to the Child Poverty Pathfinders. There is a significant cross over in Child Poverty Pathfinders and WFWF including in:

The families they are supporting. The families that the two Child Poverty Pathfinders are supporting often have multiple and complex needs arising from, and contributing to, their level of poverty. The families that are being supported through the WFWF and are known to services, such as children and families social work, are commonly in poverty.

The organisations working together for the Child Poverty Pathfinders and WFWF. Both programmes require the involvement of the Council, HSCP, Scottish Government, other agencies that interact with children and families and third sector organisations that work with children and families.

The objective of creating a joined-up approach that changes the way that families are supported. Both Child Poverty Pathfinders and WFWF are targeting holistic support that requires significant changes to the systems and organisations.

Given the complexities of addressing child poverty and of the existing service landscapes in Scotland, understanding and attributing the impact of the Child Poverty Pathfinders is likely to be challenging. There is some obvious cross-over in Scottish Government programmes and efforts that complicate the ability to evaluate the Child Poverty Pathfinder – particularly in Glasgow. We feel that there could be scope to more closely align and coordinate Scottish Government's activity on children and families to help give the evaluation a better chance of being able to draw conclusions as part of an evaluation. Other local activity to take into account and which may increase the difficulty of attributing the impact of the Glasgow Helps element of the Glasgow Pathfinder includes Citizens Advice Bureau and Glasgow's Advice & Information Network (GAIN), which both offer telephone helpline support.

Local evaluation activity already happening

We understand that Glasgow City Council is already working on evaluating Glasgow Helps (a key aspect of their Child Poverty Pathfinder) which includes a cost-benefit analysis of the programme. There is a risk that there will be duplication of evaluation effort. However, if well-coordinated it may offer useful information and evidence for analysis.

The existing evaluation is being led by the University of Glasgow and is designed to monitor and articulate the ongoing progress within the Pathfinder, using proactive critical reflection, data collecting and continuous feedback. It will also provide reflection through workshops and sessions throughout the course of the Pathfinder to support continuous improvement. This will help to identify methodologies to inform the monitoring system that can be used for future projects.

Existing evidence on the approaches taken by the Pathfinders

An evaluation of the Child Poverty Pathfinders would be able to understand the effectiveness of the Pathfinder in Glasgow and the key worker approach taken in Dundee. There is a reasonable body of evidence on the effectiveness of the key features of both approaches which raises a consideration for Scottish Government of the value added by additional insight into these two models to inform decisions on the future of the Pathfinders programme.

The literature supports that several key features of both Pathfinders can help in making them effective in reducing child poverty in a sustainable way. Although the Pathfinders have taken different approaches to recruiting participants and organising service co-ordination, there is evidence to support both service models. In particular, the literature emphasises the importance of offering personalised, holistic, and whole-family support as being crucial for addressing poverty. For example, the Supporting Families framework outlines that strong local partnerships that can identify and support families in need seamlessly across a range of services will be critical for addressing child poverty.^[10] Similarly, a review of best practice in this area conducted by the government of Northern Ireland concluded that successful interventions were characterised by: parental engagement, targeted approaches, harnessing existing resources, and holistic services.^[11] This is discussed in more detail in the literature review at appendix 2.