A Co-constructed and Co-produced Evaluation of the Anchor Project in Shetland

Section 1: Evaluation Aims and Objectives

1.1. Aims

The aims of this Anchor project evaluation were to:

• Undertake a co-constructed and co-produced evaluation study in collaboration with Shetland based families, planners, community, and professional stakeholders which identifies the key components, theory of change, mechanisms of action and contextual factors that are operational within the Anchor project, as family-led problem solving and early intervention support.
• Through co-production involving local participants and external evaluators, using a three-phase action research approach, establish key elements, the system changes requirements to replicate and mainstream the Anchor approach of embedding family-led problem solving and early intervention within existing health, social care, and educational systems within Shetland.
• Produce an Anchor ‘blue-print’ or ‘programme map/theory’ that could be used to guide the introduction, implementation and sustainability actions and scalability of this intervention in other island or remote and rural contexts in Scotland as a key lever to reduce child poverty, see for instance Tackling Child Poverty Delivery Plan 2022-26¹ and Poverty in rural Scotland: evidence review².

1.2 What Did the Evaluation Involve?

This evaluation had three distinct Phases, along with three key objectives:

• to map Anchor in Phase one (January 2023)
• to explore mainstreaming the change in Phase two (February 2023)
• and in Phase three, to identify the opportunities and challenges from the learning of Anchor to mainstreaming the Anchor approach to other Island/Scottish contexts (Network meeting 5^th October 2023).

All three Phases are complete and now reported. This evaluation was designed in close collaboration with Shetland Island Council’s organisational development lead and manager for the Anchor Project (Emma Perring) and one of the project workers (Lynsey Hall) who was fundamental to engaging and working with families through the evaluation. Early in the planning stages of this evaluation, the researchers recognised the need to use participatory and co-produced methods to capture the stories and experiences of all key stakeholders involved in Anchor. This accorded with the Scottish Government policy approach with an essential focus on person centred approaches by placing children, young people and families at the centre of decision making. Furthermore, espoused policy values of compassion, dignity and fairness required an in-depth participatory exploration of Anchor to understand its component parts.

These key contacts have been crucial to the conduct and rigour of this evaluation process, building on their knowledge of the local landscape along with the relationships and trust that they have developed with families and other key stakeholders over three years of this project. Ethical approval for this evaluation was given by the School of Nursing, Midwifery and Paramedic Practice Ethics Review Committee at Robert Gordon University (SERP reference number: 22-10 approval granted 12/01/23) (Appendix i).

Fulfilling the co-production ethos, this project included those with lived experience, community leaders, and professionals in the planning and implementation. For the research team, physical engagement in Shetland on two occasions, facilitated working with parents and key stakeholders, added to the depth of information, and understanding we have on the context of Anchor and how it works. Working with a wide spectrum of the Shetland community, we have enabled an ongoing dialogue with the community to analyse, challenge, innovate and mainstream local solutions and implement ongoing evaluation which produces locally designed change and solutions to Shetland and its islands.

Within an action research framework we have used Photovoice³ (i.e. participant-generated images as a data collection and generation methodology) with both parents, who were supported by Anchor project workers, and the Anchor project workers themselves. Photovoice being a qualitative method used in community based participatory research⁴. Using photovoice helped people record, reflect, and document their realities of experiencing Anchor support using images and voice notes (Parents (Appendix ii) and project workers (Appendix iii) photovoice data) which can be a more participatory, open and less intimidating form of eliciting participant perspectives.

Phase One: Anchor Mapping (data collection January 2023)

We conducted a two-day workshop with eight Anchor project workers. We also interviewed six parents supported by Anchor and two headteachers who have strong links with Anchor. A project worker, at the request of the parents, was present during the interviews/conversations so the researchers were able to observe the interactions between the parent and project worker. For example, we noted the relationship and bond between the parents and the project workers which was calm, supportive, non-judgemental and parent led. In one such instance, the project worker took a child to school when their mother was struggling to cope that morning while remaining encouraging and engaging with the child’s mother.

As at October 2022, 48 parents/families had engaged with Anchor and nine key workers who worked at various times over the duration of funding for Anchor. The staffing numbers had altered because of staffing changes and appointments over the duration of data collection for the project (January – October 2023). Our original intention was to have workshops with 8-10 parents, but none of the proposed parents felt comfortable enough to participate in a group, despite this, six were happy to communicate on a 1-1 basis with us. Three of those parents submitted photovoice data and we used these to base the dialogue with those who had submitted images.

We were informed of the importance of the participating primary and secondary schools who are key to identifying children whose parents/families might benefit from support. We held two interviews with two headteachers who were engaged with Anchor, at an early stage in its evolution, to understand the role of schools in enabling and supporting parents’ access to support from Anchor.

Using the photovoice images and data generated by eight project workers, six parents and two headteachers we addressed the following questions:

• What is delivered as Anchor specific activity?
• Who is involved in the delivery of this activity?
• What development processes and communications underpinned this activity?
• Map of demand and local contextual factors across the islands
• Who has benefitted and how from Anchor activity - clients as well as other service/ system stakeholders? What are perceived as valued outcomes of this activity?
• Stakeholders’ perceptions about who has missed out on benefitting from Anchor activity?
• Identification of the critical success factors in the delivery of Anchor in Shetland?

The research team mapped and produced preliminary Anchor programme theory/ logic models detailing Context, Mechanisms and Short and Medium-term Outcomes (Appendix iv). These were shared with the project workers for accuracy, feedback, and refinement. We also asked the project workers to identify a ‘Day in the life of …….’ to help the researchers further understand what Anchor delivers and how Anchor workers operate typically day to day to provide further information on the nature and extent of the support provided (Appendix v).

Phase Two: Mainstreaming the Change (data collection February 2023)

We conducted two in-situ workshops with key leads/managers, community stakeholders and project workers to establish their views about:

• The relevance and applicability of the Anchor project for their communities / settlements based on their reflections of the Phase one mapping undertaken by the research team.
• Perceptions of barriers or risks to implementation of the Anchor-approach in non-Anchor sites.
• Views about the facilitators and opportunities that exist to introduce Anchor in non-Anchor sites in Shetland.

We produced a logic/theory of change model which integrates our findings (Appendix iv). From this we extracted our preliminary co-produced set of findings to facilitate our discussions about next steps in embedding Anchor-based approaches beyond those sites currently using these (Phase three).

Phase Three: Mainstreaming the Anchor approach to other Island/Scottish contexts (5^th October 2023)

A key aim of the Anchor evaluation (Phase three) was to identify the opportunities and challenges from the learning of Anchor to mainstreaming the Anchor approach to other Island/Scottish contexts. A symposium was organised in association with the National Coordinator (Hanna McCulloch, of the Improvement Service) who is the national lead for the Local Action on Child Poverty Peer Support Network, which brings together colleagues from across local, regional, and national bodies, to discuss Anchor and how its evaluation findings might apply to other areas in Scotland. Three meetings were held with various Scottish Government officials representing policy directorates concerned with Child Poverty, Social Inclusion, Whole Family Wellbeing Support, the Islands, and stakeholders from Improvement Services Scotland. Through those meetings it was agreed to organise a one-day symposium on the 5th of October 2023 in collaboration with the Child Poverty Network.

The purpose was to share learning from Scottish Government funded Pathfinder pilot programmes based in Dundee, Glasgow and Clackmannanshire alongside presenting the Anchor project evaluation findings and allowing the evaluation team to run two workshops to address the Phase three questions around mainstreaming the Anchor approach.

This was attended by around 80 delegates who represented:

20 local authorities – Angus, Clackmannanshire, Dundee, East Renfrewshire, East Lothian, East Ayrshire, Edinburgh, Falkirk, Inverclyde, Moray, Glasgow, North Lanarkshire, North Ayrshire, Perth and Kinross, Renfrewshire, Shetland, Stirling, South Lanarkshire, West Dunbartonshire, West Lothian.

Five health boards - Ayrshire & Arran, Forth Valley, Lothian, Greater Glasgow and Clyde and Lanarkshire.

National bodies represented included the Improvement Service, Public Health Scotland, COSLA, Scottish Government (including Child Poverty Delivery Unit, COVID Recovery Team and Public Sector Reform Team).

A presentation contextualising Anchor and the key findings from Phase one and two was underpinned by asking participants to consider the following questions as they listened. Two facilitated workshops followed to discuss the questions in depth:

Reflecting on our briefing paper and our presentation:

1 What, if anything, of our findings resonated with you, in relation to your local area and your work?

2 Do you think the Anchor approach could work in your area?

a If yes, why is that?

a If no, why not?

3 What do you think would need to be in place, in terms of supports (e.g. organisational) for Anchor to work in your area?

Post it notes, field notes and facilitators observations were collated, and the three members of the research team held an analysis workshop (November 2023) to consider the feedback and findings. The data from the workshop and findings from Phases one and two have been synthesised to produce the ‘Ten Anchor Enabling Factors/considerations for implementation in other contexts’ and these are reported in Section 7.