The Realigning Children's Services Programme: evaluation

Chapter 3: The impact of the Realigning Children's Services programme

This chapter considers the changes that interviewees thought had occurred as a result of the RCS programme; how each programme element has supported these changes and the factors that helped CPPs to engage in and achieve the short and medium term outcomes.

Short term outcomes

The RCS programme, delivered over an 18 month period, aims to bring about changes that enable local planning partnerships to use evidence in their joint strategic decision making. By engaging in the RCS programme, stakeholders across all five CPPs in Tranche 1 and 2 of the RCS programme reported to having improved their:

• collaboration and multi-agency working;
• understanding of local need and how that aligns with service provision;
• use of evidence to inform decision making;
• collective understanding of what the local priorities should be; and
• knowledge and skills about approaches that can bring about change.

The extent to which these improvements have taken place relies on many local factors that will be explored later in this chapter. However, the local stakeholders in CPPs identified that changes that had taken place in their CPP areas were linked to the combination of the elements of the RCS programme.

The impact of the data and evidence element of the programme

Notably, the CPPs reported that the experience of undertaking the key activities of the survey programme and the mapping exercise has shown local stakeholders that they can:

• gather data on this scale across their CPP;
• work across agencies to draw information together in one place;
• produce valuable information; and
• build a more insightful picture by using existing locally held information.

Whilst the mapping exercise was a challenge for all CPPs, many contributors could see the value of the activity. For the first time it brought together information about children's services delivered across different sectors in the CPP. This alone increased awareness and knowledge of the services, particularly those being delivered by other agencies. For example, a third sector organisation in one CPP found the mapping very helpful in understanding what was available, where the gaps were in local provision and where they could maximise the impact of their work. Other stakeholders used the map to look more closely at finances, to fully understand where core and discretionary funding was being invested.

Although the visualisation tool had not been rolled out to Tranche 2, the survey data in all five CPPs was being used in various ways to support the development of children and families services.

In the schools, as idenitified by headteachers and education staff, there were examples of using it to:

• inform their school improvement plans;
• make decisions about the use of their Pupil Equity Fund;
• use of the questions for in-house pupil surveys; and
• explore some of the survey themes in more detail.

For example in one school where the perception of the area and sense of safety was highlighted in the data, the school is working with local stakeholders to use the Place Standard with pupils to explore the issues in more detail and identify actions that could improve perceptions.

At the area and CPP level, there was variety in the use of the cluster reports for school groupings and full datasets. This included using them to:

• inform local outcome improvement plans;
• improve knowledge of the population for other partnerships like the Community Justice Partnership;
• create school profiles that includes attainment and the RCS data for every school;
• inform school mini-inspections by including sample surveys based on questions from RCS and comparing the results;
• create a partnership dashboard of measures using live and RCS data for the CPP to access; and
• amend the Children's Services Plan and add new priorities based on the evidence from the RCS data, as one CPP had done.

There were also unintended outcomes from the data and evidence element of the programme. One CPP saw an increase in the uptake of the SALSUS survey and they attributed that to the work they had done, the previous year, with the schools to engage with the RCS survey. The same CPP had also used the RCS data in a successful £750k bid to the Big Lottery's Early Action System Change (EASC) Fund looking at a test of change at the early stages of youth justice.

The outputs from the survey programme, linked with locally held data, provided another lens through which CPP stakeholders could explore and review their services. The local stakeholders found this layering of data hugely helpful. For example, in one CPP, the data was showing wellbeing gaps for girls, not just in relation to attainment and physical health but across all of the survey domains. This was powerful evidence to encourage discussion amongst partners about girls' mental health and the factors that were affecting it and how services could better meet those needs.

The experience of gathering information, or securing support to do that, from local stakeholders has also led to a greater understanding of how best to engage particular organisations and professionals, such as head teachers and third sector colleagues. It has also raised awareness of individual agency structures and processes. This knowledge will inform the approaches used by CPPs in future when communicating or carrying out joint work.

Underpinned by the programme support, the achievement of gathering whole school population data, undertaking the mapping exercise and data linkage to create the new evidence set, had convinced many stakeholders of the need to take an evidence based-approach to planning and the delivery of services. This is of particular relevance to the forthcoming Health and Wellbeing Census that will be piloted in 2019.

The impact of the programme support

In order for the CPPs to make better joint strategic decisions about how to improve outcomes for children and families, the stakeholders needed support that would facilitate and strengthen partnership working lead to opportunities to explore and discuss changes that would best meet their local needs, and provide knowledge of how to bring about change. The development days, learning events and the support from the RCS team created the conditions for stakeholders to do this. As a result, amongst the CPPs, there is:

• an increase in knowledge and awareness of methodologies for improvement and implementation;
• a shared and common understanding of a range of local issues;
• more meaningful partnership working; and
• a readiness for a changed approach to meeting the needs of children and families.

As described in Chapter 2, the development days provided opportunities to learn about and discuss approaches that could bring about change, and the quality of this input encouraged and motivated participants to apply that learning. The CPP participants also benefitted from working alongside the RCS team members. The RCS team brought skills and expertise and several interviewees felt that they had been upskilled as a result of that regular engagement and opportunity to learn.

The RCS data and evidence was considered and discussed in various sessions. By exploring the themes together and reaching a common understanding of what the local priorities should be and how these could be addressed, there was a collective agreement and confidence about what was needed. This shared view amongst the CPP stakeholders created a strong foundation to build on and make joint strategic decisions.

Although CPPs are well established and the integrated joint boards have brought together health and local authorities, there is still solo working. The programme's facilitated meetings, learning exchanges and discussions where partners learnt together has led to increased joint working. This fostered better relationships between agencies, for example between health and education colleagues, which led to increased local collaboration which stakeholders will continue to build beyond the lifetime of the RCS programme.

The added value of the RCS team in the programme of support was the ability of a Scottish Government team to flex their role to address the individual needs of the CPPs when necessary. The team shared their knowledge and expertise and provided practical assistance, from challenging stilted progress to facilitating honest discussions between stakeholders. The CPP stakeholders viewed this support as a unique element of the programme that maintained their focus and momentum, encouraged them to embrace the needed change, and helped to move them towards joint strategic decision making and eventually, commissioning.

Lessons learnt about the factors that helped CPPs to engage in and progress the programme

The local context was a crucial element in the extent to which progress was made in each CPP. A number of factors heavily impacted of the CPPs' readiness to embrace the programmes principles and ability to undertake key programme activities and collaborate to achieve short term changes. These factors are summarised in the diagram below and discussed in turn.

Leadership – whilst all CPPs had top level commitment, the visibility and involvement in the programme was needed to help maintain momentum and then lead or secure joint strategic decision making. For those involved in operationalising RCS, without the direct link and support to the Executive sponsor, taking forward CPP level changes could be delayed or they could stall.

History of partnership working – in areas with a tradition of working well with their partners, e.g. health colleagues, the relationships were more established and agencies had a better understanding of each other. This helped in reaching a shared and common understanding.

Motivations for participation – the factors that triggered a CPPs' eagerness to be part of the programme varied slightly. For example, for some the driver was a poor inspection report and real need for change to address required actions. For another, who had considered the Darlington model, RCS was a more affordable alternative. When there was a greater perceived need for change, this helped maintain the commitment to the programme and to progress actions and decisions.

Response to austerity – all CPPs were working in an environment that required significant savings to meet the reduced budgets. For some, this meant that they were focused on core 'business' which impacted on their ability to respond to and work collaboratively with their local partners. Others saw that the RCS work could make collaborative gains that lead to efficiencies and savings.

Effective engagement at a local level – the scale of the involvement of local partners varied. In the CPPs covering smaller geographical areas it was generally easier to make contact (for example with the schools) but often the effectiveness of the engagement was a result of existing structures and systems for awareness raising and communicating and again, the established local relationships. Where these were in place and robust, securing the broader local commitment to the programme was a smoother journey.

Stability at the top level – a change of administration or the senior leadership team part way through the programme had an impact. The programme activities would pause whilst key personnel or local authority power dynamics changed. This meant they needed to be brought up to speed and interest needed to be reignited before real progress continued.

The medium to long term impact of the RCS programme

The RCS logic model identifies the medium to long term outcomes as:

• an embedded local infrastructure and capacity across Scotland to enable evidence informed joint strategic decision-making;
• an established culture of evidence-informed joint strategic decision-making across Scotland; and
• Scotland being seen as a leader in how we plan and commission children's services

It is still early days in terms of the programme's impact on the evidence informing joint strategic decision making. However, as described in Chapter 2, there are several examples of how the evidence is shaping local policy, and one CPP has made the joint strategic decision to change its Children's Service Plan as a result of their RCS data.

As these examples highlight, in some CPPs there is now an appetite for evidence-informed decision making and a confidence that the local partners can use the tools they now have to work together to reach decisions and take action. The extent to which the programme has supported joint strategic commissioning will need to be determined over a longer period.

Whilst the logic model did not identify outcomes for the RCS team, as well as building capacity amongst local CPP partners and providing challenge and scrutiny, the team has developed skills and knowledge that can benefit other policy colleagues and inform the policy cycle.