The Realigning Children's Services Programme: evaluation

Executive Summary

Background

On behalf of the Scottish Government, CELCIS (the Centre for Excellence for Looked After Children in Scotland) commissioned Blake Stevenson to evaluate the multifaceted Realigning Children's Services (RCS) programme.

RCS is a two-stranded programme. The evidence strand gathers data on the experiences of children and young people and maps expenditure on children and family services. The second strand provides development and facilitation to Community Planning Partnerships (CPPs) to improve their joint strategic decision making in relation to children's services.

The evaluation aimed to improve the effectiveness of RCS by identifying lessons learned, determine the impact of the programme and provide recommendations for its future delivery. While RCS now works with a third tranche of CPPs, this evaluation only relates to the five CPPs in Tranches 1 and 2.

The evaluation approach included a secondary analysis of RCS documents and primary research through fact-finding telephone calls, visits and interviews with local stakeholders in each area, and interviews with the RCS programme team and their line manager.

Experiences of the Realigning Children's Services Programme

The evaluation showed that the RCS programme has engaged, encouraged and enabled CPP partners and local stakeholders to gather and use data for local service planning and improvement.

CPP stakeholders were knowledgeable about RCS programme principles and whilst there was an understanding of, and commitment to, RCS at a senior level, there was sometimes a disconnect between those tasked with delivering elements of the programmes.

In order to clarify the purpose and fully reflect the programme, the emphasis of RCS shifted from supporting CPPs to improve joint strategic commissioning to supporting CPPs to make better joint strategic decisions about improving outcomes for children and families. There is now a description of RCS and its five elements that is used consistently in communications with all three tranches.

The evidence programme

The data and evidence to map current need, services and expenditure is drawn from the Child Wellbeing Survey Programme, the data linkage and the mapping exercise.

Undertaking the school survey and securing the involvement of the majority of primary and secondary school head teachers was a significant task successfully achieved in all CPPs with assistance from the RCS team. However, the parent survey was a costly and resource-intensive element of the survey programme. The RCS team recognised that the data was limited by its sampling method and size, so it was removed from Tranche 3 and the resources were reinvested.

The self-reported data from each survey response were linked to a set of additional fields drawn from Social Work and Education records in each area. This created a single data set that contained information about child wellbeing and need, and the receipt of children's services.

The mapping exercise aimed to comprehensively map children's services in each CPP and the amount of expenditure on statutory services across education, social work, health, leisure and culture, as well as funding to voluntary organisations to deliver projects focused on children and families. While there were some examples of successful contributions, CPPs struggled with the mapping exercise. The RCS team recognised that making stronger links to how CPPs could invest in preventative work and make long term savings would be more useful for those bringing the information together.

The outputs from the survey programme included a set of school reports, cluster reports for groupings of schools and an overview report for each CPP containing a summary that identified key themes and messages of service use and need to inform decision making. Each CPP received a fully anonymised dataset and a visualisation tool to further analyse the data.

The development and facilitation programme

As part of the RCS programme, development support and facilitation helped CPPs to create the environment and conditions to effect the change needed for improved and more targeted children's services.

The content and delivery of the three development days were described as high quality and they provided the rare opportunity to learn about and discuss a wide range of issues with local colleagues. Interviewees felt that the timing of the three days could change so that they occurred at a point in the programme when the learning could be applied; this has been adjusted for Tranche 3.

Each CPP received tailored support from the RCS team. CPP stakeholders appreciated the engaging and knowledgeable support that the team provided and this helped to maintain local momentum and guide them on how to use the new evidence to inform decision-making.

The workshops and learning exchanges were enjoyed by all interviewees and they would welcome more opportunities to connect with others and share learning.

The RCS national team were perceived as responsive and adaptable, and they worked in a less conventional way to support and challenge the CPPs. Whilst there was some frustration about the personnel changes within the team and the current vacancies, all CPPs were keen to see the RCS team being involved over a longer period of time. Some CPP stakeholders also felt that the RCS team and its work could feed in better to other Scottish Government programmes to maximise learning and inform policy.

The impact of the Realigning Children's Services Programme

By engaging in the RCS programme, stakeholders in all five CPPs reported to having improved their:

• collaboration and multi-agency working;
• understanding of local need and how that aligns with service provision;
• use of evidence to inform decision-making;
• collective understanding of local priorities; and
• knowledge and skills about ways of bringing about change.

Undertaking the survey programme and mapping exercise has shown local stakeholders that they can collect data on this scale across their CPP, work across agencies to bring information together in one place, produce valuable information and use locally held information to build a more insightful picture.

Although challenging to undertake, many contributors saw the value of the mapping exercise in bringing together information about, and increased awareness of, children's services and the investment in services across various sectors in the CPP.

With respect to the survey data, schools used it in several ways, such as informing their school improvement plans and making decisions about the use of their Pupil Equity Fund. At area and CPP levels, cluster reports and full datasets were used for various purposes, such as creating school profiles, informing local outcome improvement plans and identifying new priorities. One CPP amended its Children's Services Plan and in another they used the data and evidence in a successful bid to the Big Lottery Fund.

Outputs from the survey programme, alongside locally held data, helped CPP stakeholders to review their services and to encourage discussions about how services could better meet children's needs.

The experience of collecting information from local stakeholders has increased knowledge of how to engage particular organisations and professionals and improved understanding of individual agency structures and processes. This will aid CPPs when communicating and conducting joint work in the future. It also raised awareness amongst local stakeholders of the need to take an evidence-based approach to planning and delivering services.

The development days, learning events and support from the RCS team has increased CPPs' knowledge and awareness of methodologies for improvement and implementation, developed a shared understanding of local issues, led to more meaningful partnership working and created a readiness for a changed approach to meeting children's and families' needs.

Facilitated meetings, learning exchanges and discussions have resulted in increased joint working and improved relationships between agencies, which has in turn led to increased collaboration at the local level. This will continue beyond the RCS programme.

Having a Scottish Government team flexing their role to address individual needs added value to the programme. Their knowledge and provision of practical help enabled CPP stakeholders to maintain focus and momentum.

Lessons learnt

There are key lessons about the factors that enable CPPs to engage in and progress through RCS:

• Leadership – visibility and involvement at senior level helped to maintain momentum and then lead or secure joint strategic decision making;
• History of partnership working – areas with a strong history of collaboration had more established relationships and greater mutual understanding of each other, leading to a common understanding;
• Motivations for participation – commitment to the programme was maintained when there was a greater perceived need for change;
• Response to austerity – this had both positive and negative effects on CPPs' abilities to work collaboratively with local partners;
• Effective local engagement – often aided by existing structures and established local relationships, and sometimes by geographical factors;
• Top-level stability – administrative changes and changes in senior leadership during RCS had a negative effect on programme activities.

Evidence from RCS is beginning to shape local policy and there is now a desire for evidence-based decision making and a confidence that local partners can work together to reach decisions and take action. As well as building capacity within the CPPs, the RCS team members developed their own skills and knowledge which can also benefit other policy colleagues and inform the policy cycle.

Recommendations and future considerations

Based on the evaluation findings there are immediate and short term recommendations that could improve the programme's impact, some* of which have already been made for Tranche 3:

• Recommendation 1: The RCS team should be cognisant of the factors that affect CPP progress, in particular the top level commitment and this should translate and be maintained amongst the senior and middle managers tasked with implementing the programme. Some consideration should be given as to how the executive sponsors can be more visible and active throughout the lifetime of the programme.
• Recommendation 2*: The sample survey of parents of 0-8 years olds was limited in the extent to which the evidence it produced could be used CPP wide and it was expensive to administer and this should be, and has been removed from the programme.
• Recommendation 3: The mapping exercise needs to have more immediate benefits for those completing it and a better understanding of how it will be used. Specialist support to analyse the information and model changes and their short to long term cost savings is likely to encourage completion of the map and provide more use of it by the CPPs.
• Recommendation 4*: The three development days would have an even greater impact if they delivered at times that relate to the CPPs' stage within the programme. This is a change that will benefit Tranche 3.
• Recommendation 5: The workshops and learning exchanges provide opportunities for CPP representatives to share learning and offer mutual support were particularly useful to CPPs. There could be more frequent opportunities for the tranches to come together and learn from each other.
• Recommendation 6: Once a CPP has completed the programme, the RCS team should continue to maintain a level of engagement with the team. This could involve an annual visit to discuss progress, or remaining as a consultancy resource (capacity permitting) available to CPPs engaged in repeating aspects of the initial programme themselves. This engagement between RCS and CPPs will also assist in the ongoing evaluation of the programme.
• Recommendation 7: The RCS team play a critical role in the programme and the current vacancies need to be filled as quickly as possible with the appropriate staff, particularly the RCS manager post, to avoid a loss of momentum and progress with Tranches 2 and 3.
• Recommendation 8: Scottish Government should maximise the knowledge gain within the team so that it can inform the policy cycle within other areas of the Children and Families Directorate.
• Recommendation 9: Scottish Government should consider how to strengthen the links between the Children's Service Planning and the key messages and evidence arising from the RCS programme.

In thinking about the future direction of the RCS programme and links with the wider health and wellbeing policy agenda, there will be future developments that could impact on the need for all RCS elements.

The roll out of the national school health and wellbeing census will provide CPPs with access to the school population data. If CPPs are given the appropriate support to analyse, interpret and act on the census data then the Child Wellbeing Survey Programme element of RCS may not be required.

In the future Children's Services Planning should be more established and the plans should be built on robust information about needs, costs and services. To do this CPPs will need to undertake some form of mapping, therefore the mapping exercise within the RCS programme would become unnecessary.

However, the much valued and key element that has helped the CPP stakeholders to drive change is the programme support which includes the bespoke offering from the RCS team. There will still be a need for these elements and the facilitative, supportive but challenging approach that brings partners together. The model for this delivery of a programme of events and the operation of a 'nimble' consultancy team could then be reviewed to see if Scottish Government is best placed to deliver and learn from the programme or if another agency could fulfil this role.