Scotland's children's services plans 2023-2026 review: improving outcomes for children, young people and families

8. Monitoring and evaluating progress

This chapter discusses the monitoring and evaluation approaches set out in Children’s Services Plans.

Monitoring indicators (Criterion 13)

The Statutory Guidance states that a Children’s Services Plan should set clear progress measures and indicators for monitoring and evaluating the effectiveness of services in responding to and addressing wellbeing needs of children, young people and families.

Half of the 2023-2026 Children’s Services Plans (15 out of 30) fully achieved this criterion. This represents a decline from the previous review, when 17 fully met the criterion (Figure 8.1).

CSPs which fully met this criterion provided a detailed description of the approach to monitoring and measuring progress of their Children’s Services Plan, discussing the organisation, service, or strategic group responsible for measuring progress, and the methods or data sources to be used as evidence, such as surveys, evaluation, consultation feedback, population/socio economic data, and service level data. They also listed clear and relevant monitoring indicators attached to actions and priorities. Most used data, notably numbers or percentages as monitoring indicators, and some added targets, timescales or indicated the direction of travel desired, e.g. an increase in the number of people accessing a service or percentage of people reporting positive outcomes.

Six CSPs were deemed to have partially met this criterion; in these cases, there was some effort to describe the approach to evaluating progress, but more detail was required. For example, a few provided progress measures which lacked specificity or referred to sources of data rather than specific measures, such as ‘evaluations and reports’ or ‘feedback’. The 9 CSPs which did not meet this criterion did not include any indicators for monitoring progress.

12 out of 30 CSPs incorporated the Core Wellbeing Indicators from the Children, Young People and Families Outcomes Framework (CYPF OF) into their approach to measuring progress.

Examples of good practice

Aberdeen City: Priorities are linked to measurable indicators with lead agencies given for each measure. A number of the Children, Young People and Families Outcomes Framework (CYPF OF) Core Wellbeing Indicators have been included alongside local indicators. A wide range of outcome and service quality indicators are used, which feed into specific targets/deliverables, such as: “Increase the % of children assessed at 27-30 months from 83% to 95% by 2024” and “Increase the number of ELC settings achieving good or better from 83.7% in 2022 to 90% by 2025.” The summary report contains a chart which shows an overview of the progress being made to improve outcomes against each target, reflected as red, amber or green.

West Lothian: The Children’s Services Plan maps each priority against a set of performance measures; these include a mix of local monitoring indicators and the 21 Core Wellbeing Indicators from the Children, Young People and Families Outcomes Framework. For example, under the ‘Raising achievement and attainment’ priority, monitoring indicators include: “% of school leavers that achieved five or more level 6 awards or above” and “% school leavers in positive destinations at 9-month follow up.” Some of the measures included align to other programmes of work such as HENRY outcomes, and delivery actions from the Corporate Parenting Action Plan.

South Ayrshire: The ‘How will we know if our journey was successful?’ section of the Children’s Services Plan sets out 10 clear indicators for monitoring progress. These include: children (0-5 years) meeting expected developmental milestones increases; children, young people and families report experiences of mental health, children with additional support needs and wellbeing are listened to and supported; staff report increased knowledge and skill to support mental health and wellbeing. Each priority has a suite of specific and relevant performance measures attached.

West Dunbartonshire: The Nurtured Delivery and Improvement Group is responsible for reporting on progress and references to monitoring and evaluation mechanisms are embedded in the Children’s Services Plan. A wide range of indicators of progress and performance are mapped against each of the priorities. These include ‘improved positive destination rates’ and ‘percentage increase of staff trained in Mental First Aid’.

Additional Policy and Engagement Context

Improvement activity to streamline and join-up outcomes and data on child and family wellbeing has remained a key theme within discussions at the Children’s Services Planning Strategic Leads Network, as well as at the Children and Families National Leadership Group.

The Children, Young People and Families Outcomes Framework (CYPF OF) has been developed to provide an overarching understanding of children and young people’s wellbeing in Scotland. This complements the National Performance Framework, and has a holistic approach grounded in Getting it right for every child (GIRFEC). With children’s rights under the UNCRC at its core, the framework’s approach is consistent with national and international definitions of child wellbeing.

The framework aims to embed a more cohesive approach across the suite of outcomes and data relevant to babies, children, young people and families, by drawing together different aspects of the policy, delivery and practice landscape to tell the story of child and family wellbeing in Scotland. As well as providing assurance on what is going well, the framework informs decision-making by highlighting where further improvements are needed. The CYPF OF sets out a strategic approach which aims to support greater consistency within, between, and across national and local reporting; to better coordinate data development and data improvement activity, build capacity, reduce duplication of effort, and maximise collective use of resource.

Further information on the Children, Young People and Families Outcomes Framework is set out in Appendix C.

Core Wellbeing Indicators

The 21 core wellbeing indicators which form part of the CYPF OF provide a level of data consistency within local Children’s Services Plans (CSP) and annual reports. The core wellbeing indicators were agreed by key stakeholder and governance groups as part of the wider CYPF Outcomes Framework development work, and these were made available for use at national and local level as from April 2022.

Children’s Services Planning Partnerships (CSPP’s) were encouraged to use these indicators as part of their 2023-2026 Children’s Services Plans and annual reports for 2022/23 onwards as far as possible - recognising that not all local areas had taken part in the Health and Wellbeing Census. These indicators provide a consistent ‘core’ within shared reporting across all CSPPs which is intended to be supplemented by additional deep-dive and locally available data, as appropriate to national outcomes and to local strategic priorities and service planning. This collective data informs delivery monitoring, evaluation of impact, and progress reporting on how improvements to the wellbeing of children, young people and families are being driven by the collective leadership and delivery of area’s Children’s Services Plan.

At national level, the Core Wellbeing Indicators aim to provide some consistency between and across policies which have a contribution to the wellbeing of children, young people and families, as well as in national and local level reporting. By providing a high level and holistic shared picture, the indicators facilitate progress monitoring of the collective impact national policies, strategies, and delivery or action plans are having on the wellbeing of children, young people and families. They also help to inform cross-directorate areas of policy across Scottish Government portfolios through an understanding of wellbeing gaps for particular groups of children and young people, to consider any action required to address these, drawing on additional evidence and data as required.

The national report ‘Children, Young People and Families Outcomes Framework – Core Wellbeing Indicators: Analysis’ was published in September 2023. This report presents headline findings, key socio-demographic breakdowns and time series data, where available, to establish an initial baseline against which progress can be measured in future years, as well as providing a benchmark national picture of the current status of wellbeing of children, young people and families in Scotland. It is intended that national reporting on the core wellbeing indicators will be repeated at regular intervals.

A further application of the core wellbeing indicators at national level, has been analytical work focussed on eradicating child poverty. Child poverty targets are not an end in themselves, as ultimately, they are about improving wellbeing of children, young people, parents and/or carers, and enhancing quality of life. Evidence shows us that poorer outcomes are driven by experiences of poverty, and so, reducing child poverty through increasing household income and reducing costs of living, is one important mechanism for doing so. But it is not the only mechanism.

Many important actions are being taken forward across the Scottish Government and national stakeholders, as well as by local Children’s Services Planning partners in the public and third sector, to plan and deliver services and supports which improve the experiences and life chances of people who access them. To deepen our understanding of these wider policies on Scotland’s progress towards child poverty outcomes, ‘Child and parental wellbeing: measuring outcomes and understanding their relation with poverty’ was published in March 2024. This report looks at the Core Wellbeing Indicators from the CYPF Outcomes Framework in more detail, by indices of deprivation.

The Children, Young People and Families Outcomes Framework and the approach it aims to deliver, has also been referenced in, or applied to:

• Long term evaluation measurement of WFWF Outcomes (Core Wellbeing Indicators)
• Wellbeing Economy Monitor Reporting (Dec 2023)
• Scottish Government’s Initial Response to the United Nations Convention on the Rights of the Child - Concluding Observations 2023 (March 2024)
• CELCIS Children's Services Reform Research study: Mapping Integration and Outcomes Across Scotland: A Statistical Analysis

Keeping the Promise

The Promise Progress Framework has been developed by COSLA, the Scottish Government, and The Promise Scotland, working with other stakeholders, to answer the question: ‘How is Scotland doing in its progress to #KeepThePromise? Work is underway to align the Promise Progress Framework with other relevant outcomes and data frameworks such as the National Performance Framework, and Children, Young People and Families (CYPF) Outcomes Framework, to establish a shared understanding of what "good" looks like, and to minimise additional reporting burden. The Promise Scotland Change Programme ONE also identified opportunities through the CYPF Outcomes Framework to support improvements in: Policy Cohesion, Governance, and Data relating to child and family wellbeing.

The approach of the Promise Progress Framework recognises that The Promise is about improving the lives of care experienced children, families, and adults, as well improving supporting systems. As people do not experience life in policy or practice silos, neither can work focussed on improving our understanding of their lived experiences, nor the extent to which outcomes are being improved. It is anticipated that the Promise Progress Framework will be available on the Plan 24-30 website by the end of 2024.