Health and Wellbeing Census proposal

Options appraisal

The original proposal for introducing a HWB Census dates back to 2015, as relevant national and locally commissioned surveys had been set up by the Scottish Government, individual local authorities and health boards. These collections did not provide sufficient, consistent or timely enough data to support annual reporting at local authority level to meet current and future obligations; exploration of inequalities within and across local areas; or service improvement activities at very local level. The disparity of sources and approaches to collecting the data meant that there was little potential for using the data systematically to enable national level evaluation of policy initiatives.

In 2015, a short-life Working Group proposed conducting a structured ‘options appraisal’. The membership of this group included colleagues from Scottish Government, NHS Health Scotland (NHSHS), Public Health Intelligence (PHI), Education Scotland, COSLA, the Improvement Service, one Local Authority, Greater Glasgow & Clyde Health Board and Young Scot.

Following Ministerial approval, an options appraisal on the future of children and young people’s health and wellbeing surveys in Scotland was undertaken to develop a better approach to

• meet data users’ needs,
• reduce the burden on schools and young people, and
• improve the quality of the data produced.

The options appraisal found a strong case for collecting comprehensive, high quality data about the health and wellbeing of children and young people. It noted there was a pressing need for information on children’s health and wellbeing driven by the need to capture data on wellbeing for GIRFEC, indicator data for the associated SHANARRI framework, evidence requirements for Single Outcome Agreements, evidence for children’s services plans introduced by the Children and Young People (Scotland) Act 2014 and data for the Child Rights and Wellbeing Impact Assessment (CRWIA).

There was also a clear interest and demand from health professionals and the academic sector for robust data on many aspects of children’s lives. It also noted the importance of understanding the circumstances, characteristics, behaviours and experiences of children in Scotland. The goal of raising attainment for all children and closing the attainment gap between children in deprived and better off areas signalled the need for robust and comprehensive evidence – not only about attainment itself, but other aspects of children’s lives that may shed light on patterns of differential attainment and how those might be addressed.

The findings noted that to respond to the demands, data would be needed at a range of geographies, from the national to the very local. To tackle inequalities, other types of highly disaggregated data will also be required - data that allow a focus, for example, on the experiences of children and young people with a disability, on those from minority ethnic groups, or those living in poverty.

It was also an opportune moment to examine the potential for greater use of data linkage, how data collection is commissioned and managed, how the transfer of evidence into policy and practice is supported, and to declutter the plethora of surveys that were being used nationally and locally to understand the health and wellbeing of children and young people.

The recommendation from the options appraisal was for a new national survey that would sit at the centre of a streamlined school survey landscape, allow for greater synergy with other data collection and maximise the value of the data. Following an additional feasibility study in 2017, the recommendation for a single national survey was amended to become that each local authority would collect its own data as this would reduce the cost of survey fieldwork; provide timely access to data by local authorities; and provide flexibility for use at both national and local level. This approach was discussed and agreed with the local authorities (see chapter 4 Approach to data collection).

The options appraisal identified key features of the new survey as:

• A census, rather than sample, of participating year groups. This would enable small area and school-based analysis across Scotland, and make it possible for schools to be clustered, locally and nationally, on key institutional characteristics. It would also greatly enhance the analytical possibilities of the survey by enabling highly detailed sub-group analyses to provide evidence on smaller groups. In 2021/22, all pupils in P5 – S6 in local authorities who were conducting their HWB Census were invited to take part.
• A biennial cycle of data collection and analysis. The survey would operate on a biennial cycle, which would allow repeat cross-sectional analysis of whole year cohorts. It would reduce the timetabling demands on schools (compared with an annual collection) and would also allow the data collection to be scheduled alongside other surveys such as the Health Behaviours in School‑aged Children (HBSC) and Programme for International Student Assessment (PISA). As this new survey would provide flexibility for locally-derived questions and offer a large local sample size, the need for the local commissioning and funding of additional school surveys would be reduced. However, the frequency of the Census was left open for discussion and agreement with local partners and stakeholders.
• Data collection at key stages within primary and secondary rather than all school years. This was amended to all P5 – S6 pupils following stakeholder engagement in development.
• A move towards wholly electronic data collection as soon as is practicable. In 2021/22, SmartSurvey was used as the digital data collection platform.
• A core/modular questionnaire design allowing for both national and local needs to be met. The core topic coverage would be structured around the SHANARRI framework, where possible using existing measures that would allow for time series comparisons. A bespoke survey design would allow the survey findings to be tailored specifically to the current policy and outcomes framework. This would provide consistent information across Scotland as a whole and at a highly local level, enabling robust time-series and cross-school/area comparison. The Content Group agreed a set of stage questionnaires, including topic with questions within each topic, structured around the SHANNARI framework, and also including key measures for local and national reporting (for example key substance use measures collected in the Scottish Schools Adolescent Lifestyle & Substance Use Survey (SALSUS) in 2018).
• The modular elements of the survey would allow for specific topics to be covered at just one sweep and also, potentially dedicated space for locally selected questions. Local stakeholders would be able to pursue topics of particular relevance in their area. The approach for this flexibility is set out in Chapter 4 Approach to data collection.
• A voluntary process for both children and parents based on opt-out rather than opt-in. An opt-out approach, which is established practice in school-based surveys of this kind, is expected to yield very high response rates. By contrast, an opt-in approach is expected to introduce very significant non-response bias, effectively meaning that the survey was not viable. The approach taken in the 2021/22 HWB Census data collection is set out in the HWB Census Technical Report Section Background Data Quality Statement.

A key feature of a new national survey was identified as its design, governance and on-going development based on a co-production approach involving a range of stakeholders. To provide real benefits for all those involved, representatives of local authorities and health boards were closely involved in the initial design of the survey, sat on the content group advising on questions, and local authorities were members of the Implementation Group advising on detailed plans for implementation. The effective involvement and engagement of the groups was absolutely essential to maintain the shared ‘investment’ in the survey. All 32 local authorities were kept informed of the topics/themes that were to be included in the Census as the Content Group developed the questionnaires - via the Implementation Group and KnowledgeHub platform. All local authorities were invited to be members of the Implementation Group and had access to all materials shared via the KnowledgeHub platform, irrespective of whether their local authority was planning on conducting their own Census.

The initial options appraisal undertaken in 2015 identified the need for a strong commitment to dissemination, utilisation and support to stakeholders (especially in local authorities and health boards) to access and use the data effectively.

Engagement of the academic sector in secondary analysis of the data to investigate issues of interest was also seen as critical, being an important element in the effective use of the data.

The options appraisal also noted some key risks:

• The costs of the census approach would be greater than those for a smaller sample survey, mitigated in part by implementing electronic data collection in full.
• The logistical demands on schools would be considerable, mitigated through advance planning and a relatively lengthy fieldwork period.
• The scale of the exercise would also place significant demands on local authorities and national government, mitigated in part by a co-production model.

• Without an adequately resourced programme of dissemination and utilisation, the next sweep might be underway before key stakeholders have seen real benefits from the first.

• New or different question wording, together with other changes (relating to overall survey context or question order) affects comparability (or loss) with existing time series measures from SALSUS and other surveys.

The outcome was given Ministerial approval, in September 2015, for a feasibility study into the option of a new data collection based in schools in Scotland, based on the recommendations from the options appraisal.

Feasibility studies

Following on from the above, in 2016, a feasibility study for a new data collection based in schools focused on stakeholder engagement, technical/logistical issues, and questionnaire coverage.

The feasibility study identified the following assumptions for further development:

• The data collection would be a census, not a survey.
• Starting in 2018-19 school year, it would be biennial with HBSC and PISA occurring in the intervening years. Following feedback this was changed to be undertaken every four years, to reduce burden on schools and local authorities.
• It would include P7, S2 and S4 pupils (additional stages if local areas opted to). Following feedback this was changed to include all P5 to S6 pupils.
• The survey would be administered electronically, informed by the feedback from the SALSUS 2015 online data collection.
• It would use data linkage.
• It would be confidential, in that individual children and young people will remain unidentifiable in any published results.
• The questionnaire would have a core/modular format allowing for both national and local needs to be met.
• The questionnaire would be structured around SHANARRI outcomes.
• The questionnaire would be designed with input from national and local partners, including children and young people.

With the launch of the National Improvement Framework for Scottish Education (NIF) in January 2016, the importance of the need for good consistent health and wellbeing data increased significantly as one of the four key priorities as set out in the NIF was to “improve the health and wellbeing for children and young people”.

However, following consideration of the draft plan, an additional feasibility study was undertaken throughout 2017 to explore options for collecting health and wellbeing data for children and young people, and whether the earlier findings still stood.

The 2017 feasibility study involved colleagues from across the Scottish Government, NHS Health Scotland, the Improvement Service, and selected local authorities and health boards (who had conducted their own health and wellbeing surveys). This feasibility study reaffirmed the strong need for consistent and reliable information collected at local level, primarily aimed at informing and driving local improvement, and consistently gathered across all local areas to monitor and develop government policy at a national level.

The feasibility study also concluded that addressing the need to reduce the costs of survey fieldwork, finding a solution that offers both robust and comparable information together with local flexibility, and a quick turnaround of results in order for local partners to use timely information was needed to drive forward local improvements more quickly. 

In October 2017, Ministerial approval was given for the development of a new national Health and Wellbeing data collection to be conducted on a regular basis.  Consideration of costs, frequency, content and coverage of the census were to be taken forward to conduct a census in the 2019/20 school year.