National Improvement Framework - enhanced data collection: consultation analysis

Detailed analysis

The Scottish Government invited the views of anyone who is directly or indirectly involved with the Scottish education system aged 3-18 and those who have an interest in the issues covered in the consultation paper to respond to the consultation. We were particularly interested to hear from children and young people, practitioners, school staff, early learning and childcare staff, local authorities, professional associations, universities and third sector organisations.

Question 1

Our proposals for the key measures of progress towards closing the poverty related attainment gap are based on a number of key principles. Are there any other principles that should be included?

In total, there were 63 responses to this question. 49 responded "yes" and 14 responded "no".

Most respondents were supportive of the current key principles and felt that they were clear in their ambition to align with the vision and 4 capacities of Curriculum for Excellence and the need to reflect a progressive shift of what is valued in Scottish education. Respondents felt that there would be value in ensuring that the principles were explicit in recognising and responding to the reform agenda.

There was emphasis on the need for all data-related activity (i.e. local authority stretch aims, NIF key measures and the outcome of the reform process) to be joined up and aligned, and there must be clear milestones and outcomes set out which relate back to the key principles. There must be a clarity of understanding on the use of data at all levels within the system.

Respondents welcomed the reference to the importance of the voice of the learner and 'the need to place the human rights and needs of every child and young person at the centre of education'. Over the course of the pandemic, children and young people felt that they had no direct influence on the decisions made in relation to their learning, and it is vital that they have their say in the conversations regarding reform, improvement and data gathering. The key measures should also be informed by a diverse range of children and young people.

There was a reference to the National Performance Framework and the wider policy framework of the Scottish Government in aligning with the UN Sustainable Development Goals. Some respondents stated that Learning for Sustainability should be mentioned in the principles, given the key concerns from young people around the issues of equity and sustainability, including actions to address climate change now and in the future.

In relation to the definition of 'progress', most respondents agreed that there should be an accurate and balanced picture of attainment and achievement, and the importance of wider achievement should be explicit, and should also reflect the different needs of learners, including those with additional support needs.

Question 2

Should the two sub-measures covering attendance and exclusion at secondary schools be promoted to key measures?

In total, there were 60 responses to this question. 41 responded "yes" and 19 responded "no".

Overall, the majority of respondents agreed that attendance should be a key measure, however, many of the respondents raised concerns about adding exclusion to the basket of key measures because of a perceived risk of creating perverse incentives. Some of the points raised in relation to both attendance and exclusion are below:

Attendance

Most respondents were of the view that attendance should be promoted to a key measure as strengthening the engagement and participation of children and young people in education is key to ensuring that outcomes improve, particularly those who are from deprived backgrounds or with additional support needs. To benefit fully from all that the school experience has to offer, children and young people need to be in attendance. Including attendance as a key measure would also encourage continual dialogue between schools and wider partnerships to focus on working collaboratively on ways to resolve issues relating to low attendance levels.

Exclusion

There was a fairly mixed response in relation to exclusion. Some respondents felt that data on exclusions can provide an important barometer of the culture of a school system, a key factor which needs to be addressed to aid in closing the attainment gap.

However, the majority of respondents were of the view that exclusion is a complex issue with many external factors which would need to be taken into consideration. Exclusion data can also highlight a failure to be inclusive and nurturing, particularly in support of more vulnerable groups of children and young people, or those with protected characteristics. In addition, there would need to be very careful interpretation of the data, particularly given that the number of exclusions are primarily small in comparison to other data sets. Expectations in relation to the recording of the data across all sectors may be required to serve to maximise the consistency in approach and in turn improve the reliability and validity of the data.

Question 3

Should data on confidence, resilience, and engagement from the new Health and Wellbeing census be included in the basket of measures?

In total, there were 62 responses to this question. 43 responded "yes" and 19 responded "no".

Overall, the majority of respondents agreed that data from the Health and Wellbeing Census should be included in the basket of key measures, particularly given the impact of COVID-19 and the need to put wellbeing at the heart of education. More specifically - confidence, resilience and engagement should be included in the basket of measures, as they are all important aspects of wellbeing and 'confident individuals' is one of the 4 capacities of Curriculum for Excellence, therefore it seems appropriate to include as a key measure.

However, it was suggested that more consideration needs to be given to how confidence and resilience are measured as these tend to be subjective and only measure a point in time. Many agreed that if this data is used there should be an agreed set of definitions and metrics for 'confidence, resilience and engagement' between all education partners as, without a shared understanding and application of tools for measuring wellbeing, there is a risk in the inconsistency of the data across local authorities which would not be suitable for national comparison. It was also suggested by some respondents that it is too early to use the HWB census as a data source until there is a greater degree of confidence and consistency in the data. As there is not universal uptake of the survey, there would be issues around validity. If the data was fully representative of young people across the country with equal participation, it would be a worthwhile addition.

In terms of engagement, some felt that any measure in relation to engagement in extra-curricular activities should provide a valid, reliable and equitable measure of the benefit of such activities to young people. This can be challenging given the difficulties of categorising appropriate extra-curricular activities and identifying the relative benefit of different activities. These activities are often voluntary and are determined out with the control of the school. However, a move towards a consideration of how learners are experiencing education, and a focus on other skills and experiences obtained through 'non-traditional' activities would be welcomed.

Questions 4 & 5

At the moment, the measure of achievement in the senior phase is the National Qualifications achieved by young people at the point which they leave school (SCQF levels 4, 5, and 6 – 1 or more on leaving school). Do we need to add other measures to cover wider achievement and attainment?

In total, there were 62 responses to this question. 55 responded "yes" and 7 responded "no".

If you answered yes to Q4, some options for consideration are set out. However, we would also welcome any other suggestions for additional measures.

The options provided were:

Option 1

In response to feedback from users, and to improve the evidence base on the attainment of broader achievements and skills as part of the Curriculum for Excellence, a new 'all SQA qualifications' measure has been developed which includes National Qualifications (National Courses, Skills for Work) and other SQA qualifications (Customised Awards, Higher National, National – Workplace, National Certificates, National Progression Awards, Professional Development Awards, Scottish Vocational Qualifications, Ungraded National Courses). Details can be found in section 6.3 of the School Leaver Attainment and Initial Destinations publication. The 'all SQA qualification' measure details the proportion of school leavers who attained a number of passes (e.g. one pass or more, two passes or more etc.) at a given SCQF level or better across all of the qualifications outlined above. One or more combination(s) of passes and SCQF levels could potentially be used.

These statistics are currently labelled as Experimental Statistics, reflecting that they are undergoing development and subject to revision based on informed feedback from users

Option 2

A measure of attainment in vocational qualifications. Section 6.1 of the School Leaver Attainment and Initial Destinations publication contains a measure covering only 'vocational' qualifications. Unlike the existing NIF key measures on school leaver attainment and the 'all SQA qualifications' measure outlined above, this measure does not include attainment in National Qualifications but focuses on vocational qualifications. Specifically, the measure includes National Certificates, Higher National Qualifications, Scottish Vocational Qualifications, National Progression Awards and Skills for Work. It shows the proportion of school leavers with one pass or more at a given SCQF level. The proportion of school leavers with one pass or more at SCQF level 5 or better is used as a Key Performance Indicator for Developing Scotland's Young Workforce.

Most respondents agreed that the key measures should reflect a broad range of achievement and not be limited to school attainment. There was a range of views on whether either, or both of the options presented in the consultation paper should be added to the key measures. Some of the other points suggested (unrelated to the presented options) were:

• Other measures which cover wider achievement should be included in the basket of measures, such as vocational courses and apprenticeships.
• Vocational qualifications must have parity of esteem and should be included in the 'all SQA qualifications measure'. If vocational qualifications continue to be treated separately from more traditional SQA courses, parity of esteem will never be achieved.
• There needs to be a clear national narrative about the attributes of a successful school system and clearer guidance on the meaning and value of specific awards.
• All course awards must be valued in an equitable manner.

Question 6

In terms of measuring progress beyond school, should the percentage of school leavers going to a "positive destination" on leaving school be included alongside the participation measure?

In total, there were 59 responses to this question. 49 responded "yes" and 10 responded "no".

The majority of respondents felt that positive destinations should be included as a key measure alongside the participation measure. Some respondents that were supportive of this agreed that it would provide an opportunity to monitor progress and identify emerging trends, particularly as this data is also outcome focussed and articulates with the Insight data used by schools which would help to strengthen accountability and better support effective quality improvement in schools. Positive destinations can also be seen as a form of attainment in their own right, therefore it can be argued that they would be a useful measure.

Others mentioned that it is important to ensure that there is clarity and understanding within the system as to what is meant by the term "positive destination". There may also be value in assessing the nature or quality of destination, and how this reflects the Scottish Government's commitment to fair and sustainable work.

However, others stated that there are concerns over the lack of long term tracking of learner journeys, and given positive destinations data only shows a snapshot in time, it does not reflect whether a destination is sustained. The participation measure is a more realistic and helpful indicator, but system wide this should be extended to track participation across one, two and three years post school to indicate how well these levels of participation are sustained by young people over time.

Question 7

What more do we need to do in order to ensure that a wider range of measures are in use across the education system, and that they are valued as equally as traditional attainment measures?

There were a range of views on what more we need to do in order to ensure that a wider range of data and measures are in use across the education system, and that they are valued as equally as traditional attainment measures. The main points from respondents have been summarised below:

• Culture, attitudes and behaviours underpin the ways in which data is used. It is important that expectations are clearly and consistently communicated by stakeholders across the education system in their use of data.
• Improving the quality and robustness of data is key. This needs to be considered both at national and local level.
• There must be a strong focus on ensuring that education professionals are supported in in their capacity to interpret data and how they can use data to drive improvement in their own local authority/school context.
• There is a need for guidance on the meaning and value of specific awards.
• Developing the Insight tool across the primary and secondary sectors would be helpful.
• A wide range of improvement partners (including those within and out with the system) should be brought together to review data from a number of different sources. A clear line of sight can be developed and collaborative approaches to improvement can be formulated.
• Local authorities gather data differently which leads to an inconsistent approach. Consistency could be addressed by a national standard and expectations including a common system for tracking attainment and achievement from ages 3-18.
• Partnership working with stakeholders is crucial to capture the wider range of data to inform a more holistic approach.
• There needs to be a greater recognition of the influence of partners and carers on choices.

Question 8

Are the existing wider data collections, and the new data developments enough to ensure that the National Improvement Framework reflects the ambitions of Curriculum for Excellence, national policy priorities such as health and wellbeing and confidence, and key priorities for COVID-19 recovery and improvement, as recommended by Audit Scotland?

In total, there were 59 responses to this question. 21 responded "yes" and 38 responded "no".

The majority of respondents stated that the existing and new data developments are not enough to ensure that the NIF reflects the ambitions of CfE, national policy priorities such as health and wellbeing and confidence, and key priorities for COVID-19 recovery and improvement, as recommended by Audit Scotland.

For those that responded with "no", the key points have been summarised below:

• A more equitable set of quantitative measures, reflecting the ethos of CfE needs to be developed.
• There needs to be more research and intelligence on the impact of COVID-19 on attainment in schools but most importantly on how it affected the health and wellbeing of children and young people.
• A measurement of young people's engagement in outdoor learning throughout their learner journeys should be reflected in the data sets, given research shows that this form of learning can help in the development of self-esteem, social skills and educational attainment.
• Data from employers and post-school educational institutions such as colleges or universities would be desirable to understand and track the pathways children and young people have taken once they leave school.
• Further links to The Promise could be made and strengthened which would help to ensure that all children have equal access to support and appropriate information is gathered to measure the impact of this.
• Data on its own will not close the poverty-related attainment gap – there needs investment in time and funding alongside central sharing of good practice by national agencies with a supportive, coherent and consistent inspection system.
• A clear narrative about the evidence of a successful learning system, supported by qualitative evidence (e.g. case studies) is needed to help us understand and evidence the impact of a well-designed curricular offer for all young people, supported by high quality learning, teaching and assessment.
• There is a need to ensure that there is a continued balance between the NIF and the subsidiarity that is embedded within the system through CfE and the school empowerment agenda. Professor Ken Muir's report on education reform includes a number of recommendations which are important to consider in the context of the changes to the data collection to support the NIF.

For those that responded with "yes", the key points have been summarised below:

• There are many measures already in place – the ambitions of CfE are covered as well as national and covid recovery priorities. The key issue is how can the system simplify and make sense of what we already have?

Question 9

How can we make better use of data to focus and drive improvement activity at school, local, regional and national level?

There were a number of suggestions from respondents on how to make better use of data for improvement. Most of the responses had similar wording, and these comments have been summarised below:

• There should be continued development of educational professionals capacity to interpret data and understand how to use it effectively to drive improvement.
• We need to reset the culture in which data is collected and analysed and have genuine trust in the teaching profession. The evidence base shows that the system and professionals perform better when interactions and activity is built on trust.
• Enhancements in training for all teaching staff in the primary and secondary sectors.
• Data collection should be supported by a broad and meaningful set of measures that encompass a balance of types of data (qualitative and quantitative) to ensure a complete view of the priorities within the NIF and support improvement activity, whilst achieving equity and should contribute towards closing the poverty related attainment gap.
• Improving the quality and robustness of data is key and needs to be considered at both local and national level.
• Schools hold a large amount of data about young people and this could be used to identify clear patterns and create solutions (i.e. low attendance)
• There should be a focus on the health and wellbeing of teachers and school staff following COVID-19 to look at the impact it has had on them and how we can support them going forward.
• We need to better understand the outcomes and variation in performance across Scotland to drive improvement.

Question 10

How can we make better use of data to help reduce variation in outcomes achieved by young people in different parts of the country?

There were a number of comments from respondents on their views on how to address the variation in performance across the country. Some commented that their responses to the previous question (9) also apply to this question, so these have not been repeated in the points below.

The main suggestions from respondents were:

• Collaboration approaches should be considered, especially in terms of those from similar demographics working together to share good practice. Where data suggests positive impact on closing the poverty related attainment gap, it would be helpful to look at the approaches they have taken and if they could be tried and tested in other areas of the country.
• System improvement needs to be built on a holistic understanding of the range of factors which affect attainment and any strategic approach to mediate change needs to recognise this complexity.
• The education reform process (recommendations from the Muir report and the Hayward review) have the opportunity to review how we address this issue and the next steps that need to be taken.