Qualifications and Assessment Review: consultation analysis

Introduction

Background to the Review

In October 2021, the Scottish Government announced the Independent Review of Qualifications and Assessment. The aim of the Review is to ensure that all Senior Phase learners in Scotland - predominantly learners aged 15-18 - have an enhanced and equal opportunity to demonstrate the width, depth, and relevance of their learning.

The Review, led by Professor Louise Hayward and supported by an Independent Review Group (IRG) including learners, teachers, employers, universities and colleges, embodies an inclusive approach to policy design and will contribute to the development of final recommendations which are both principled and practical. Following the first phase of consultation on vision and principles via schools and colleges and CCGs, a phase began in October 2022, including a public consultation hosted on the Scottish Government's Citizen Space portal between October 2022 and January 2023.

Containing 11 questions, the consultation allowed learners, parents, teachers and stakeholders to express their views on how the vision and principles established in Phase 1 might be implemented. Questions covered: what information about achievements might be gathered and how; the best balance for a future assessment system; proposals for introducing certification at different stages of learning; the potential role of technology; and ensuring the achievements of all learners are recognised.

This report provides a thematic analysis of responses to the Qualifications & Assessment Review. Our approach to the analysis and report structure are outlined below.

Respondent profile

This report presents an analysis of 708 consultation responses, which contained over 5,500 open-ended comments. Within this sample, there were 526 responses from individuals and 182 from various organisations, including many schools.

The analysis also included notes from 22 CCG discussions facilitated as part of the Independent Review. These discussions typically considered the same questions as the consultation but focused on how the proposals could impact different stakeholders, including school leaders, teachers, parents, colleges, trade unions, education bodies, and employers. We have also reviewed data from a survey of parents and carers, with 164 responses.

Individual respondents were not asked if they were learners, teachers or parents, but the analyst team created a separate category of learner/teacher/school responses, where the respondent type was clearly identifiable, for analysis purposes. The 262 responses in the learner/teacher/school category include responses from schools and individual responses from email addresses with a Glow or sch suffix. As we cannot be certain of the status of other individual responses, they remain classed as individuals.

A number of consultation responses were based on discussions with, or submitted on behalf of, multiple individuals. Very few responses, however, specified the number of individuals they represent, so we cannot be certain of the total number of people reached by the consultation or represented by each response.

In particular, different approaches were taken by individual schools and colleges to completing the consultation. For example, learners in one school may have submitted their own individual responses as part of their lesson. Whilst in another school, a single response may have been submitted which incorporated the feedback obtained through multiple classroom discussions and potentially included the views of hundreds of learners and teachers. Similarly, responses from trade unions and professional associations were prepared by multiple individuals and are intended to represent the view of their membership. The Collaborative Community Groups involved discussions with many individuals but a single response was submitted on behalf of the group.

While responses have come from various individuals and organisations, it is unclear how many people are represented by each response. For example, a class from one school may have discussed the questions and submitted one response, whereas learners from another school may have submitted their own individual responses as part of their lesson. For analysis purposes, each submission has been treated as a separate response, and all views are included in this report regardless of whether a large or small number of respondents raised them.

The table above shows the percentage and number of respondents along with their classification

Analysis approach

The Lines Between was commissioned to provide a robust, independent analysis of the responses to the public consultation. Public consultation of this kind means anyone can express their views; individuals and organisations interested in the topic are more likely to respond than those without a direct or known interest. This self-selection means the views of Respondents do not necessarily represent the views of the entire population. The main purpose of consultation analysis is not to identify how many people held particular views but to understand the full range of views expressed.

The analyst team coded responses using a coding framework which was developed following an initial review of a sample of responses. Through an iterative coding process, new codes were created if additional themes emerged, depending on the breadth of the discussion. Notes from the CCG discussions were reviewed to identify any differences in opinion compared to the main sample and to identify any new themes. The themes evident in the CCG discussions typically aligned with those evident in the main sample, but any additional relevant points have been noted in this report.

Where appropriate, quotes from respondents are included to illustrate key points and provide useful examples, insights and contextual information. Reflecting the large number of people who took part in the consultation, there is no scope to detail every response in this report; some respondents shared lengthy submissions which reflect their specific area of interest or expertise. All responses to the consultation, where permission for publication was granted by the respondent, can be found on the Scottish Government's website.

Report structure

This introduction explains the analytical approach and details the profile of respondents. The remainder of the report presents a question-by-question analysis:

• Chapter 2 summarises the cross-cutting themes evident in multiple questions. These themes are also noted in the subsequent question-by-question analysis to explore the nuance in responses to each question.
• Chapter 3 presents an analysis of Q1 to Q3. These asked if and what information about learners should be gathered across all four capacities of the Curriculum for Excellence (CfE), whether information on achievements outside school or college should be included, and if learners' skills and competencies should be recorded in their Senior Phase.
• How best to evidence achievements, awards and qualifications are considered in Chapter 4, which presents the analysis of Q4 and respondents' views on the balance between exams and other forms of assessment.
• The advantages and disadvantages of recognising achievement at the end of Broad General Education (BGE) and the Senior Phase are presented in Chapter 5, which considers Q5 and Q6.
• Chapter 6 presents an analysis of Q7, which considers how Scotland's qualifications and assessment system could make the best use of digital technologies.
• Ensuring a future qualification system upholds the rights of all learners to demonstrate their achievements is the focus of Q8 in Chapter 7.
• Chapter 8 presents the analysis of Q9, where respondents were able to express their views on any other aspects of reform.
• A summary of responses from different sectors is included in Chapter 9.
• Chapter 10 presents the conclusions from the analysis.

Weight of opinion

Throughout this report, the themes evident in each question are typically listed from most to least commonly mentioned. All themes, including those mentioned by a small number of respondents, have been included.

Qualitative analysis of open-ended questions does not permit the quantification of results; an insightful view expressed by a very small number of respondents is not given less weight than more general comments shared by a majority. However, to assist the reader in interpreting the findings, we have developed a framework to convey the most to least commonly identified themes across responses to each question:

• Many respondents, more than one in five, a prevalent theme.
• Several respondents, between one in 10 and one in five, a recurring theme.
• Some respondents, fewer than one in 10, another theme.
• A few / a small number; fewer than one in 20, a less commonly mentioned theme.