Independent Review of Qualifications and Assessment: consultation analysis

Introduction

Context for the Review

The Independent Review of Qualifications and Assessment was announced in October 2021. The aim of the Review is to ensure that all Senior Phase learners in Scotland (predominantly those aged 15-18) have an enhanced and equal opportunity to demonstrate the width, depth, and relevance of their learning.

The Review was initiated in response to a number of factors including the experience gained and lessons learned as a consequence of the Covid-19 pandemic (such as the need for a more resilient assessment system); subsequent reports on Scottish education; the OECD report on Scotland's Curriculum for Excellence - Into the Future (2021)^[1]; and international comparisons which suggest that, without innovation and change, the Scottish education system risks lagging behind its economic competitors.

Structure of the Review

To provide a structure for engagement at key points in the process, the Review adopted a three phased approach:

• Phase One ran from August to September 2022, and focused on the development of an underpinning set of Vision and Principles. Engagement in the Review was supported by the Independent Review Group (IRG) who facilitated discussions with a series of Collaborative Community Groups (CCGs) and by materials sent to all secondary schools and colleges in Scotland to facilitate discussions and encourage participation.
• Phase Two ran from October 2022 to January 2023, and focused on detailed questions designed to lead to a preferred Qualifications and Assessment model. Again, the IRG and the CCGs participated, and supporting materials were sent to all schools and colleges. The options for change were also open to a twelve-week public consultation.
• The findings from Phase One and Phase Two, together with the body of research that underpinned the initiation of the Review, informed the development of a proposed future high-level model for qualifications and assessment.
• Phase Three commenced in March 2023 and sought views a high-level overarching model (see page 13 for a description of the model). Engagement was carried out through facilitated discussions with the CCGs and an online consultation survey sent to all schools and colleges in Scotland. This phase did not include a public consultation. The results of Phase Three were then used to further develop the model.

Consultation approach

Consultation on Phase Three was undertaken during the period 3March until 31 April. The information and analysis contained within this report covers responses received until 14 April only. Responses received from 15 April to 31 April have been analysed separately.

CCG discussions: By 14 April, a total meeting reports from 19 CCGs and allied discussion groups, representing over 400 people were produced. This process involved a series of detailed in-depth discussions facilitated by an IRG member. CCG groups include groups of learners, parents/carers, teachers, headteachers, colleges, Directors of Education, academics, employers. Visits to a number of schools, to meet with learners and staff were also undertaken. A full list of CCG membership can be found here.

School and College survey: Materials were also sent to all secondary schools and colleges in Scotland, and settings were asked to complete a consultation survey. A total of 311 valid^[2] responses were received to the survey by 14 April: 259 were from school communities, 15 from colleges, 32 from others, and 5 unassigned. Most of the School and College survey responses were from teachers.

In many cases, schools and colleges held group discussions with multiple teachers/lecturers, parents and learners, with the responses submitted on behalf of the group. It is, therefore, not possible to determine exactly how many individuals were engaged in producing the School and College survey responses.

A further 34 non-survey responses were received by email by 14^th April^[3], mainly from schools and colleges. Further details on the profile of respondents are provided in Appendix A.

The Review questionnaire sought views on the model, each of the three elements (Subject Studies, Learning in Context and the Personal Pathway), the Diploma, the changes in practice needed to deliver the new system and naming of the qualifications, as well as providing opportunities to comment on the Review process. A copy of the questionnaire is provided in Appendix B.

It is worth noting that many of the Review respondents, principally those responding to the School and College survey and occasionally those taking part in CCG discussions, said that they felt they had insufficient information about the proposed model to comment (fully) and in some cases their responses were questions about the model/proposals.

Linked to this, the CCG discussions tended to be more positive and constructive in their approach, while the School and College survey responses were more likely to be negative and critical. There are many possible explanations for these differences.

• The profile of the CCGs differed to the School and College survey: most of the responses received via the School and College survey were received from school communities, rather than colleges, and the majority of these were from secondary school teachers. The CCG responses comprised a far broader range of stakeholders and whilst this included teacher groups it also included learners, parents and carers, universities, Directors of Education, employers, researchers and policy professionals. Appendix B includes a profile of the survey respondents.
• The method of engagement was different: a deliberative consultative process was adopted with the CCGs, which meant they had greater access to information about the Review via their IRG link member, opportunities to ask questions and time to reflect during and between each of the three phases. The School and College survey was not always accompanied by a group discussion and while it was suggested by the Review that educational settings discuss the proposals before responding, it was left up to each setting to determine if/how to do this. The act of discussing as a group prior to completion may have resulted in more nuanced responses, with those completing the online survey only, potentially feeling that they could be more direct.
• Nearly all who participated in the CCG discussions participated in all three phases of the Review. There were many more responses to the Phase Three School and College survey than the Phase One survey, which indicates that some respondents to the Phase Three School and College survey did not participate in all three Phases.

Analysis

Progressive Partnership was commissioned to provide a robust, independent analysis of the responses to Phase Three. The information collected reflects the views of respondents but cannot be extrapolated to the wider population. The analysis and interpretation of Phase Three responses is therefore descriptive and qualitative.

Coding of open questions

Coding of the open question was done by Progressive's in-house team of experienced coding specialists. Prior to coding beginning, an analytical framework was developed for each open question in the consultation. The framework sets out the range of key issues and themes for consideration. It is derived from the key themes within the Review and from a review of a cross-section of responses for each question.

All the responses were examined thoroughly and coded against the analytical framework. Small teams of experienced coders worked on each question, to minimise bias in the analysis. It is noted that, given the nature of the Review, several of the responses were complex/lengthy, and not amenable to coding. The coders therefore highlighted these responses, and they were reviewed separately by the executive team. All the CCG responses were analysed by the executive team.

Description

The report includes the full range of views presented by respondents, not just those mentioned by the majority. The report adopts the convention below in relation to reporting prevalence of the response. However, it is stressed this relates to responses received; and many of the responses were submitted on behalf of a group of people (sometimes a group of teachers or one or two classes, but in a small number of cases the submissions reflect large consultations covering hundreds of individuals).

• Many respondents: a prevalent theme.
• Several respondents: a recurring theme.
• Some respondents: another view.
• A few: a less commonly mentioned view.

Further, the report does not assign varying levels of weight to the responses received. Where relevant and practical, it identifies sub-group differences across the responses (in particular, it presents the CCG and the School and College survey responses separately), and it is mindful that the different sub-groups have different experiences, technical expertise and levels of engagement in the Review.