Independent Review of Qualifications and Assessment in Scotland: interim report

The Review Process

Independent Review Group - IRG

Central to the process of collaborative review has been the Independent Review Group (IRG). The IRG operates as a matrix model.

On one side of the matrix, three distinct groups of participants are involved in the Review:

• those for whom qualifications matter most including individual learners and, as appropriate, parents or carers;
• those involved in the design, development and offering of qualifications -educational professionals including teachers and lecturers, school and college leaders and local and national policy makers;
• and users of qualifications such as colleges, employers and universities.

On the other side of the matrix are research-based groups including researchers on aspects key to the Independent Review. The purpose of these groups is to ensure that the work of IRG is informed by leading edge thinking. These areas include Equity, Curriculum, Assessment and Qualifications, Change Processes and Policy Alignment

All of these groups are crucial if a qualifications system is to be credible and practical.

Each member of the IRG links with a broader group of members from their community. The number of groups has increased since the start of the Review as interest has grown in the topics and sub-topics under discussion. These groups include a wide range of participants and we will continue to ensure an inclusive and participatory approach as the Review progresses. The aim is to recognise and embrace the diversity of Scotland's learners and communities. These groups seek to involve people whose voices are seldom heard in policy discussions. IRG members do not represent their own organisations, but instead act as facilitators to communicate with their wider communities exploring how the qualifications system in Scotland might best support all learners.

The IRG has been considering issues such as:

• the purposes and uses of a qualification and awards system, including recognition of learning, accreditation, selection and accountability, and the relationship between qualifications and lifelong learning;
• approaches to assessment in professional and technical subjects and lessons that could be learned from these approaches;
• fairness, equity and the impact of different approaches to assessment for qualifications;
• from ideas to practice - the process of change and learning from our past to understand why there are significant gaps between the original intentions for qualifications in CfE and current practices, where rote learning, formulaic approaches to examinations, constant examination rehearsal and complaints of having no time to learn in the 'two-term dash' are commonly reported experiences;
• wider National and International approaches to the future of Qualifications Assessments.

This Review has predominately focused on learners aged 15-18 in all educational settings, this will usually be school or college but also includes learners in other educational settings and learners who are home educated. There are also implications for some adult learners who study courses which fall within the remit. The focus of this work has been on the framework of school and college qualifications accessible in the Senior Phase. The Review will not consider or make recommendations on the content of individual courses.

Collaborative Community Groups (CCGs)

A key responsibility of IRG members has been to facilitate wider community engagement. Each IRG member was asked to bring together individuals from their extended community to form a CCG. Through their CCGs, IRG members have gathered perspectives and views from individuals and groups that reflect the diverse make-up of their communities, including seldom heard voices. Ideas being considered in IRG are discussed with CCG members and evidence and insights from the CCGs has been brought back to the monthly IRG meetings to form part of the wider evidence gathering process.

Three Phases

To provide a structure for engagement at key points in the Review process, we adopted a 3 phased approach:

• Phase One – August to September 2022 – this focused on the development of an underpinning set of Vision and Principles. The IRG and all CCGs were involved and supporting materials were sent to all secondary schools and colleges in Scotland.

• Phase Two – October 2022 to January 2023 – this provided more detailed questions designed to lead to a preferred Qualifications and Assessment model. The IRG and all CCGs have been involved and supporting materials were sent to all schools and colleges. The Review also engaged in a public consultation which closed on 13 January 2023.

• Phase Three – this phase will commence in March 2023. The aim is to consider a future model for qualifications and assessment the design of which is drawn from the reports previously cited, responses to consultation in Phases One and Two of this Review. The intention in Phase Three is to seek views on a proposed approach and, crucially, the practical steps that would be needed for this model to be successful in practice. During Phase Three the Review will also take into account relevant findings from the National Discussion on education. Phase Three will operate in a similar way to Phase One. Materials will be circulated to all schools and colleges and there will be IRG and CCG discussions. In addition, a number of school and college visits will be undertaken to inform thinking on the model and its implications in practice.

Phase One – Vision and Principles

The draft Vision and Principles were developed in partnership with colleagues from the Scottish Youth Parliament and the Children's Parliament. Phase One of the Review was live between August and September 2022. Responses were received from Collaborative Community Groups and allied discussion groups. A total of 221 responses were received from schools and colleges. A number of schools and colleges held group discussions with multiple teachers / lecturers and learners, and these responses were submitted on behalf of the group. The feedback from Phase One was independently analysed. The report on the analysis can be found at Analysis of Phase One consultation.

The evidence from the Phase One consultation suggested that there was little challenge to the fundamental ideas in either the draft Vision statement or the draft Principles. However, there was a clear request to simplify the language and to sharpen statements being made. To address these points, we have clarified the language of both the Vision and Principles and have reduced the number of principles. The latest iteration of the draft Vision and Principles is provided below. The Vision and Principles will remain as 'draft' until we have concluded the final phase of the Review.

The redrafted Vision and Principles were then used to inform the second phase of the Review, where options, consistent with the Vision and Principles, were explored.

Current Draft of Vision and Principles

Vision

An inclusive and highly regarded qualifications and assessment system that inspires learning, values the diverse achievements of every learner in Scotland and supports all learners into the next phase of their lives, socially, culturally and economically.

Principles

Scotland's qualifications and assessment system should:

1. Recognise, value and promote the rights and achievements of every learner.

2. Reflect the Scottish curriculum whilst being responsive to the changing needs of individual learners and of society, creating a positive and sustainable future for learners, their communities and the wider world.

3. Develop and maintain an appropriate range of approaches to assessment including through digital mechanisms.

4. Be clear, coherent, credible and easily understood as part of a lifelong learning journey.

5. Be adaptable and subject to regular review using the vision and principles as a touchstone against which change can be tested.

6. Ensure that all groups* with a stake are involved in future decisions related to design, implementation and practice.

*This should include learners, parents / carers, teachers / lecturers, national bodies, colleges, universities, employers and the voluntary sector.

Phase Two - Options for Change

Phase Two of the Review began in October 2022 and closed in January 2023. The consultation for this phase was designed to explore options that would help define the parameters of a possible new approach to qualifications and assessment in Scotland.

IRG discussed the options in detail and held meetings with their CCGs. In addition to feedback from IRG, almost all CCGs submitted detailed views and there was just over 700 responses submitted from the public consultation on this phase of the work. As with Phase One, the responses have been independently analysed. All consultation responses will be published where permission to do so has been granted alongside a final copy of the consultation analysis. Discussions in the IRG, CCGs, other Convener led meetings the Review has undertaken, and school / college visits suggest that there is almost universal consensus on the need for change in qualifications and assessment.

In this Interim Report, the section that follows reports only on the broad-brush messages emerging from the analysis of the consultation. The final report will include more detailed evidence from individual CCGs, from learners and parents/carers; those who design, develop and offer qualifications and those who use qualifications (colleges, employers, universities and the voluntary sector).

Phase Two Consultation – Analysis

This section includes extracts from the independent analysis comprehensive summary written by The Lines Between. In considering responses to the Phase Two consultation, a framework has been developed to convey the most to least commonly identified themes across responses to each of the questions posed:

• Many participants; more than one in five, a prevalent theme
• Several participants; between one in 10 and one in five, a recurring theme
• Some participants; fewer than one in 10, another theme
• A few / a small number; fewer than one in 20, a less commonly mentioned theme

This analysis is primarily drawn from 708 consultation responses. Within this sample, 526 responses were from individuals, and 182 from a range of organisations, including many schools and colleges. Individuals were not asked if they were learners, teachers or parents, but for analysis purposes a separate category of Pupil / Teacher / School response has been created. The 262 responses in this category include all organisation responses submitted by schools, as well as all individual responses submitted from email addresses which include Glow or sch suffix.

Respondent classification	N=	%
Individual – unclassified	372	53%
Pupil/Teacher/School response	262	37%
Organisation:	74	10%
Colleges and Universities	10	1%
Trade Unions and Professional Bodies	13	2%
National Agency or Public Body	15	2%
Young person's organisation	9	1%
Other non-education	8	1%
Local Government and Local Authorities	7	1%
Other education	7	1%
Parent body/group	3	<1%
Unknown	2	<1%

Q1a: Should information be gathered across all four capacities?

Almost all (96%) respondents answered Q1a. Among those who answered there was clear support for gathering this information: 61% agreed, 23% disagreed and 17% were unsure.

Q1b: Please consider each of the capacities in turn. What kinds of information should be gathered on learners' progress and achievements in each capacity?

Successful Learners - Several respondents noted that progress in this capacity can be evidenced through achievement and attainment in subject qualifications.

Confident Individuals - Respondents gave various suggestions about how to gather information related to this capacity including family and caring responsibilities.

Effective Contributors - As with Confident Individuals, several respondents suggested that Effective Contributors could be evidenced through coursework, especially group tasks and presentations. Several felt information about extra-curricular activities could be gathered.

Responsible Citizens - Responsible Citizens was viewed as the most challenging capacity to gather information about. Some respondents noted this is another subjective principle, with some uncertainty about what measures should be used to identify a Responsible Citizen.

Other themes - Some respondents commented on the importance of considering a learner's strengths, weaknesses, and circumstances as an individual, including any additional support needs, learning difficulties and socio-economic disadvantages. These respondents emphasised the importance of recognising all learners' achievements and acknowledging that progress will look different for some learners than others. Another theme raised by some respondents was a concern about the workload implications if teachers were expected to devote additional time to collating information related to the four capacities out-with what is already collected for subject qualifications.

Q2: What, if any, information on learners' achievements obtained outside school and college should be gathered? Please explain your response.

Many respondents called for information about learners' participation in extra-curricular activities to be gathered. This included participation in a broad range of clubs and hobbies, including sports, music, drama, art, baking, youth organisations such as the Guides, Scouts and Cadets, religious organisations and learning opportunities outside school, including lessons in modern languages. Among these respondents, opinion was evenly split on whether only accredited activities should be recorded. Several respondents pointed out that gathering information about achievements outside of school or college could discriminate against those who may not be able to access extra-curricular activities to the same extent as others for various reasons, including cost, time limitations, parental support, local availability, health issues and caring responsibilities.

Q3a: Should information be gathered on learners' skills and competencies as part of their Senior Phase?

Q3a was answered by almost all respondents (96%). Among those who answered there was strong agreement that information about learners' skills and competencies should be gathered as part of their Senior Phase: 75% agreed, 11% disagreed and 14% were unsure. The initial analysis shows slightly higher agreement among organisations, at 80% compared to 74% among individuals.

Q3b: If you have views on how this might best be done, please provide them here.

The most prevalent suggestion, mentioned by many respondents, was for skills and competencies to be embedded and evidenced through subject-based qualifications. There was a clear preference among many for coursework and continuous assessment, rather than formal end of year exams, to capture evidence of skills and competencies. Some respondents identified challenges in gathering objective, fair and consistent information about learners' skills and competencies across different schools, particularly in terms of the types of information that should be recorded.

Q4: Please share your thoughts on what a 'better balanced' assessment system would look like. As well as considering the balance between external examination and internal assessment you may also wish to comment on the frequency of examinations.

There was no clear consensus on what a better-balanced assessment system would look like. Many participants favoured a mixture of continuous assessments or smaller, more regular tests and exams. This included mixed views on the weighting between regular assessments and final exams, including the need for different weightings to be used depending on course level, e.g. Higher vs Advanced Higher. There was acknowledgement that some courses currently take continuous assessment into consideration alongside exams, such as Higher and Advanced Higher English, and there was mixed preference without clear consensus as to whether those courses should be kept as is, or changed.

Eliminating end-of-year exams entirely was also requested by many, who supported more regular or continuous assessment as an alternative. Others supported more, smaller tests, occurring at regular intervals, such as end of term, end of topic, monthly or weekly. Some respondents argued that exams are a poor measure of learners' abilities. They pointed out that there was potential on the day to underperform or that learners may be affected by anxiety that hinders their performance. Conversely, the current exam structure was supported by some respondents who felt exams are fair, that qualifications are valued, and that final exams allow students to gain a recognised qualification.

Several respondents suggested or recommended using a variety of alternative types of assessment. These included: open book exams, oral exams, practical assessments, group projects, longer-term projects, multiple choice assessments, and support for creating portfolios to assess work produced over extended time periods. Digital or online exams were highlighted as a way to manage workload and ensure consistency in marking. The idea that assessment type should depend on the course, subject or the learner's educational pathway was suggested by several respondents.

The frequency of exams was mentioned by some respondents. This included a preference for less frequent exams over S4-S6, i.e. two rather than three exams over the three years. A few highlighted the intensity of the exam diet, suggesting there was insufficient time between exams, whether internal or external, to allow students to learn, study or have moments of less stress. The lack of a National 4 exam was highlighted by some respondents as especially problematic. They expressed a preference for an exam, suggesting that the lack of an exam has devalued the course in the eyes of employers and left students unprepared for Nantional 5 if they were to continue. This was mentioned most often in relation to Nantional 4 Maths.

Several respondents highlighted concerns about the consistency and equity of using internal assessments alone. Some felt it devalued a qualification as it would mean marking lacked national consistency and was open to marking bias.

With regards to internal assessments, some were concerned about the extra pressure it would put on teachers in terms of time and resources required to mark assessments.

Employers involved in the CCG discussion indicated they did not have a strong preference for internal or external exams but needed assurance that standards would be maintained.

Q5: Please share your thoughts (advantages/disadvantages) on the idea of introducing an achievement, award or qualification at the end of the BGE.

Several respondents suggested this proposal could incentivise more focus among BGE learners. Learners might work harder at this stage if they know it will lead to an achievement, award or qualification. Similarly, some felt it could enhance teachers' focus on BGE if learners are working towards an achievement, award or qualification. Another recurring theme was the opportunity to give every learner evidence of their skills, which could be especially beneficial for those likely to gain fewer academic qualifications.

The most common argument against the proposal, put forward by several respondents, was the pressure it could put on teachers to achieve attainment targets in BGE and to have results compared with other schools via league tables. Similarly, some referred to teachers' lack of time and resources to administer an achievement, award or qualification amidst an already busy curriculum.

Q6: Please share your thoughts (advantages/disadvantages) on the idea of introducing a type of leaving certificate in the Senior Phase.

The general consensus was in favour of this proposal. Several respondents would welcome a leaving certificate that recognises a broader range of learners' achievements, skills and competencies than are evidenced through the current qualifications and assessment system. Respondents noted a leaving certificate would provide a more holistic record of learners' achievements. Another recurring theme, mentioned by several respondents, was that a leaving certificate would provide useful information for employers and universities about learners' skills, competencies and achievements. Similarly, several respondents felt it would help learners to identify achievements to include on their CVs and UCAS applications.

Several respondents argued that this proposal could put added pressure on time among school staff if they were expected to compile evidence for the certificate. Other recurring themes mentioned by several respondents included doubts over whether a leaving certificate would be of value to employers and further education providers unless it provides information that is helpful to them - for example in making recruitment and admissions decisions - and is not available elsewhere.

Q7: How should Scotland's qualifications and assessment system make best use of digital technologies?

Several respondents proposed creating a digital platform to record learners' skills and achievements, possibly using existing resources like Glow and My World of Work. Enabling learners to submit coursework online was another idea put forward by several respondents. Some respondents highlighted a need for national guidance to promote consistency in the use of digital technologies across Scotland.

Advantages of using digital technologies were also noted; several highlighted the benefits for learners with additional support needs who may find handwritten exams challenging. While the consensus was largely in favour of using digital technologies, respondents identified several barriers that must be addressed. Most commonly, many respondents identified unequal access to digital devices as a barrier to using digital technologies in qualifications and assessment, particularly among learners from less affluent backgrounds.

Q8: How can we make sure that proposals for a future qualification system will uphold the rights of all learners to demonstrate their achievements?

Most commonly, many respondents highlighted flexible teaching and assessment approaches as a means to better satisfy different learning needs and styles among all learners. Suggestions included enabling learners to select different forms of assessment (for example, verbal or written, digital or paper-based) depending on their preferences. There was some disagreement about the importance of exams. Some respondents called for reduced focus on end of year exams and more on coursework or continuous assessment. However, a few felt externally assessed exams are more likely to uphold learners' rights by removing any potential for conscious or unconscious bias in marking.

Another suggestion made by several respondents was the need for more standardisation and consistency across schools and local authorities, regardless of their size, location and socio-economic profile. This mainly focused on consistent availability of subjects but also included standardisation in terms of resources and options for assessment.

Q9: Is there anything else in relation to the reform of qualifications and assessment which is not covered in this consultation which you would like to raise?

Some respondents discussed the approach that should be taken to establishing a vision for Scotland's qualifications and assessment system. However, opinion was split between those who felt there should be radical change now, and those who advocated a more gradual, incremental approach. A few called for more collaboration among schools, colleges and universities. Ideas included co-designing subjects' curricula to ensure they meet the needs of further education, more alignment between the evidence collected in both schools and colleges, and creating more flexible links between schools and colleges to assist learners interested in a vocational pathway such as foundation apprenticeships.

In Conclusion: Themes emerging from consultation in Phases One and Two. The consultation on the Vision and Principles revealed an emerging consensus. The vision of a future qualification and assessment system in Scotland is: An inclusive and highly regarded qualifications and assessment system that inspires learning, values the diverse achievements of every learner in Scotland and supports all learners into the next phase of their lives, socially, culturally and economically.

The Principles provide a framework for the design of any future system of qualifications and assessment. It should

• Recognise, value and promote the rights and achievements of every learner.
• Reflect the Scottish curriculum whilst being responsive to the changing needs of individual learners and of society, creating a positive and sustainable future for learners, their communities and the wider world.
• Develop and maintain an appropriate range of approaches to assessment including through digital mechanisms.
• Be clear, coherent, credible and easily understood as part of a lifelong learning journey.
• Be adaptable and subject to regular review using the vision and principles as a touchstone against which change can be tested
• Ensure that all groups* with a stake are involved in future decisions related to design, implementation and practice.

The first four bullet points will be used as the basis for consultation in Phase Three of this Review. These are the criteria which should inform the design and development of the future model.

The consultation in Phase Two offered insights into the parameters for the model.

The spread of views was greater in Phase Two than in the responses to the Vision and Principles. In this section, views that were held more commonly are reported.

• The future qualification system should include evidence on achievements in individual subjects but should go beyond those to include other aspects of Curriculum for Excellence.
• The future system should consider a wider range of ways of gathering evidence related to individual subjects/programmes including for example, project work, teacher/lecturer assessment, examinations, open book tasks, end of module tests, oral examinations, digital assessments, photographic or video evidence. There should be fewer examinations but an external element to an assessment was often an important feature of achievement status.
• The system has to be sufficiently flexible to be able to respond to the fact that if it is to be inclusive then what constitutes achievements for individual learners may be very different. In future, this may also mean allowing learners to choose different ways of demonstrating their achievements.
• Information on learners' skills should be gathered as part of qualifications
• The relationship between academic and vocational programmes should be better integrated and the language of 'academic' and 'vocational' reconsidered
• Learners should have the opportunity to gather evidence of achievements beyond the subject/programme curriculum. These should be broadly defined to allow them to reflect different learners' interests in and out of educational settings but all learners should have the right to opportunities that would allow them to demonstrate achievements.
• Manageable ways to share and to promote consistency of standards should be part of any qualification system to ensure that the system is fair and has credibility
• There should be a type of leaving certificate to provide a more holistic record of a learners' achievements. This would offer learners an evidence base, e.g., to construct a cv, and would provide better evidence to colleges, employers and universities and could become the basis of a lifelong learning profile
• Digital approaches to assessment are the future. A profile/leaving certificate should be digital although there were concerns about the capacity of current systems. Examinations should also, in future, be digital.
• Any proposals have to be possible and thus cannot simply be added to teachers' existing workload. Careful consideration will have to be given to the practical implications of proposals made.

It is interesting to note that many of the issues raised in response to this second phase of consultation mirror ideas reflected in the original intentions of CfE. The concern to reflect a broader range of learners' achievements, the desire to reduce the number of examinations taken by individual learners, the concern not to introduce high stakes assessment at too early a point in the learning process were all aspirations for CfE, e.g., as outlined in Building the Curriculum 3, (2008) which notes "There should be no early presentation for examinations unless in exceptional and individual circumstances".

A further area of commonality can be found in the original CfE desire to offer a wider range of approaches to gathering evidence, e.g., Building the Curriculum 5, (2011) notes - "Learners should be engaged in all aspects of assessment processes and be afforded an element of choice and personalization in showing that they have achieved the intended outcomes.".

The evidence from Phases One and Two have now been used to inform the design of a model that will be the basis of consultation in Phase Three with communities across Scotland. However, given the previous experience of ideas in the original version of CfE not becoming part of practice, Phase Three consultation will investigate what practical actions should be recommended if ideas are to succeed in practice.

Phase Three – A Model for Change

Phase Three will take place during March 2023. This phase of the work is designed to test, through consultation, a possible model for qualifications and assessment, that builds on the evidence emerging from Phases One and Two of the Review process. Phase Three will also seek to identify any possible unintended consequences and to agree on potential mitigating actions.

Views will be sought on the draft model through the CCGs, allied discussion groups, via schools and colleges and a range of other meetings. This phase will also focus on the practical implications for change including phasing and the support needed for learners, parents, schools and colleges, employers and universities if change is to be successful.

The main areas currently under investigation in advance of Phase Three and as part of a potential new approach to qualifications and assessment include (please note that the terms used to describe each part of the proposed new qualification are working titles rather than agreed terms):

• Subjects and Learning Programmes: Individual subjects, courses and learning pathways. These already exist and are perceived to be the current focus of the current qualification system. However, in the new model, courses/ programmes would be progressive over two years. Additional ways of gathering evidence that are less susceptible to formulaic responses will be explored. There exists a strong perception of overemphasis on gathering evidence of achievements in subjects. While the study of individual subjects is a crucial part of education, in the new model other areas of CfE would be recognised as being equally important.
• Learning in Context / Interdisciplinary studies: The new model recognises the increasing importance in society of learning across different areas of the curriculum and proposes the introduction of an interdisciplinary element to the qualification. This would be a project-based approach where evidence is gathered based on achievements across knowledge, skills and competences in action. For some learners, this might focus on a global challenge, e.g., climate change, migration or social justice. The kinds of area learners have indicated are of real interest to them and that they see as being crucial for the future of society. For other learners, this might involve communication or independent travel. This would also recognise the growing importance of skills and competences for future learners. Currently, there are many different skills frameworks in play in Scotland. A single skills framework would be more helpful. Interdisciplinary studies would form a key element of an overarching qualification. However, although many schools and colleges are already involved in projects and programmes like this, the Review recognises that the timeline for the introduction of this part of the qualification profile will require careful consideration and particular forms of support.
• Personal pathway: Here learners have opportunities to select aspects of their experiences that reflect their interests, e.g., in drama, music, sport, film making or photography; the contributions they make to society, e.g, supporting young children to read or in sport, or supporting members of their own community or taking a leadership role in a school or college activity; and the experiences they might have that would help inform their career aspirations, e.g., in employment or as entrepreneurs and enhance their understanding of the world of work.

This component would be subject to discussion with every learner. It would, however, have common characteristics. In their personal pathway, each learner would include evidence of social, cultural and economic activity. Again, this could form a key element of an overarching qualification.

In Phase Three The IRG and CCGs will continue to explore a range of key issues including:

• The creation of an overarching qualification or graduation certificate that would be designed to allow for the building of credits over time and would bring together different forms of learning.
• The balance between internal and external assessment within a new system, with examinations continuing to form part of the new approach, where appropriate.
• How better to integrate the offer of 'academic' and 'vocational' qualifications and the language that should be used to describe courses and programmes.
• The potential of a digital learning profile that would allow evidence to be effectively gathered
• What changes might be proposed to the wider education system to support future reform of Qualifications and Assessment

In addition, the IRG and CCGs will consider aspects of the consultation in Phase Two where there was no clear view, e.g., whether or not there should be an award at the end of Broad General Education.