It's Our Future - Independent Review of Qualifications and Assessment: report

2. Changing the Culture of Education in Scotland: Methodology of the Review

2.1 The Integrity Model of Change

The Putting Learners at the Centre Report (Muir 2022) advocated for cultural change in Scottish education. First, in accordance with the UNCRC (1989), learners should be at the heart of decision making. Second, that Scotland’s educational aspirations would be realised only if all communities were willing to work together. Professor Muir recognised that deep and meaningful change would be possible only when it was recognised that the success of any reform depends on the actions of all those who are connected to it. In an effort to follow the recommendations of Professor Muir, this Review has developed an engagement strategy designed to ensure meaningful opportunities for all interests to contribute to the work.

The approach to engagement underpinning this Review is based on the Integrity Model of Change (Hayward & Spencer, 2010) that pays attention to three overlapping and interacting areas:

Educational Integrity - the need to ensure that what is done will lead to better educational opportunities and better life chances for every learner, for example, is informed by policy, practice and research.

Personal and Professional Integrity - the importance of ensuring that everyone who has a role to play in making the innovation successful is involved in its design and development, recognising and valuing the crucial role that each will play.

Systemic Integrity - the need to ensure that the various parts of the system are aligned to support the reform. Learners, parents and carers, The Government, the national agencies, professional associations, local communities, teachers, education providers, colleges, employers and universities, all have to actively support the change process if the reform is to be successful.

Figure 1: The Integrity Model of Change

The Integrity Model of Change

• Personal and Professional Integrity
• Educational Integrity
• Systemic Integrity

Hayward & Spencer (2020), UNESCO (2020), OECD (2021), NEU (2021)

Any innovation that seeks to lead to change that is deep, meaningful and sustainable has to pay attention to a number of features. Often change focuses on what has to change and, of course, that is crucial. However, it is equally important to consider why the change is important, how it might be achieved and when it might happen, for example, what a pragmatic and manageable timeline would look like. In the context of this Review, the case has to be made that changing qualifications and assessment is important for both individual learners and for Scottish society as a whole.

Finally, for change to be sustainable, the Review has identified what needs to be different in the wider educational system to support the innovation in practice including in respect of data collection, inspection and Initial Teacher Education (ITE).

The design of the Independent Review sought to address each of the three dimensions of the change model.

2.1.1 Educational Integrity

First, to build Educational Integrity, the Independent Review began its work with a focus on learners. The first task of the Review was to develop a statement of a possible Vision and Principles for the future of Qualifications and Assessment in Scotland. Colleagues from the Scottish Youth Parliament (SYP) and from the Children’s Parliament worked with us to develop a draft statement of Vision and Principles as the starting point for the Review.

Three phases of engagement followed. The first phase of engagement sought views on the development of a shared Vision of what qualifications and assessment in Scotland should seek to achieve. Linked to the Vision, the Independent Review also sought views on proposed Principles. The draft Vision and Principles were discussed by the IRG and the CCGs (described in the next section). A discussion pack, including video, a background paper and survey were sent via Local Authorities to every school in Scotland and directly to every college. Responses were independently analysed, and the findings used to revise the draft Vision and Principles.

The Vision and Principles were used to inform the design of the reformed qualifications and assessment system. In addition, it is proposed that they become the touchstone for the reform. To avoid the recurrence of previous problems with innovation in Scotland, where practice over time became different from the original reform intentions, the Vision and Principles should be used over time to monitor the relationship between ideas and enactment.

The second phase of engagement, using the Vision and Principles as the starting point, sought feedback on options that would help define the parameters of a possible new approach to qualifications and assessment in Scotland. This phase of engagement sought views on a range of issues including:

• whether or not evidence of learners’ achievements should be gathered only for successful learners or more broadly to include achievements across the other three capacities of CfE, confident individuals, effective contributors and responsible citizens;
• the balance between assessment based in an education setting and external assessment;
• the use of technology to enable new approaches to assessment and qualifications and as a means of recording achievements and;
• whether or not there should be an award at the end of the Broad General Education (BGE).

Again, topics were discussed in detail in the IRG and in CCGs. A second discussion pack with a supporting video, background paper and PowerPoint presentation was sent directly to all schools and colleges. In response to problems arising from Phase One, where the distribution process to schools had worked better in some parts of the country than others, in Phase Two discussion packs were simultaneously sent to local authorities and directly to every school and college, to ensure that all practitioners had the opportunity to contribute to thinking about the future direction. As part of Phase Two the Review also issued a public consultation.

The responses to the second phase were again independently analysed and the findings used to develop the model that became the focus for discussion in Phase Three.

The third phase of the Review sought views on a model for future qualifications and assessment in Scotland. The Phase Three model was designed to be consistent with the Vision and Principles agreed in Phase One and with views expressed on the parameters established following Phase Two of consultation. Respondents in the third phase were asked to consider the extent to which the model was consistent with the agreed Vision and Principles and, crucially, to identify the practical steps that would be needed for this, or an adapted, model to be successful in practice.

During the three phases of the Review, members of the Review team visited schools and colleges across Scotland to engage directly with learners, teachers, lecturers, school leaders and college principals. In total 21 visits to schools and colleges were undertaken (see Appendix Six).

The findings from all three phases of the Review inform the recommendations in this report.

2.1.2 Personal and Professional Integrity: roles and responsibilities

Within the Integrity Model of Change, for the reform to have personal and professional integrity everyone who has a role to play in making the innovation successful should be involved in its design and development. This recognises and values the crucial role that each will play in making the innovation successful. The development of the IRG and CCGs, the engagement with schools and colleges and the public consultation were all intended to be vehicles to promote personal and professional integrity within the Independent Review.

IRG has been the core of this collaborative Independent Review. The IRG operates as a matrix model.

On one side of the matrix, three distinct groups of participants were involved in the Independent Review:

• those for whom qualifications matter most including individual learners and, as appropriate, parents or carers;
• those involved in the design, development and offering of qualifications-educational professionals, including teachers and lecturers, school and college leaders and local and national policy-makers and;
• those who use qualifications such as colleges, employers and universities.

On the other side of the matrix were research-based CCGs each focused on an aspect key to the Independent Review. The purpose of these groups was to ensure that the work of IRG is informed by leading-edge thinking. They included national and international researchers and leading educational thinkers. One group focused on Equity, the need to ensure that any proposals would pay attention to the implications for every learner. A second concentrated on Curriculum, Assessment and Qualifications and how we might promote strong alignment between them. The third group advised on the evidence base for Qualifications and Assessment to make sure that the thinking of the Review was well informed. The fourth group considered Change Processes, advising on evidence to inform recommendations to support the process of change. The fifth group advised on Policy Alignment, given the number of reviews and reforms in Scotland, recently completed or in process, how we would ensure that the review of qualifications was well aligned with them.

All the groups and themes listed above are crucial to the success of a qualifications and assessment system that is both credible and practical.

Each member of the IRG linked with a broader group of members from their community. IRG members worked with 13 core CCGs and a number of allied discussion groups. For example, within IRG, there were three groups from professional practice. One focused on the Profession, whose CCG included teacher professional organisations and members who worked with learners from a wide range of communities and had specialist expertise in, for example, race, socio-economic disadvantage, carers, able pupils. A second also concentrated on engaging directly with classroom teachers, a third engaged with school leaders and a fourth with college principals.

The number of CCGs increased over the course of the Independent Review as interest grew in the topics and sub-topics under discussion. For example, in the original CCG plan, there was one CCG that brought together employers from national and global companies. During the course of the Review the employer community was extended to include allied discussion groups for public and third sector employers and one for small and medium employers (SMEs). A number of other allied CCG groups were created during the course of the Review.

Each CCG included a wide range of participants to promote an inclusive and participatory approach to the Independent Review. The aim was to have an Independent Review process that embraced the diversity of Scotland’s learners and communities. There was a further feature of IRG that was different from the common pattern of committee membership. IRG members did not seek to represent their own organisations, but instead each acted as a facilitator to stimulate communication between IRG and their wider communities on how a future qualification and assessment system in Scotland might best support all learners in all educational settings.

A key responsibility of IRG members was to facilitate wider community engagement. Each IRG member brought together individuals from their extended community to form a CCG. Through their CCGs, IRG members debated issues and gathered views from individuals and groups to reflect the diverse make-up of their communities. This was a more straightforward process in some communities than in others (see Appendix Five). However, we believe that more than 400 people from a wide range of communities were engaged meaningfully in the three phases of consultation.

The relationship between IRG and CCGs was iterative. Ideas considered in IRG were discussed with CCG members and evidence and insights from the CCGs was brought back to the monthly IRG meetings to form part of the wider evidence gathering process. The three phases of the consultation process provided the structure for engagement with the CCGs.

2.1.3 Systemic Integrity

Systemic Integrity is the third part of the Integrity model that is crucial to the success of any process of reform. In Scotland, a wide range of organisations have to align to support the process of change in any innovation. The Government and all political parties, national agencies and organisations, HMIE, teacher and headteacher, professional organisations, local policy communities, ITE, colleges, employers and universities, all have contributions to make to support the change process if the reform is to be successful.

In addition to the IRG and CCG meetings, we met with key individuals from groups and organisations across Scotland, as well as with strategic committees to explore the contributions that each might make to support the introduction and the longer-term change process to ensure attention was paid to systemic integrity. A list of meetings, events and visits can be found at (Appendix Six).

2.2 Evidence from the Review Process: Phase One – Consultation on the Vision and Principles

As previously discussed, the draft Vision and Principles developed in partnership with the SYP and the Children’s Parliament were discussed in IRG and CCG meetings and became the basis for Phase One of the engagement. Responses were received from CCGs, allied discussion groups and 221 responses were received from schools and colleges. Many of the responses from schools and colleges were based on group discussions with multiple teachers/lecturers and learners, and these responses were submitted on behalf of the group. The feedback from Phase One was independently analysed and the report is available here.

In summary, the evidence from the Phase One consultation supported the fundamental ideas in the draft Vision and Principles statements. However, there was advice to simplify the language and to sharpen statements. The language of both the Vision and Principles was amended, and the number of Principles was reduced to sharpen the ideas. It was agreed that the Vision and Principles should be used as the basis for the design of future qualifications and assessment and as a touchstone to check the relationship between Vision and practice over time.

The redrafted Vision and Principles informed the second phase of the Independent Review, where options, consistent with them, were explored.

2.3 Evidence from the Review Process: Phase Two - Engagement on the Options for Change

Phase Two of the Independent Review began in October 2022 and closed in January 2023. There was also a public consultation. The consultation for this phase explored options to help define the parameters of a possible new approach to qualifications and assessment in Scotland.

IRG discussed the possible options in detail and held discussions with their CCGs. In addition to feedback from IRG, CCGs submitted detailed views and more than 700 responses were submitted from the public consultation. As with Phase One, the responses were independently analysed by the Lines Between, and their report of the analysis is available here. In summary, the main issues raised were:

2.3.1 The Need for change

Discussions in the IRG, CCGs, other Convener led meetings the Independent Review has undertaken and in school/college visits suggested almost universal agreement on the need for change in qualifications and assessment and in how they are used in the Senior Phase. This finding was supported in the responses from almost all consultees responding to the broader public consultation including schools and colleges.

2.3.2 Types of evidence that reflect learner achievement

There was a broad agreement across communities that gathering evidence on learners’ progress and performance in subjects or curricular areas should remain an important part of qualifications. However, across communities there was recognition of the need to go beyond individual subjects/curricular areas. For example, learners should have opportunities to demonstrate skills as they use knowledge from across subjects/curricular areas in real life contexts. Further, learners should have opportunities to have recognised broader evidence of their achievements socially, culturally and economically. Many learners held a strong view on this broader view of recognising their achievements.

“We need to think radically rather than retrospectively here. We need to look at the different subject areas and how best to demonstrate learning in each area and understand that this should look different for each subject. We need flexibility within our assessment system rather than working to a system of when the SQA design and set exams. We need to value all learning and we need to realise the digital opportunities available to enable us to assess learning.” – (CCG member, Teaching Profession)

Although there was a desire for a broader range of achievements to be recognised, the ways in which the four capacities were described were perceived to be problematic. A number of respondents suggested that the four contexts for learning as presented in the refreshed narrative for CfE would offer a better framework for qualifications and assessment. (The Scottish Government, 2019)

Figure 2: CfE Refreshed Narrative

The Curriculum

'The totality of all that is planned for children and young people throughout thier education'

• Opportunities for personal achievement
• Interdisciplinary learning
• Ethos and life of the school as a community
• Curriculum areas and subject

There was also significant support for a broader range of approaches to qualifications. Learners in Scotland were perceived to be over-examined. Many of those taking National Qualifications, had high stakes examinations in three consecutive years, in S4, S5 and S6.

“Scotland may consider ‘de-cluttering’ the historical diets of examinations during upper-secondary years S4-S6, and reflect on when and why Scottish students should take examinations, and consider alternative ways to acknowledge the end of compulsory schooling.” (Stobart 2021, p 41)

“Do we actually need exams to gauge learning and progression? Assessment can do the same job and every does need assessed to demonstrate their knowledge.” – (BOCSH, Headteachers)

“Teachers’ attribute much of the pressure and workload placed upon themselves and on learners to the repeated “two-term dash” and the accompanying “examination rehearsal” which leaves little room for creativity, depth, breadth or enjoyment of learning in the Senior Phase.” – (EIS)

Although pre-COVID-19, almost all National Qualifications included a variety of coursework tasks, consultation responses focused on the significant amount of time spent on rehearsal for high stakes examinations, for example prelims, past papers, rehearsal of approaches to question responses. The idea of two-year learning programmes was attractive to many. However, they argued for an appropriate ‘safety net’, an exit point for learners who decided that they wanted to end their progress in a particular subject at the end of the first year.

Overall, CCGs supported a move towards progressive courses over two years, and the accumulation of course credits throughout the Senior Phase. More broadly, while there is support for a move away from the three separate single year cycles of examinations in S4, 5 and 6, the suggestion that learners would only be presented for external examinations when they exit a subject caused concern.

“This would appear to suggest that a pupil studying English through to S6, for example, would only be examined in that year. That would appear to introduce even higher risk to an already high-stakes system.” – (NASUWT)

2.3.3 Ways in Which Evidence Should Be Gathered and Assessed

There was broad agreement that evidence of learners’ achievements in different areas should be gathered and assessed in different ways. For example, in all subjects, approaches to evidence gathering should include a balance of internal and external assessment including examinations (graded or ungraded) and, as appropriate, projects, assignments, oral assessments, performances and practical work. Each subject should identify methods of assessment that would be best matched its. Alternatively, evidence of learning in wider contexts, for example being part of a school play, or playing in a band, or supporting younger children’s reading, or having a school leadership role should be gathered in very different ways perhaps involving learners in self-reflection or witness statements to verify their contribution. Fundamentally, the approaches to gathering evidence and the ways in which evidence is assessed should be appropriate to the nature of the activity.

“Most staff were in favour of the use of continuous assessment (with external verification sampling), exams at point of exit only (where relevant – i.e. not Practical Woodworking etc) and on greater use of digital technology and oral assessments. How much would be weighted towards continuous assessment is an important consideration – this must be available for all subjects as currently Maths relies purely on the external exam, and this isn’t helpful.” – (Kings Park Secondary School – Learners and Staff)

There was strong support for a wider range of assessment approaches to be introduced. For example, there was evidence that day to day practices in schools in relation to qualifications had become formulaic. It was reported that a great deal of learner time was spent on rote learning and examination rehearsal. In addition, commonly the content of courses and the full range of knowledge and skills relevant to deep learning in a subject area was narrowed to focus on what was assessed. It was the view of some that we spoke to that the qualifications system was driving all work in the Senior Phase, an unsatisfactory position in which the examining and awarding body was in effect determining the curriculum. Whilst it was recognised that neither rote learning nor examination rehearsal are intrinsically unhelpful, these elements have become dominant. It was suggested that an approach such as open book examinations, oral tests and structured observations of activities should be used more widely to broaden the opportunities for learners’ experiences to be better balanced.

There was further agreement about the need for a better balance between school-based evidence and external examination. However, what that better balance might look like was unclear. There were differing views about what school-based assessment might mean. For some, school-based assessment meant a series of shorter tests similar to end of module assessments. Those who argued for this model suggested that it reduces the stress of a single end of course examination. It was noted that within the college system, continuous assessment and internal and external moderation was commonplace.

“Assessment should be a good balance of continuous internal assessment and external assessment.” – (Windsor Park School and Sensory Service)

Those who argued against it suggested that this approach led to a higher assessment burden and resulted only in a series of high stakes experiences. For others, school-based assessment meant gathering naturally occurring evidence as part of learning and teaching. A collection of such evidence would then be moderated to ensure comparability of standards within and across schools and colleges. While it should be noted some learners do prefer examinations, a significant number of people from across communities argued for different approaches to assessment being available to allow learners to have the best chance of demonstrating their achievements. For example, some learners thrived on the deadlines imposed by examinations whilst for others the stress of examinations limited their potential to demonstrate their achievements. For those learners, the less stressful environment of evidence gathered as part of learning and teaching was perceived to improve their performance. Learners often spoke of the stress of examinations and the negative impact it had on their ability to demonstrate their knowledge, skills and understanding.

“We need to ensure fairness and consistency in any model of assessment and examination, while ensuring we provide the less able an opportunity for them to evidence their achievements. I think it is a good idea to move away for the rigid pass and fail system. I really like the idea of teacher / continuous assessment and less exams.” – (CCG member, Employers)

Although the idea of a greater emphasis on ongoing assessment was broadly welcomed, retaining confidence in the assessment system was recognised as being crucial. There was strong agreement that processes should be established to ensure comparability of standards and that this was a national issue, i.e., one where there should be a national plan. The approach to moderation should be proportionate and not overly bureaucratic. Some argued that additional internal assessment could present a potentially unhelpful increase in teacher workload pressures and that workload was an issue that could not be ignored. It would be important to identify what teachers should stop doing if new practices were to be adopted. Adding to existing workload was not an option.

“It would have to be collected in a fair, transparent, reliable, verified, and consistent manner.” – (University of St Andrews)

While information on learners’ out-of-school achievement would be valuable if dependable, it should not be collected by teachers as this would provide added pressure to an already heavy workload.

“It should not be for teachers to judge any intrinsic value of such activities. The role for teachers here is to make opportunities available to record out-of-school or in-school achievements based on what the learners see of value. Encouragement and direction would be the contribution of teachers here.” – (CCG member, Teaching Profession)

Home educators highlighted issues noting it was very challenging for home educated learners to access qualifications and in particular the facilities needed to sit an examination. There was a plea to address this problem.

“I think when home educators are still paying taxes for local services then any reasonable support should be made available for home educated children seeking either practical subjects or qualifications. There should be attempts to offer a blended approach for young people who struggle to attend full time for good reasons.” (CCG member, Parents and Carers)

2.3.4 How Evidence Should Be Presented

Greater use of technology

There was broad agreement that technology must play an increasing part in qualifications and assessment. The design of new qualifications and assessment should assume the use of technology. However, the pace at which learners could benefit from the potential of technology, depended on how quickly the education system could address issues of access, dependability and flexibility of IT systems and the level of skill in the use of technology amongst professionals in learning, teaching and assessment. The different IT systems in local authorities are perceived to create levels of complexity that are not conducive to the creation of a coherent national system. It was argued that the system could learn lessons, both positive and negative, from the introduction and use of GLOW. The need for a system specifically designed for the delivery of qualifications was seen as a major priority. The idea of a technology-based profile of achievements was perceived to be the natural solution for future learners to present their achievements.

Some respondents argued that a technological solution was the only possible way to present evidence of learners’ progress and achievements. They referred to the reasons behind the failure of the previous National Record of Achievement (NRA). It had become a bureaucratic exercise largely because of its paper and folder-based approach.

“I agree that as much information as possible should be gathered however there should be a system in place to ensure that pupils are not disadvantaged by a lack of opportunity outside school or a lack of parental communication regarding out of school learning experiences.” – (Kibble)

While there was some support for creative thinking about the use of technology and its capacity as a time saver, school infrastructure was commonly perceived to be outdated. It was argued that technology would be a key strategy to re-engage learners for whom the existing largely paper and pencil system felt out of date. Some respondents noted a greater potential for cheating provided by a digital system.

“Exams should be digitally produced, and the pupils should type their answers. This would remove many issues like scanning exam scripts and trying to read pupils' writing.” – (Individual, Public Consultation – Phase Two)

2.3.5 Leaving Certificate

Responses highlighted the value of a leaving certificate or graduation certificate as a useful way to recognise a broader range of achievements. This approach had the potential to include wider community achievements, and to give value to the achievements of learners, some of whom may not demonstrate what they are able to achieve in a school context. This approach was perceived to be more consistent with the aspirations of CfE and the cultural shift needed to achieve these.

A leaving certificate was recognised as being a way to offer a more complete picture of every learner. Offering information on a broader range of achievements was argued to be more consistent with the kinds of information learners wanted to offer and employers wanted to receive. Some respondents identified that this approach was consistent with existing practices in a number of high performing countries internationally.

“A leaving certificate is very common in many international and comparable education systems, and they are regularly used in our admissions process to assess wider skills and competencies. We welcome the consideration of a leaving certificate available as part of the Senior Phase, potentially include a broader range of evidence of achievements than subject-based qualifications.” – (University of St Andrews)

“We welcome the consideration of a leaving certificate available as part of the Senior Phase, potentially include a broader range of evidence of achievements than subject-based qualifications.” – (Children in Scotland)

However, concerns were expressed that the additional workload involved in gathering evidence would not be acknowledged and the time needed would not be made available for teachers. A few respondents suggested that there was a danger of repeating what was already on an SQA certificate (even if not always recognised) and how, if not properly designed, this approach could disadvantage the most vulnerable students who could not achieve all the requirements.

“This would be more information and data gathering that would land at a teacher’s desk at the expense of delivering high-quality lessons in class.” – (Individual, Phase Three Consultation)

2.3.6 Certification at BGE

Views varied as to the value of the introduction of a certificate at the end of the BGE. Some argued that this could bring greater focus and purpose to BGE. Others argued that the gathering of information to support a certificate at this stage was a pointless exercise.

“I think this would definitely be advantageous as the jump from BGE and Senior Phase can seem daunting to pupils particularly our young people who may have gaps in their learning and have experienced negative experiences of education in the past. To have a sense of achievement at the end of the BGE phase could motivate pupils to achieve more in the Senior Phase. Less able pupils may not experience the success of attaining an award in the Senior Phase so this could boost confidence.” – (Kibble)

2.3.7 General Issues

In response to the open question, education professionals firstly raised the need to tackle a number of problems: unnecessary bureaucracy, outdated and inadequate technology, workload pressures, national agencies that were perceived to be unresponsive, the need to maintain the integrity and validity of the qualifications system and a degree of uncertainty in interpreting the CfE capacities.

Secondly, they emphasised the importance of system change to support any new approach to qualifications, for example, data and accountability measures (the most common response), inspection processes, relationships within and across national agencies, initial teacher education, continuing professional learning and professional standards (Priestly et al, 2023).

It was interesting to note how many of the issues raised in Phase Two of the consultation reflected ideas that were part of the original Vision for CfE. The concern to reflect a broader range of learners’ achievements, the desire to reduce the number of examinations taken by individual learners, the concern not to introduce high stakes assessment at too early a point in the learning process, for example., as outlined in Building the Curriculum 3. A further area of commonality can be found in the original CfE desire to offer a wider range of approaches to gathering evidence, for example, Building the Curriculum 5 notes – “Learners should be engaged in all aspects of assessment processes and be afforded an element of choice and personalisation in showing that they have achieved the intended outcomes.”

The evidence from the analysis of the Phase Two consultation, together with the revised Vision and Principles from Phase One, were used to design a model that became the basis for the third phase of consultation with communities across Scotland.

2.4 Evidence from the Review Process: Phase Three – A New Model for the Future of Qualifications and Assessment in Scotland

Phase Three engagement took place from 3 March until 31 April. The initial timescales for the engagement were 3 March to 7 April, but this was extended following a request for more time from teachers’ Professional Associations.

An independent contractor, Progressive Partnership, was commissioned to undertake an external analysis of Phase Three responses based on the original timescales. Following the extension, the contractors were able to extend their analysis of responses that were received up until 14 April. This totalled 365 responses via the School and College survey and detailed submissions from 17 CCGs and allied discussion groups. Responses received between 15 April and 31 April were analysed by the Independent Review Secretariat using the coding framework developed by Progressive Partnership. This comprised 120 responses of which 90 were received via the School and College Survey and 30 were received by email.

During Phase Three the Review invited views on a Scottish Diploma of Achievement (SDA) and its three components. Views were sought via the CCGs and invited from every school and college in Scotland. Both respondent groups were asked the same questions. CCG views were elicited through meetings facilitated by the link IRG member and views from schools and colleges sought through an online survey, the “School and College Survey”. This survey was emailed to every secondary and primary school and college in the country along with supporting documentation. The School and College Survey and supporting documentation was also sent to independent schools, to the Grant Aided Special Schools (GASS) and to units providing secure care to young people aged under 18 in Scotland.

Views of Respondents Overall, there were varied views on the Diploma. In general, the responses from the CCGs tended to be more positive than the responses received through the School and College Survey. There are a number of possible explanations for these differences, including:

• The profile of the CCGs differed to the School and College Survey. Most of the responses received via the School and College Survey were received from school communities, rather than colleges, and the majority of these were from secondary school teachers. The CCG responses comprised a far broader range of stakeholders and whilst this included teacher groups it also included learners, parents and carers, universities, Directors of Education, College Quality Managers, employers, researchers and policy professionals. See the full list of CCG membership (Appendix Four).
• The method of engagement was different. A deliberative consultative process was adopted with the CCGs, which meant they had greater access to information about the Review via their IRG link member, opportunities to ask questions and time to reflect during and between each of the three phases. The School and College Survey was not always accompanied by a group discussion and while it was suggested by the Review that educational settings discuss the proposals before responding, it was left up to each setting to determine if/how to do this. The act of discussing as a group prior to completion may have resulted in more nuanced responses, with those completing the online survey only potentially either coming cold to the proposals or feeling that they could be more direct.
• Nearly all who participated in the CCG discussions participated in all three phases of the Review: there were many more responses to the Phase Three School and College Survey than the Phase One survey, which indicates that some respondents to the Phase Three School and College Survey did not participate in all three phases.

2.4.1 The Scottish Diploma of Achievement

While many respondents were positive about the objectives underpinning the Diploma and its three components, there was also concern about how the model would work in practice. There were concerns voiced in respect of equality, particularly in relation to the Personal Pathway component.

“The Diploma as suggested, we feel, reflects many of the original aspirations of Curriculum for Excellence; aspirations which in some ways have not been attained over the course of the last 20 years. In that respect, the opportunity to achieve these aspirations through the introduction of a diploma is laudable. However, we feel that there are real concerns over practicalities, and over the kind of culture change necessary to allow such a development to be successful”. – (Cults Academy)

“Equity is an issue again. Less affluent young people will not get as many opportunities.” – (Kyle Academy, Extended Leadership Team)

In general, the CCGs welcomed the overall Diploma proposal. The Learner CCG was particularly enthusiastic.

Many of the respondents in the School and College Survey also welcomed the proposals for the Diploma. The main reasons given, include:

• would showcase the full range of learners’ achievements;
• would provide a more rounded picture of learners’ achievements and would therefore be beneficial to future employers and universities;
• achievements and certification would align with the learner’s aspirations and next steps in their education/employment and;
• would offer opportunities to reflect the attainments of all learners, not just the most academic.

2.4.2 Subject Studies

In general CCGs, and those responding via the School and College Survey sought further detail around the mechanics of the Programmes of Study component and how it would work in practice. There were some concerns in respect of equality in relation to teacher assessment and the possibility of teacher bias. This was particularly evident in, although not limited to the CCG concerned with Equity.

“Increased diversity in the education workforce - and among professionals who are involved in designing, developing and grading qualifications and instruments of assessment - is a way of avoiding bias in assessment, given the useful diversity in perspectives and lived experience which this brings with it. It remains the view of the DiTPEW group that the diversity of staff not only in the teaching profession but also in the national agencies with strategic responsibility for assessment and qualifications should be promoted.” – (AREP - Diversity in the Teaching Profession and Education Workforce sub-group)

In the main, CCGs agreed with the proposals set out in the Programmes of Study including the suggestion to reduce the amount of external assessment in the Senior Phase. Overall, CCGs supported a move towards progressive courses over two years, and the accumulation of course credits throughout the Senior Phase.

There were differing views among respondents in the School and College Survey in respect of the proposals on the Programmes of Study. Although many responses were positive, a greater proportion had reservations. Overall, teachers responding via the School and College Survey tended to be critical of proposals under Programmes of Study, whereas learners responding via the survey tended to be positive. Interestingly, teachers involved through the CCGs were largely positive, albeit they did raise concerns around potential workload and resourcing. The Learners CCG was very positive.

Teacher workload was a key theme of responses received through the School and College Survey. Teachers were concerned about the impact of increased workload in moving towards more continuous assessment as well as the additional support and training required to ensure the profession was prepared for such a change.

We like the idea of there being the opportunity for diversity in approach to assessment. Concern about the implication in school especially surrounding the workload implications of the accrual of evidence. Consistency a concern – will there be exemplification and clear criteria to support teacher judgement? Assessment needs to remain robust. Workload concern. Important to retain formal tests for some subjects. Will there be a narrowing of curriculum for example if a subject pathway doesn’t go beyond level 5? Or are they going to change the curricular areas? Is the two years S4/5 or S5/6? – (Garnock Community Campus)

Retention and recruitment of teachers and teaching support staff was also acknowledged as a contributing factor to teacher workload issues with one respondent noting that this would need to be addressed as part of wider changes in order for any new system to be implemented effectively.

The issues raised in consultation had a number of implications for the recommendations from this Review (see Chapter Six)

• More detail on the mechanics of the Programmes of Study should be provided without allowing the detail to become prescription.
• There was support for a reduction in external assessment, the option of progressive courses over two years and credit accumulation.
• There would be a need for a national approach to moderation, to build confidence in standards across the country to allow the advantages of classroom assessment to be realised whilst ensuring that no learner would be disadvantaged.
• Professional learning about bias should be an essential component of moderation and should draw on the expertise of the increasingly diverse teaching population.
• There should be specific reference to resource implications including workload.

2.4.3 Project Learning

In general respondents were positive about the objectives underpinning the Project Learning component but were concerned about how the model would work in practice and further detail was sought. There were some concerns in respect of equality.

The CCG discussions on the Project Learning component were positive overall, with learners especially enthusiastic. The key benefits identified included:

• learners would develop skills for future careers; prepare for employment, university and for adult life;
• learners would have opportunities to explore (new) areas of interest and;
• learners could connect and transfer learning and skills into other areas of study and demonstrate what they had learned in an applied setting.

All the CCGs supported the component in principle, but they all indicated concern around its practical implementation and sought further information. Concerns were also raised around how best to assess this component with many of the respondents indicating that some form of assessment, verification or endorsement would be required.

Some respondents through the School and College Survey, in particular learners and colleges, were very positive about this element of the model, with respondents mentioning the key benefits from undertaking project work around a topic of personal interest as motivating learners, developing skills in a range of disciplines, and encouraging innovation and creativity. Other respondents via the School and College Survey, while typically welcoming the proposals, felt that more detail was required, especially in relation to implementation and assessment. This was particularly the case for teachers. Some respondents, largely teachers, noted that past experiences of interdisciplinary learning (IDL) in schools have not always been successful.

“The qualifications and assessment system should lead to the development of skills - there is a challenge here that we are focused on knowledge and not the application of it.” – (Colleges Scotland)

Many respondents both in the CCG discussions and the School and College Survey commented that the proposal demands additional skills from secondary teachers who have often been trained to teach a single subject, and that support for new teachers through initial ITE and ongoing support for all teaching staff through continuing professional development (CPD) would be essential.

“There has to be appropriate training and time for staff to ensure they understand the nature of the changes and prepare for these changes.” – (Shetland Islands Council Children's Services Directorate)

There were concerns about equality across all respondent groups. Respondents wondered whether some educational settings would be better placed to offer opportunities to their learners than others; schools in urban areas were seen to be at an advantage as were schools in more affluent areas. Concerns were also raised about how this element might be more challenging to complete for learners with ASN.

“Focusing on skills and competencies could disadvantage learners with ASN in areas such as communication however (it) could absolutely flourish in practical areas and IT. It definitely needs recognised especially for home educated children with ASN.” - Parent/Carer

The issues raised in consultation had a number of implications for the recommendations from this Review (see Chapter Six)

• support for the proposal across a wide range of communities, in particular learners;
• need for further detail as part of recommendations in particular on proposals for assessment;
• important to understand why previous attempts to introduce project work across subjects had been difficult and to learn from those experiences;
• important to build in time and support for staff and in ITE to develop this strand of the SDA and;
• a need to ensure that opportunities were equitable across the country and for all learners

2.4.4 Personal Pathway

Overall, respondents generally supported the principle of the Personal Pathway but indicated concerns around how it would be implemented and delivered in practice. Significant concerns were raised in respect of equality across all respondent groups.

“I do understand the concerns around equity, but I suppose I think about it, young people with greater access to opportunities through school or out of school will still have that access, whether a light is shown on it by the use of a diploma or not. When learners come to write their personal statements or job applications or complete CVs, they'll be able to bring that richness of opportunities. Rather than shy away from that, does this challenge us as a system to create greater equity of access of opportunities for young people in a better engagement with young people to help them understand how their role, perhaps as a young carer or in undertaking activities out of school. We are allowing them to acknowledge the kind of skills that are going to be really significant in their future, and could actually maybe be richer than somebody who's got access to lots of clubs and activities.” – (CCG member - Public & Third Sector Employers)

“Teachers were worried that the finances of individual families would overly influence outcomes, and that some pupils would be able to secure the relevant badges of attainment while others would have fewer opportunities to accrue the needed cultural capital.” – (NASUWT)

The CCGs largely welcomed the Personal Pathway stating that:

• it would promote achievement beyond academic subjects;
• could be particularly valuable for learners for whom academic learning was not an area of strength;
• could give learners the opportunity to broader their learning, reflect on that experience and to evidence it and;
• would give a rounded and holistic picture of a learner.

“The EIS welcomes an inclusive approach to recognising young people’s achievements, including those achieved outside a formal educational setting. For many learners, such achievements can be among their most valued and transformational accomplishments. Such an approach would have the potential to have real utility for learners in further understanding their own learning to inform future choices; and could correlate well with other elements of the SDA.”– (EIS)

Users of qualifications (employers, universities, colleges) were particularly positive about it noting that it, offered a way to learn more about the learner, and possibly an opportunity to distinguish between several applicants with the ‘same’ subject qualifications.

“Again, welcome this and often when interviewing young people for their first role when they talk about their leisure pursuits, sporting achievements, clubs and volunteering this is when they are much more passionate and speak with real feeling about the subject.” (CCG member, Employers)

Many of the responses received via the Schools and College Survey were positive for similar reasons. It would:

• encourage learners to recognise the skills they are developing in everyday life;
• encourage learners to take responsibility for their own learning via participation in extracurricular activities;
• enable learners to contribute more to the community and volunteer in different organisations, and they can be recognised for their achievements;
• allow learners to fully consider what they want to do when they leave education;
• provide an opportunity to reflect on achievements, but from a wider perspective than just their grades and;
• offer an opportunity to appreciate and celebrate different cultural literacies.

There was a broad agreement amongst all respondent groups that the Personal Pathway component should be validated but not graded. However, there were significant concerns about equality amongst all respondent groups with many commenting that the component could exacerbate existing inequality since learners from affluent/socially advantaged backgrounds may have greater access to extracurricular activities and opportunities. Almost all the respondents highlighted equalities issues, noting for example that learners in disadvantaged areas and rural areas lack access to the same opportunities as city learners from affluent families.

“If they take part in any clubs, for lots of young people it can be a challenge engaging with clubs, meeting new people etc so if they are actively involved in clubs it should be recognised. Taking part in any sponsorship activities and volunteering out with school. It would also be good to recognise work placements and the achievements within them.” – (Kibble)

Many respondents thought measures should be put in place so that those from disadvantaged backgrounds could benefit from the opportunities offered by the Personal Pathway. However, many felt addressing systemic inequalities would be challenging and very resource intensive, with some concluding that on balance it might not be successful.

It is worth noting that since the public consultation was issued, and in response to feedback during Phase Three of the Review, the proposed Personal Pathway Component in the Diploma has been amended. The focus of the Personal Pathway component is not on the number of experiences the learner has undertaken but on what an individual has learned through an experience i.e., the reflection.

The issues raised in consultation had a number of implications for the recommendations from this Review (see Chapter Six)

• Support in principle for the Personal Pathway. Identified as a positive way to offer a more rounded, personalised picture of a learner and to offer all learners wider opportunities to demonstrate achievements.
• Concerns about equity must be addressed. The proposal did not create the inequity but it did highlight its existence.
• Personal Pathway should be validated but not graded.
• The focus of the Personal Pathway should not be on the number of experiences but on what an individual has learned, for example, their ability to reflect on what they have learned.

2.4.5 Parity of esteem

Parity of esteem across qualifications is a longstanding aspiration of Scottish education. Respondents were asked: to promote parity of esteem across all qualifications, academic or technical and professional, should all qualifications at a particular SCQF level have the same name?

Many CCG responses were supportive of this proposal although some stressed that language changes can only go so far in changing attitudes and perceptions of the relative merits of academic and technical, vocational and professional courses.

“The name of an individual course matters less than the integrity and professional level of that course. For example, it is understood that Level 6 Games Design is not as hard as Level 6 Computing; that Level 6 Photography is not as challenging as Level 6 Art. Teachers are looking for parity between courses at the same level. They are also keen to ensure pupils at all levels are able to attain and that schools are not trying to force pupils to sit in a class which is academically rigorous when that is not for them”. – (NASUWT)

Just over two-fifths of the respondents in the School and College Survey simply replied ‘yes’ in response to this question. Some felt that qualifications differ in terms of level, rigour and degree of challenge. Others felt that the name should also reflect the type of achievement and whether the qualification is academic, technical or professional.

“No. The title of the qualification should relate to the knowledge, skills and assessment approaches within it. Highers have credibility due to the connection with a robust and credible assessment approach. Employers and education/training providers will want to understand what the learner has demonstrated they can do and the title of the qualification relates to this. Whilst the same name/title would be more inclusive, there is positive work underway in most schools relating to parity of esteem within the SCQF framework. This can be achieved whilst still recognising different qualifications contain different demands.” – (Mearns Castle High School)

“Overwhelmingly positive response to this. Many schools are now SCQF Ambassadors as they believe this is really important. Definitely support same name for the same level.” – (Eastwood High School)

2.4.6 Reflections on the process

The Review was designed to engage all those with an interest in qualifications and assessment from the earliest stages of the design. The Review process included an extensive range of communities, including learners throughout the process. Having the range of communities within IRG was very positive. All IRG members were given the opportunity to see the issue of qualifications through the eyes of different groups, learners, parents, teachers and lecturers, schools and colleges, local and national policy makers, national organisations, employers and universities. The thematic groups brought different evidence to discussions. The IRG recognised that there were bound to be tensions and openly discussed the issues, with the aim of reaching a design that best meets the needs of Scotland. It is testament to the commitment of all those in the IRG, that despite the highly challenging nature of the conversations and the often-competing views, people stuck with the process and worked through problems.

Having two members of the SYP in the IRG who have recent experience of the Senior Phase ensured the IRG kept a clear focus on learners being at the centre of the Review.

Perhaps most importantly, members of IRG knew the communities with whom they worked and how best to engage with them. They were able to tailor communication from the Review to the specific needs of their community and they had contacts within it that made the Review process more effective and better targeted. The quality of feedback received was rich because of their involvement.

The model embodies collaborative working. However, there are areas where further work needs to be done if education in Scotland is to draw on the creativity and expertise of all those involved in the process. First, despite the wide range of approaches the Review adopted to engage with classroom teachers and to bring resources to their attention, we are aware that not all had the opportunity to contribute. The engagement of all teachers in policy development remains an issue for the sector. It will be crucial that this is addressed if reform of qualifications and assessment further to this Review, is to be successful in practice.

Parental engagement is another area where our collective practice needs to improve. The IRG members who organised the parental CCG could not have been more pro-active. Supported by a grant from Scottish Government, they adapted materials, constructed surveys, held meetings and thought creatively about how best to involve parents. In addition, some, but not all, schools involved parents in their consultation process. Despite this we retain the sense that not all parents have had the chance to engage with the Review. This is another area where a good start has been made but there remains much to do.

Recommendation 2: Continue the process of cultural change

Scotland should retain the structure of the Independent Review Group and allied Collaborative Community Groups as a key method of engagement, as the country introduces and develops new approaches to qualifications and assessment. It should also build on this Review’s attempts to involve every school and college in the country, learn from where it worked well and how that learning should influence future consultations.