A research strategy for Scottish education

A Research Strategy for Scottish Education: Detailed proposals

The context

The National Improvement Framework and Improvement Plan priorities

The four priorities set are:

• Improvement in attainment, particularly in literacy and numeracy;
• Closing the attainment gap between the most and least disadvantaged children;
• Improvement in children and young people's health and wellbeing;
• Improvement in employability skills and sustained, positive school leaver destinations for all young people.

The primary aim of the research strategy is to support the achievement of these priorities and the delivery of the Improvement Plan. In identifying the actions required we have also drawn on the recommendations of the OECD in Improving Schools in Scotland: An OECD Perspective.

The OECD recommendations

The OECD was tasked with making recommendations which would ensure that Curriculum for Excellence ( CfE) effectively delivers on behalf of children and young people. They, and the implications we have identified as a result, include:

• Be rigorous about the gaps to be closed and pursue relentlessly "closing the gap" and "raising the bar" simultaneously. This implies work to identify and collect appropriate data for measuring the social patterning of attainment, and ensuring that the overall strategy enacted helps both the targeted demographics, as well as the system as a whole.
• Ensure a consolidated and evidence-informed strategic approach to equity policies. This implies work to identify evidence for the most effective interventions, and the key facilitators for them.
• Develop metrics that do justice to the full range of CfE capacities informing a bold understanding of quality and equity. This implies efforts to collect data for a wider range of indicators.
• Focus on the quality of implementation of CfE in schools and communities and make this an evaluation priority. This implies work to find out what has gone on in schools and what has worked "across the board and not only in exemplary sites".
• Develop targeted, networked, evaluated innovation in secondary school learning environments to enhance engagement. This implies development and verification of interventions which are successful in engaging secondary school pupils.
• Develop an integrated framework for assessment and evaluation that encompasses all system levels. This speaks to the need for data throughout the system that can be used to attest to progress.
• Strike a more even balance between the formative focus of assessment, and developing a robust evidence base on learning outcomes and progression. While much of the recommendation speaks to existing assessment processes under CfE, ensuring the system is generating readily understood data that provides a simple read-out of progress and allows more complex evaluations, is necessary to give useful information to practitioners.
• Strengthen evaluation and research, including independent knowledge creation. This implies supporting work for the above recommendations, but also ensuring that there is both opportunity for independent challenge to the system, and the chance to develop long-term research which may extend the bounds of knowledge.

The NIF priorities, the Improvement Plan and the OECD recommendations, taken together with the outcomes of consultation activity with academic and education stakeholders, suggest a range of research activity which can be set out in three distinct categories:

• Supporting research infrastructure and independent research;
• System performance and 'what works'; and
• Empowering practitioners to produce and use evidence and data.

This approach is designed to transform the production and use of evidence, and become an integral part of the promotion of better outcomes. In addition, a common thread throughout the strategy as a whole, is the importance of increased collaboration and co-operation within the education system. This increase speaks to both within level co-operation (e.g. academics working with other academics) and between level co-operation (e.g. academics working more closely with practioners). The aim of this joint up approach is to reduce "silo working" and help strengthen the capacity of the system overall.

We believe that each level of the education system in Scotland has a vital role in harnessing the power of evidence and data in order to deliver continuous improvement in the education system. Leadership at system, local authority, school and class level is key to ensuring that effective collaborative approachs are in place to learn lessons for successful intervention. In particular:

• The Scottish Government and Education Scotland must work effectively with national bodies such as GTCS, SCEL, ADES and SCDE to ensure that international evidence is effectively sourced and made available to the school system;
• Local authorities must collaborate effectively with school leaders and teachers to identify patterns in outcomes across their areas, design and deploy appropriate interventions and monitor impact to learn lessons for improvement, feeding success and learning back into the national policy context;
• School leaders must collaborate with their teachers and support staff, parents, children and young people to identify patterns and outcomes within their schools, design and deploy appropriate interventions and monitor impact to learn lessons for improvement;
• Teachers must collaborate with their support staff, parents, children and young people to identify patterns and outcomes for individuals. In addition, they are expected to design and deploy appropriate interventions, and monitor impact to learn lessons for improvement;
• All levels must show show leadership and exemplify inquiry to those above and below them, seek to collaborate, and commit to learning from data and evidence, whether gathered routinely, or to attest to specific interventions. They should promote a culture of sharing their lessons candidly and seek to use these as an opportunity for improvement in a spirit of mutual support.

Supporting research infrastructure and independent research

To take this forward, we have identified a range of actions to be undertaken. The partnership between the Scottish Government and the Education Endowment Foundation and the development of Education Scotland's Interventions for Equity web resource will help develop the Scottish research infrastructure and resource. To develop the research infrastructure we will also examine:

• Provision for self-directed research by academia and other non-governmental institutions. This implies funding, either by non-specific grants or endowments. This could be provided through existing mechanisms (e.g. Scottish Funding Council, Research Councils) or by challenge funds whereby organisations are encouraged to bid for grants within broad parameters.
• Research on inequalities in education requiring a multidimensional approach. Further facilitation of cross-cutting research could be encouraged through examination of changes to the funding regime and through the commissioning of research.
• The utilisation of existing datasets in order to provide new analysis or focussed, evaluative work. Again we will consider the funding mechanisms in place, but also the training of new quantitative researchers, both in fundamental methodology and in familiarity with our education databases. Additional work will take place on engagement and boosting networks for quantitative analysis in education and maximising access to Scottish Government data.

In addition the following projects may be explored :

• Research to evaluate non-standard approaches to education and engagement (in part-fulfilment of the recommendation to evaluate innovative engagement) to see if these are approaches or models that should be available to local authorities in a more systematic way, and whether these are effective approaches in closing the attainment gap.
• An options appraisal to consider the way forward for educational research following a potential withdrawal of EU Research and Development funds.

System characteristics and performance: What works and what has worked?

The National Improvement Framework emphasises the importance of evidence and performance information. The choice of information is crucial, and for public confidence, there must be clarity of focus. But there are key research questions to establish how a full picture of system performance is gathered. This involves looking across and considering all four of the capacities stated in CfE, as we are encouraged to by the OECD, and also maintaining a long-term perspective which ensures that "performance" is never reduced only to technical fulfilment of chosen indicators.

What works in Scotland?

In a sector which has been undergoing long-term change with a new curriculum, new assessment models and the advent of new technologies, there is a need to maintain and keep up to date a comprehensive evidence base on what works. What has been shown to work in the past may no longer be true now or in the future. In addition, we need evidence from Scotland to understand if lessons from other countries are genuinely applicable in the Scottish education system where organisational assumptions may differ greatly.

There is a need for a more co-ordinated approach to knowledge creation through the synthesis of existing knowledge, which is then translated into a form that would be easily digestible for use by those on the front line. Education Scotland's Interventions for Equity web resource ^[2] is the beginning of such an approach.

Identifying the more useful research from around the world and making it available in a user-friendly format is an on-going and time-consuming process. To that end, we have entered into a partnership with the Education Endowment Foundation to provide succinct summaries of the international evidence appropriate to the Scottish system in a way that enables practitioners to readily act upon what has been shown to work.

The following underlying principles are key to developing a programme of research:

• We aim to undertake focussed research which will provide insight into how the twin aims of raising attainment for all, and closing the attainment gap will be achieved. We are mindful that there is a risk that universal measures can sometimes exacerbate social patterning. The approach of the Scottish Government is to "level up" not "level down". Research will be undertaken to identify both universally available interventions which do not leave lower performers, and people from deprived backgrounds, behind; and interventions targeted at specific groups of learners to help ensure that all children and young people have the same opportunity to achieve their potential. This will link up with the work of the National Improvement Hub in partnership with the Educational Endowment Foundation.
• The PISA 2015 results also made clear that performance of highly able children and young people is a cause of concern, and we must also ensure that this is not neglected while we hope to raise attainment across the board.
• What makes the difference? We will undertake analysis which seeks to quantify the differences that school interventions make relative to social change, and within both these categories, identify the most effective interventions. While we know that the relationships between poverty and attainment are complex, we must make every effort to identify what has the most impact in bringing change.
• Learning from outliers. As well as identifying programmes successful in raising average attainment, we must also learn from all individuals and schools who thrive in unexpected circumstances.
• Policymaking. Evaluation should not necessarily be confined to identifiable projects, but to overall policy and stated values - both in performance against their own objectives, as well as against the wider values and policies espoused by the Scottish Government. This includes wider consideration of Curriculum for Excellence as a whole. Of itself, this does not imply large-scale evaluation projects, but is a factor when designing project-specific evaluations, or evaluation programmes. There are also existing monitoring commitments, such as PISA and the data gathered on achievement of Curriculum for Excellence levels, which provide an insight into policy impact.

There are drivers of change that are particularly important as we look to 'what works'. These include:

• Leadership. Change will only happen with the aid of leadership at all levels. The OECD has identified in particular the need for "the middle" to take a greater role in delivering educational change, rather than more top-down leadership from the Scottish Government and Education Scotland. School leaders in particular need to be able to support practitioners to take part in networks that effectively share good practice, while being empowered to take decisions on effective collaboration.
• Learning and Teaching: Pedagogy already attracts a lot of interest in current research. However, what is crucial is identifying effective practice that works in improving the relative attainment of students from poorer backgrounds. To date, the evidence available in Scotland may have focused more on measuring attainment and describing the problem rather than interventions that serve to reduce the gap. In addition to a more general focus on attainment and wider achievement, updating the evidence bases on literacy and numeracy with evidence from Scottish schools will also be key.
• Engagement with families and communities: While schools are only one influence on student outcomes , it is reasonable to ask what schools should do to work with the grain of their students' lives, or even be a catalyst for change. Parental engagement and family learning are key contexts in which this can take place, and work is already underway to further develop practice. However, this also requires research that goes beyond the boundaries of schools and links with evidence around community learning, poverty and health and also research which identifies the effective ways whereby education professionals can link with relevant support agencies, consistent with Getting it Right for Every Child.

Performance

We will continue with the following activity already in hand that provides evidence of performance at system, local authority, school and individual level:

• Collect data under the Programme for International Student Assessment ( PISA), where we are already making preparations for the 2018 data collection, for publication in December 2019. PISA has been measuring Scotland's performance compared to the OECD and other countries since 2000, so represents a baseline that precedes CfE and has been a constant throughout its development and implementation.
• We will support the NIF-based work to identify appropriate indicators for public accountability, including agreement on progress towards closing the attainment gap. This may include commissioned research and reviews on the response to indicators in comparable education systems, and monitoring of how our own indicators are received. We will ensure that future health and wellbeing data collections can be adapted to measure the "other three" capacities of CfE (Effective Contributors, Responsible Citizens, Confident Individuals).
• The collection of teacher professional judgement data looking at the achievement of CfE levels informed from 2017/18 by the new national standardised assessments.
• The existing evaluation activity for the Scottish Attainment Challenge will continue.
• Future waves of Behaviour in Scottish Schools Research will continue looking at newly emerging issues within school such as cyber bullying.

• Additional Support Needs ( ASN). The evidence on how to include students with ASN most effectively in the education system changes all the time, with new interventions and approaches to complex additional support needs. This will remain under review.
• System review. The OECD Review represented a holistic overview of the entire Scottish school system with particular reference to the implementation of the Broad General Education Phase. The option was also left open to have a further review of the Senior Phase of CfE at a point in the future when the new National Qualifications were bedded in.

In addition we will explore work in the following areas:

• A study on the long-term development of a bespoke index of social background which will create individual-level (as opposed to area-based) data involving consideration of the data collected at school registration. A bespoke index will enable more targeted and effective intervention for disadvantaged pupils, and also better take into account disadvantage of those who do not live in deprived areas (usually the greater share of deprived students).
• We will monitor the behaviour of secondary indicators which have not been identified as the official measures of the attainment gap, but provide other valuable information about the performance of the system. The purpose of this activity will be to maintain a long-term perspective.
• We will maintain a watching brief on whether PISA should remain our only commitment for international comparisons, or whether it should be supplemented by other International Association for the Evaluation of Educational Achievement surveys.
• Existing work includes the review of Health & Wellbeing data designed to collect data for a range of outcomes under the National Improvement Framework.
• Development of a longitudinal approach to student achievement by combining pupil-level data, with data on social background and teacher judgement data to follow students through their school career, and allow long-term follow-up to gather information on destinations and labour market outcomes.

System configuration

There are key questions about how the teaching profession and school leaders are able to utilise both lessons from research, and become adept at using data. The question of capacity is a key concern expressed by teaching unions and academics. There are parallel processes underway which seek to address workload; however there is also international evidence which suggests that preparation time is key to enabling teachers to improve pupil performance by developing carefully calibrated strategies. This raises issues around what happens in schools and how teacher workload is allocated and managed.

• Research on the organisation of teaching capacity may be useful. This would look at reducing bureaucracy, the role of administrative and support staff, the role of specialist support for learning and evidence champions, larger class sizes in some circumstances or more teachers overall.
• Teaching Scotland's Future considered many new options for reconnecting with former teachers and integrating teachers trained outside Scotland. There is some progress being made in this, but what is required to make the teaching profession competitive for the most engaged graduates, and existing members, particularly if increasing numbers are required? Accordingly, workforce planning will be a key area of research.

Empowering practitioners to produce and use evidence and data

We need to support practitioners to be able to act on lessons derived from research. They also need to be able to understand the lessons from data that they have gathered themselves. Training and support to practitioners, both teachers and school leaders, are necessary as well as a continuing commitment to learn about how data is being used and to act on its implications.

There is a key challenge to build on and foster the development of new and existing networks at a variety of levels, not least as evidence suggests that personal contact is particularly effective in knowledge translation. We will aim to foster greater communication between teachers and researchers. Potential options include the creation of research champions at a school and local authority level, development of local and national events to share research practice, collaborative research with the teaching profession and establishing a research forum. This would not only enable the more effective dissemination of research, but allow practitioners to be more effectively engaged in the research process.

We will ensure that Education Scotland's forthcoming knowledge mobilisation strategy, and the actions that follow, are informed by the most up-to-date evidence. We will host further engagement events with the academic community to ensure that policy priorities more effectively inform research and there is greater awareness of the data available. Moreover, this will help ensure that policy development is more effectively informed by research.

To further these objectives we will aim to undertake a study that will cover:

• The level of evidential knowledge that a teacher has to have in order to effectively prepare for teaching interventions so that 1) the needs of pupils are met, 2) individual pupils who are falling behind are supported, 3) the impact of social background on performance is fully taken into account and ameliorated;
• How this information can best be delivered to the individual teacher;
• How evidence can be fed into overall school management;
• How teachers can be equipped to interpret the results of new standardised assessments and act on their findings;
• How school management and leadership can be equipped to identify patterns in the assessment of children's progress across their school, and act on that knowledge;
• How evidence and data literacy can be "championed" at every level;
• What system-level resources would inform and support these champions;
• How to support effective networks for knowledge transfer;
• Recommendations for how research literacy is taught in initial teacher education.

We will also consider what steps could help teachers and school leaders make the most productive use of standardised assessment data as it builds up over time.

We will also consider what opportunities can be enhanced or made available to the existing teacher workforce, that will enable research knowledge to be embedded, and research literacy in changing evidence to be improved. This may cover secondments, sabbaticals, collaborative research, Scottish College for Educational Leadership ( SCEL) and academic fellowships and masters-level learning (or doctoral study) for both teachers and leaders.

Delivery

The activities suggested in this document represent a significant potential research programme, opening a number of questions which may have numerous implications for the education system and all those involved. As such, the Strategy needs to be taken forward in a transparent and collaborative way.

Governance

We will incorporate the governance of this programme within the wider governance arrangements in place for education in Scotland, and in particular will ensure reporting lines to the National Improvement Framework Programme Board.

We anticipate that the International Council of Education Advisers will receive regular updates on the strategy and be able to comment on our work at any time.

In addition, an Academic Reference Group, containing a wider group of researchers and stakeholders, will be convened to offer advice and guidance on the future direction of the strategy.

We do not anticipate there is a need to set up an independent agency to deliver this programme in its entirety. We are committed to transparency in publishing this programme, publishing research outputs, and where appropriate, making our data available. There are elements where we seek to encourage independent research, both for the furtherance of knowledge and to create new capacity.

A certain proportion of this work may involve a continuing role - e.g. a need to maintain accessible up-to-date summaries of the state of existing evidence on interventions. Where there is enough work to require on-going capacity in a given area, we may give consideration to commissioning centres of excellence that can carry this work forward.

A National Advisory Group will be made up of organisations who have a direct role in evaluation and research. They will feed into the development and evolution of the strategy, but have a more direct role in delivering particular aspects and will be well informed about existing investments and networks in educational research. The membership will include organisations such as the Scottish Government, Education Scotland, SQA, ADES, GTCS, Research Councils and SERA (as a direct link to the academic community). The secretariat will be provided by the Scottish Government. The Advisory Group's remit will include the following:

• Commenting on the Strategy and suggesting further topics for investigation;
• Overseeing an on-going engagement process with stakeholders;
• Providing input into progress reports on the strategy for Ministers;
• Identifying and, where necessary, agreeing who should commission, delivery partners for the strategy, e.g. research projects, data collections;
• Providing input on dissemination;
• Working closely with groups with similar interests, such as Education Scotland's Knowledge into Action Strategic Stakeholder Group to ensure effective synergy and minimise duplication;
• Providing an overview of existing activity across member organisations and identifying gaps as they emerge;
• Help to ensure that the Strategy is delivered in a way which makes most use of existing relationships and complementary agenda across stakeholders.

Cost

A number of activities outlined in this strategy pre-date, or have already been committed to within the National Improvement Framework including:

• Continued participation in the Programme for International Student Assessment ( PISA) for 2018;
• Development of Health & Wellbeing data collection;
• Development of a partnership with the Education Endowment Foundation to support the National Improvement Hub hosted by Education Scotland;
• Completion of the 2016 round of the Behaviour in Scottish Schools Research;
• Evaluation of the Scottish Attainment Challenge.

Resources and priorities for additional work will be established mid in 2017 in conjunction with partners.

Evaluation

A formal evaluation of the Research Strategy will take place after five years which will seek to identify whether it has met the OECD's recommendations. In particular it should cover: increased capacity for independent research and analysis; increased utilisation of evidence and data throughout the system; and the contribution that this has made towards meeting the NIF priorities.

The current research strategy is primarily focused on the work that is being conducted in schools and school age children. However the aim is to expand the focus of the strategy in the long term is to encompass the full educational journey in Scotland

At that time, recommendations for improvements to the strategy for the next five years and beyond will also be made. The formal evaluation will be overseen by a steering group comprised of experts drawn from outside Scotland.