Scottish education system: knowledge utilisation study

Executive Summary

Introduction

This report presents the findings from the Knowledge Utilisation Mapping Study project conducted by the Robert Owen Centre for Educational Change and commissioned by Scottish Government. The research was undertaken from April to October 2018. 

Aims

Two main questions framed the research, each with related sub questions:

1. How do practitioners in Scotland engage in research and act on research evidence?

2. What factors influence practitioners’ ability to make the best use of evidence?

In defining research evidence, the study included three main types:

• School level data, often collected routinely to help understand pupil’s attainment and achievement
• Accessing secondary research findings and knowledge such as books, and academic journals
• Conducting practitioner enquiry and action research, whether individually or collaboratively.

Research strands and methods

The research project comprised three strands:

• A literature review
• A qualitative strand with interviews and focus groups with 67 key informants at various levels of the education community in six local authorities. A total of 6 head teachers, 8 depute head teachers/ senior management team members, 3 principal teachers, 4 Attainment Advisors and 5 regional improvement collaborative leads were interviewed. Ten focus groups with primary and secondary teachers were conducted across the six local authorities (41 participants in total)
• An on-line survey of 1,036 practitioners across Scotland to help validate the findings from the qualitative strand. Responses were received from all local authorities in Scotland although the numbers from each authority were not necessarily proportionate to the size of their staff complement.

Limitations of the research 

The purpose of our qualitative sampling and selection of participants was to obtain insights and perspectives from key stakeholders; however, the findings cannot be generalised to all of the participant groups. The research also included a national survey to validate the qualitative findings. This analysis suggested that there was a strong level of consensus across participants in both the quantitative and qualitative strands of the project. The relatively high levels of engagement with data and research across participants and respondents could indicate that those participating in the research are not typical of the population of Scottish teachers. Nevertheless, the findings are salient, particularly those regarding factors that influence engagement with and use of data and research evidence.

Key findings

Literature review

• There is general agreement that teachers’ roles have developed to incorporate a greater focus on research engagement and practitioner enquiry. Government, OECD and international research literature concur that teachers’ engagement with research is crucial for school and teacher effectiveness.
• There is little literature on the extent and nature of practitioner engagement with research in Scotland. In contrast, there is more literature on the factors influencing teachers’ engagement with data and evidence. This emphasises that data and research is most valued by practitioners when it informs effective learning and deals with specific aspects of practice. 
• The literature suggested that research is only likely to make a difference to practice if: the available evidence is in a form that teachers can readily understand and apply; there is a culture of research engagement in the education system; and there is time to access this material.
• Specialist and partner professionals, such as educational psychologists and speech and language therapists also have a key role in building teachers’ capacity to gather and use data/ research evidence.

Qualitative findings

• The qualitative findings revealed that engagement in research was viewed as central to teachers’ professional identity. Overall, the research, particularly the insights from Attainment Advisors and Regional Improvement Collaboratives (RIC) leads with their strategic overview, illustrate a growing but uneven capacity in the system regarding practitioner research engagement and skills. 
• The research evidence and data that practitioners engage with is most commonly school-level data or online summaries of research findings to assess school context, levels of pupil attainment and inform planning and pedagogical approaches in the school.
• Overall, the considerable body of literature regarding the factors influencing practitioner engagement with research aligns with the findings from the qualitative strand of this study. In particular, teachers are more likely to seek out and use knowledge when it can be seen to be readily applied to promote effective learning. 
• The most compelling factor seen as influencing practitioner engagement with research is that of time. Providing the time and space for practitioners to meaningfully engage in collaborative professional dialogue regarding data, research and their practice was seen as crucial. The literature indicates this, but the qualitative findings bring this into sharp relief. While school planning, accessible research sources and personal motivation can offset the impact of time pressures on research engagement to some extent, the current nature of teachers’ workload acts as a systematic inhibitor to increased research engagement. Participants, especially Attainment Advisors, RIC Leads and head teachers stressed that the current workload of practitioners places considerable time limits on the extent to which they could engage in research and innovate beyond the basic scrutiny of school-level data. Time for dialogue between practitioners and researchers is a particularly important factor in translating research findings and data-informed insights into classroom practice.
• Despite our participants reporting generally high levels of confidence in their skills regarding data and research use, they also report requiring support to analyse and critically evaluate research evidence. Guidance from Attainment Advisors, educational psychologists and academic colleagues is key for supporting this process, at least initially. Attainment Advisors have an increasingly important role in acquiring research knowledge and mobilising this across the system, particularly in their local authorities.
• Head teachers and other school leaders usually see it as their responsibility to keep abreast of knowledge on pedagogy and ‘what works’. Subsequently, they are important intermediaries in knowledge translation. Moral and practical support from school leadership, the local authority and Attainment Advisors is crucial in building a culture of research engagement and capacity at local level. 
• There were limited examples of collaborative research projects within and across schools where teams of teachers and their head teachers focussed on a particular challenge, again within the context of raising attainment. These were usually supported by external critical friends such as university colleagues and Attainment Advisors and facilitated by the Attainment Scotland Fund. Teachers and head teachers find that involvement in small-scale, collaborative interventions with associated enquiry to monitor impact can help to build confidence and capacity of staff to engage with research.
• The resources and funding provided by the Attainment Scotland Fund has helped build systems and capabilities that have fostered use of data and research. This has included resourcing staff to focus on data use and enquiry as well as drawing on external sources of expertise.
• There were examples of local authority programmes to build practitioner capacity and skills, however, in some cases; there was a reduction in local CLPL as financial cut backs continued to affect staffing levels. Strategic participants noted that the emerging brokering role of the RICs in coordinating and facilitating partnerships and sharing of information should enhance the mobilisation of knowledge across the system. Practitioners also called for improvements in the scope, accessibility and usability of research information in repositories accessed via central portals.

Quantitative findings

• The survey findings largely echoed the key themes in the qualitative findings, particularly calls for dedicated time to engage with research evidence. 
• More than  half of respondents (59%) indicated that they were currently involved in one or more research-related activities.
• Almost four out of five respondents reported using data/research material to inform teaching and learning while just over two thirds indicated its use in understanding the impact of teaching and learning.
• The supports rated as most helpful in planning and developing practice were: taking part in structured collegiate discussions, CLPL courses/opportunities or working with colleagues in other schools/centres. These seemed to be those that offered both the dedicated time and opportunity to collaborate with colleagues. Practitioners also regularly used web searches to find relevant evidence. 
• Substantial numbers of staff thought they already possessed relevant research skills, but  respondents also generally stated that they needed support to develop their skills, particularly in relation to analysis of quantitative and qualitative data.
• A large majority of respondents indicated a need for the following; dedicated time to engage with research evidence (84%), national advice and support on engaging with research evidence (79%), opportunities to work with colleagues on research activities (74%) and partnerships with research specialists (74%).

Discussion

The research findings highlight a number of issues for consideration.

Time and workload

The most important factor seen as influencing practitioner engagement with research is that of time. The findings highlighted the importance of teachers having sufficient time to access, interpret and apply data and evidence and that current workload can inhibit this process.

Relationships between practitioners, researchers and policymakers

The literature and our findings highlight the importance of researchers working closely with practitioners and other partners to better convey research findings to influence practice and educational thinking but also to enhance research skills. Given this, the education and policy community could consider how academics and others can work more collaboratively at a local level with teachers. 

The influence of key actors 

The findings identified a number of key actors that were important conduits for knowledge transfer and mobilisation in education. This included: Attainment Advisors; Educational Psychologists and other allied professionals; academic researchers; and leads at school and local authority level. These findings suggest that the education and policy community could explore how these actors can be further supported in their knowledge mobilisation and leadership roles and how such arrangements become more consistent across Scotland.

Resourcing research engagement across local systems

In addition to local authority and other personnel supporting practitioner research engagement, there were examples of local authority professional learning programmes aimed at building practitioners’ data and research capacity and skills. The value of investing in such central support to deploy specialists, provide CLPL and promote the transfer of knowledge across local authorities appears clear. 

The value of collaboration to foster engagement with research and data 

While there were limited examples of collaborative practitioner research within and across schools, such arrangements demonstrated that teams of teachers supported by their head teachers and others could enhance the capacity of staff to systematically engage with data and research.

Accessibility of research findings

A strong theme across the literature review and our empirical findings was the issue of how academic findings regarding effective education approaches could be conveyed in a more valuable way to inform teachers’ practice. This suggests there is scope for academics and policy partners, working with practitioners, to explore how research findings can be better communicated to the teaching profession while retaining appropriate rigour.

Existing educational infrastructure 

Currently, at the policy level, the Attainment Challenge and the associated Attainment Scotland Fund are working as drivers to focus teachers’ attention on the value of evidence-based practice. At the same time, aspects of teachers’ professional environment may not always be conducive to practitioners’ engagement with data and research. For example, as discussed above, time emerged as a significant barrier impacting on engagement with research. 

The findings also indicate that the education landscape is changing as the Regional Improvement Collaboratives (RICs) are established. This presents opportunities for knowledge mobilisation and teacher engagement with data and research. Further consideration, therefore, could be given to how strategic RIC policies and approaches regarding knowledge mobilisation coherently articulate with, and support, the use of data and research at regional and local levels.