Schools - impact of Regional Improvement Collaboratives on pupils and practitioners: review

Part 2: Reach and Impact on Pupils and Practitioners – Summary of RIC Evidence

Key Themes/Findings

• RICs have evidenced significant engagement with and support to practitioners across early years, primary, special and secondary settings.
• evidence indicates that engagement with practitioners and pupils has continued to expand since the previous review
• while the number of staff engaged in regional activities at any time remains a minority, RICs supports have continued to increase with over 17,450 practitioner interactions and 638 school supports evidenced
• the use of data and evidence, including self-evaluation from practitioners and other recipients to shape RIC support, has developed significantly since the previous review
• practitioner evaluation of RIC programmes is consistently strong
• RICs identified capacity and resources as a challenge to further expanding the reach and impact of regional working, and highlighted the year-to-year nature of funding support as a factor which has required management to deliver and extend their regional offer

Introduction

This section summarises the main findings from the review returns and supporting evidence provided by each RIC. This was further supported/informed by individual meetings with each RIC.

In considering the evidence provided by each RIC and the findings highlighted in this section, it is important to note that RICs operate within a complex education system and – as highlighted elsewhere this report and in previous reviews - it is not possible for the majority of RIC activities to robustly attribute or quantify their impact on practitioners or pupils. While a number of selected examples provided to this Review and referenced in Part 5 of this report do identify attainment data before and after regional interventions or support, the majority of ‘impact’ evidence provided by the RICs to this review is informed by practitioner feedback and evaluation, including in some instances external evaluation, supported by wider regional data on improvement trends.

The evidence provided to and considered in this Review has also been supported by significant developments in the evaluation and reporting frameworks, tools, drivers and progress measures which underpin the detailed work of each RIC. Examples of these arrangements are highlighted elsewhere in this report, including in Part 5, and can also be viewed in each RICs detailed review return and supporting evidence.

Some RICs have also highlighted the short timescales for this Review and confirmed that the latest evaluation reports for session 2022-23 were not all finalised or signed off by their respective Boards within the review timetable. A variety of evidence for the period since the last RIC Review has therefore been considered.

Evidence Summary and Conclusions

General points

All RICs have provided evidence on a wide range of engagement with and support to leaders and practitioners across early years, primary, special and secondary settings. The evidence submitted to the Review includes the sharing of information and best practice examples, the facilitation of staff networks, the on-line and in-person provision of leadership and pedagogical training, production and dissemination of guidance, toolkits and other resources, towards more bespoke, session-long collaboration and cohort-focussed programmes of support. These are further detailed in each RIC’s review return/s, the supporting evidence provided and in the published materials that each RIC makes available to practitioners and other stakeholders.

While a majority of evidence provided to this review focuses on support provided directly to leaders and practitioners, each RIC has also provided evidence of support to and/or designed around the needs of pupils. In considering these examples it should be noted that RICs were established to provide improvement support to leaders and practitioners and not directly to pupils.

All RICs have also evidenced through this review and through the Scottish Government funding process that the funding provided by the Scottish Government to support RIC capacity and regional working is appropriately and fully directed towards their regional teams, plans and activities.

Engagement with practitioners and pupils

Communication and awareness raising:

All RICs have evidenced a range of communication approaches to ensure that stakeholders, including schools, ELC establishments and CLD practitioners, can access the latest regional support resources and networks. Each RIC provides online access (via web/Glow/etc.) to a suite of improvement and evaluation tools, guidance, case studies, learning events and other regional resources and contact information, which each RIC promotes through the proactive publication and dissemination of, for example, newsletters, blogs, messages on X (previously known as Twitter) and other platforms/approaches, including existing local authority staff communications.

RICs have evidenced a significant expansion since the previous RIC Review in the use of focussed communication approaches across the above and other platforms. While the evidence provided to this review demonstrates increased numbers of practitioners engaging in regional programmes and thereby indicates increasing awareness of the RIC offer, it is recognised that many staff are not engaged in and are not aware of the support provided by their RIC.

While the proportion of staff engaged in and/or otherwise aware of their RIC cannot be robustly identified from the evidence provided, it seems reasonable to conclude from the overall school (638), leader and practitioner (17,466) interactions evidenced by each RIC, that around 30% of schools across Scotland engaged in RIC activity across the latest session for which evidence was provided. Further research would however be required to more robustly evidence that figure.

It is recognised that increasing awareness of, and encouraging further engagement in, RIC working requires a continued focus and further development. In this review, RICs have highlighted challenges in ensuring that as many practitioners as possible are aware of the regional support offer and how to access the resources provided. Additional points highlighted include:

• as noted elsewhere in this report, RICs highlighted that they do not currently have the capacity to interact directly with each and every establishment and/or practitioner or pupil across their region.
• it was also highlighted that the capacity of practitioners to proactively engage with/find out more about the work of their RIC was also a factor in further raising awareness.
• practitioners may not always be aware of the RIC role in the development of regionally designed or facilitated learning resources that may be cascaded to them by other routes.

Support to establishments and practitioners:

All RICs have evidenced a wide range of support to leaders and practitioners across early years, primary, special and secondary settings. These include:

• staff and specialist networks established in each region for leaders and practitioners to collaborate with their peers, build confidence, share knowledge and best practice, and to engage in further collaborative work within their subject or cohort:
  • all RICs have evidenced that membership of and engagement with professional networks in their region has expanded since the previous RIC Review, including in extending networks to wider groups of practitioners and/or to specialist groups.
  • there has also been evidence in some RICs of a shift towards self-sustaining networks, which determine their own workplans and timelines. This was also noted by focus group participants, who expressed a view that those networks still require a level of RIC support to ensure they remain active.
• professional learning opportunities, ranging from one-off in person or virtual events, to session-long cohorts. These include targeted supports and programmes to develop leadership skills, strengthen pedagogy, support confidence and consistency in assessment and moderation, and share and develop knowledge in aspects such as digital learning, pupil health and wellbeing, UNCRC and other specialist areas.
• the expansion of digital/virtual learning support and online resources. As evidenced in the last RIC Review, RICs pivoted to supporting digital/online and hybrid learning in responding to the COVID pandemic. While that demonstrated the agility of RICs and helped to further raise their profile within their regions and across the system, a theme from this review is how the use of digital resources have been significantly developed in each area. This is evidenced:
  • generally, across all RICs, in the expansion of online, guidance, toolkits, webinars, case studies and other virtual tools, recordings and learning/classroom resources which can be accessed remotely, often when is most convenient for the learner or practitioner.
  • through the expansion of online learning platforms to provide professional learning support to practitioners and/or to provide pupils with greater support in both the BGE and senior phase, including through learning platforms/resources such as: e‑Sgoil, the @SWConnects, the Tayside Virtual Campus, and the West OS, which teachers from across the region also contribute to.

While it is not possible to uniquely identify and/or collate the total number of leaders and staff actively engaged in regional working, the evidence provided demonstrates significant numbers of leaders and practitioners engaging in these networks and programmes, with over 17,450 practitioner interactions and 638 school supports highlighted by RICs in their returns and supporting evidence. When compared to the evidence considered under previous RIC Reviews, this indicates a significant increase in the number of schools and staff involved in and supported by regional working.

With regards to the impact of RIC support to leaders and practitioners, each RIC has provided evidence on internal programme evaluations, including on feedback and self-evaluation from programme participants. Participant evaluation has been consistently strong across the RIC programmes where that information has been provided by participants and submitted to this Review, including in respect of the relevance of the RIC support provided and the positive impact on participants’ knowledge, confidence and/or professional practice.

In further detailing these and other examples, both in their evidence returns and in direct discussion as part of this review, all RICs expressed the view that many of these programmes would not have been delivered without the additional capacity, structured approach and networks that their RIC provides. That view was also expressed in the focus group discussions with school leaders, practitioners and other partners.

Some RICs and focus group participants also expressed the view that RIC supports may bring particular added benefit to smaller, and potentially remote or rural, authorities, who may lack the capacity to deliver programmes at the scale or pace provided by their RIC.

Support to pupils:

The majority of evidence submitted to this review details RIC support to school and early years leaders and practitioners to strengthen improvement approaches and pedagogical practice in their school or early years settings. Each RIC has also however evidenced support which is more directly tailored to and/or focussed on wider issues and opportunities affecting children and young people. These include examples such as:

• online learning support for pupils, including the materials and virtual classes provided by the West Online School (previously referred to as West OS) and the online Advanced Higher (AH) offers provided through @SWConnects and the Tayside Virtual Campus.
• the Forth Valley & West Lothian initiative to raise attendance, which is being shared across RICs (see also Part 4 – Deepening Collaboration).
• a range of support relating to UNCRC, strengthening pupil voice, youth participation and engagement.
• a further range of tools, guidance materials and resources focussed on supporting learners in areas such as Health & Wellbeing, STEM and other subject support.

With regards to the reach and impact of these and other RIC activities engaging directly with pupils, information on pupils accessing the online Advanced Higher options and on the number of young people involved in some participation networks are included in the RIC examples set out in Part 5 of this report and in the more detailed evidence submitted to this Review. As previously stated, RICs were established primarily to enhance the improvement support that is provided to establishments and practitioners.

Data, Evidence and Evaluation:

While the collective impact of regional working cannot be expressed in terms of pupil attainment, all RICs have evidenced significant progress in their use of data, evidence and evaluation. Each RIC has evidenced robust approaches in the use of data, information and evaluation, to: inform their strategic priorities and programmes, more consistently evaluate and articulate the support provided to learning settings, pupils and practitioners, and to further review and refine their regional support offer.

The detailed approaches differ across the RICs, however the following developments in the use of data and evaluation have been observed:

• RICs have evidenced a strong focus in the use of evaluation and user reflection and feedback. This is most clearly evidenced though a range of examples from practitioners, school leaders and other recipients of support from each RIC, supporting both qualitative and quantitative analysis on the impact for their professional practice and, where available, on the observed impact for children and young people. The school and ELC-level feedback provided to this review has been consistently positive on the relevance and impact of regional programmes and support.

  RICs have also evidenced the use of toolkits and/or templates to support consistency in the systematic evaluation across programme levels. Some also evidenced ‘You Said We Did’ approaches to demonstrate to practitioners what actions were taken as a result of feedback and evaluation.

  The use of self-evaluation to inform individual learning needs within cohorts and to further refine regional programmes has also been strongly evidenced.

• RICs have evidenced the use of school and local authority/regional data, including data on attainment and pupil attendance, to inform regional priorities and/or progress. This is evidenced in different ways across the RICs, however all have highlighted the use of school and local authority improvement plans and inspection reports, as further evidence sources to ensure the relevance and responsiveness of RIC support to establishments. Many RICs have also evidenced the use of strategic and/or stretch aims, to support coherence across authorities and to ensure a regional focus on programmes which contribute to improved outcomes for learners.
• evaluation and analysis inform strategic decisions on programmes and resources. A number of RICs identified that analysis of user evaluations and needs-assessments from practitioners has informed a shift in many areas towards focussing regional resources more closely on pedagogical support for practitioners and support for assessment and moderation. In responding to those identified needs, many RICs have further refined and targeted their offer, for example in providing moderation support across different phases and groups, including for pupils with additional support needs and for moderation within and across establishments.

Regional Capacity and Funding

As outlined in Part 1 of this report, regional capacity is limited and the funding which has been provided by the Scottish Government to support RICs approximates to about 0.1% of education spend.

RICs commented that it is not reasonable in that context to expect current regional resources/teams to be able to reach all schools and early years establishments in their region, and that it was their view that the current RIC supports reflected good value for the funding provided. This view was also expressed by some focus group participants, including by the Robert Owen Centre in their external assessment of the West Partnership.

All RICs commented that, while many regional programmes were designed to be scalable and were being expanded where practicable, capacity and funding was a major barrier to further increasing RIC reach and impact. The impact of inflation on the ability to sustain and/or expand current levels of activity was also highlighted, as was the year-to-year approach to funding, which limits longer term planning.