Regional Improvement Collaboratives (RICs): interim review

3. RIC planning

Key findings: RIC planning

Overall, regional stakeholders believed that the initial RIC plans they had produced were as good as they could have been within the timescale. There was a strong feeling that the timescales for producing the plans were too tight. Two national stakeholders felt that it was a considerable achievement that the RICs had managed to develop plans and achieve political sign off.

Overall, stakeholders felt that the development of early RIC plans was led by local authority officials. Stakeholders found engagement with schools challenging within the timescales of their phase on planning - with regions large, hundreds of schools, and school holidays during the planning period. Most regional stakeholders indicated that they had focused on further involvement of stakeholders as part of phase two planning.

Most schools involved in this research said that they were not very involved in identifying priorities for the RIC. However, most felt clear about the priorities for their RIC and were positive that these were appropriate.

The process of identifying workstreams for the RIC plans was felt to work well where plans were informed by data and research; there was a clear understanding of what they needed to prioritise; and there was a focus on enhancing rather than duplicating activity. However, a few regional stakeholders had to think very hard about the concept of additionality and what this meant for the RIC, and would have welcomed more clarity on this.

Overall, schools, regional and national stakeholders felt that RIC plans were well connected to national priorities. While most school staff felt that RIC priorities were useful and relevant to the school, most regional and national stakeholders felt that plans were not so well connected to school priorities, due to the timescales.

The extent to which data had informed RIC plans varied. Most felt that they had used data, but in an "adequate" or "limited" way. This was because of the challenges around sharing, collating and analysing the data within the timeframe. Over time, most regional stakeholders felt that the RIC had helped them to share data and talk about the themes emerging across the region.

In some cases, the identification of baseline data was seen as a real strength, enabling the RIC to set clear, measurable indicators and targets for each workstream. However, most regional stakeholders indicated that more work was needed to develop a meaningful baseline and way of measuring progress against this.

Introduction

3.1 This chapter explores approaches to:

• developing RIC plans;
• identifying workstreams;
• involving stakeholders in planning;
• connecting to local, regional and national priorities;
• using improvement methodologies; and
• using and sharing data.

3.2 It draws mainly on experiences of regional and national stakeholders.

3.3 Links to each phase one RIC plan are included in Appendix One.

Developing a plan

3.4 Overall, regional stakeholders believed that the initial RIC plans they had produced were as good as they could have been within the timescale. Most were very used to developing plans, and confident about the approaches they took to establishing the plan for the RIC. Stakeholders felt that the planning process worked well where:

• external support was available - for example universities assisted with accessing and interpreting research, and some accessed expertise around improvement planning;
• time was dedicated to joint planning - for example away days and events involving senior officials and politicians across RIC authorities were found to be very useful, particularly when facilitated independently; and
• authorities were strong in terms of their performance - and understood well what would and wouldn't work in driving improvement.

3.5 However, there was a strong feeling that the timescales for producing the plans were too tight, and that deadlines came at the same time as other existing pressures and priorities. This was particularly challenging as some regional stakeholders also had to manage complex political tensions around the establishment of RICs. The process of developing plans involved a process of negotiation and compromise, exploring what partners were prepared to do.

"There were many hands that wanted to edit… In planning, you can be in danger of making things too complex."
Regional stakeholder

3.6 One national stakeholder felt that it was "a huge achievement" to achieve political sign off of the RIC plans, with another feeling that managing to develop plans had been "remarkable."

3.7 Some of those involved in the RIC were having to develop plans over the evenings, weekends and holidays to get it done. As they were committed to the overall aim of the RIC, this was not felt to be a major problem - but it did mean they felt that there were some unrealistic expectations about the RIC plans, and a lack of recognition of the resource constraints.

"It was a really busy time for councils in terms of other submission deadlines. Adding the RIC plan process on top of this made it really hard."
Regional stakeholder

3.8 In particular, most felt that plans could not be informed by the views of a wide range of stakeholders, as it was not possible to undertake high quality engagement and consultation activity within the time available. A few highlighted that the RIC plan was iterative and developed over time.

Involvement in planning

3.9 Overall, stakeholders felt that the development of early RIC plans was led by local authority officials. This included the Director of Education and Chief Education Officers. In a minority of cases other partners such as NHS, police or employability colleagues were involved.

3.10 All were strengthening their approaches to engagement and consultation as part of their phase two plans. Over time, many RICs had worked to involve schools, colleges, universities, young people, parents and other wider stakeholders. Both regional and national stakeholders felt that universities and colleges could be more involved and could offer particular expertise and assistance in relation to improvement methodologies and research.

3.11 Stakeholders found engagement with schools challenging within the timescales of their phase on planning - with regions large, hundreds of schools, and school holidays during the planning period.

"This was a particularly hard ask within the timescale."
National stakeholder

3.12 In some areas, headteachers were consulted, for example through a short survey. However, regional stakeholders would have liked to do more, as some felt that this largely focused on checking that the themes were correct, rather than more in-depth consultation. In some cases, teachers and other partners were informed of plans and direction of travel, but not formally given the chance to contribute.

"We are unsure of the extent that priorities have been driven by the need of local schools. The visibility is not there yet."
National stakeholder

Example: Strengthening involvement in planning for phase two

For the phase two plan, one area set up meetings with teachers and support staff. They analysed the available data in depth. This confirmed the priorities identified in phase one. Consultation with young people in this area also found that health and wellbeing was a key priority - and this is being added as a priority in the phase two plan. Further work is being done to involve young people in the other workstreams - for example they aim to develop a workstream on closing the attainment gap to be led by pupils.

3.13 Most schools involved in this research said that they were not very involved in identifying priorities for the RIC, due to the tight timescales.

"In my opinion it probably happened too quickly… They had to have a plan in place by a certain deadline. Is that the best way to go about developing something like this?"
Headteacher, secondary

3.14 However, some said that even in short timescales there had been good opportunities for schools to be involved - for example through online surveys or opportunities to comment on draft plans and priorities.

"It was a real opportunity for practitioners to be able to influence what was happening at RIC level."
Headteacher, primary

3.15 Whether involved in the process or not, most school staff felt clear about the priorities for their RIC and were positive that these were appropriate. A few highlighted that while they were aware of RIC priorities, they felt that there would be many other headteachers in the area who were not aware of the plan or priorities. Just two headteachers (both in the same RIC area) highlighted gaps in the RIC priorities - suggesting that there should be a priority around Gaelic medium education.

Identifying workstreams

3.16 The process of identifying workstreams for the early RIC plans felt to work well where:

• workstreams were informed by data, research and existing evidence;
• there was a focus on identifying a small number of priorities - helping to build a clear understanding of what they needed to focus on;
• there was a focus on enhancing what local authorities were already doing, rather than duplicating activity;
• where priorities build on joint work that had already been happening, often more informally, between authorities; and
• partners met face-to-face to discuss and agree priorities at senior level.

Example: Identifying workstreams through evidence

In one area, partners quickly became aware that numeracy in primary schools was a challenge. There was a reduction in the proportion reaching the reading level between p1, p4 and p7 across all the authorities involved in the RIC. The same trend was seen, not to the same extent, for literacy. Because of the local evidence, and the clear link to attainment and national priorities, these were clear early priority workstreams for the RIC.

Example: Connecting with school inspection reports

In one area the regional advisor looked at all of the inspection reports across the RIC, from early years to secondary schools. There was a strong correlation with RIC workstreams, and the RIC was clear on what practitioners were looking for.

3.17 One regional stakeholder felt that the workstreams were important as a starting point, with the aim of building a new way of working across a much wider range of areas.

"The workstreams were a vehicle for creating collaborating capacity. People would get used to the idea of collaboration and begin to evolve partnership working across schools and authorities."
Regional stakeholder

3.18 However, there were challenges to identifying workstreams. A few regional stakeholders had to think very hard about the concept of additionality, and what this meant for the RIC. A few were unsure whether sharing practice - for example where one authority had a particularly good approach - was a legitimate focus for the RIC, as they felt their activity may need to be entirely new for all partners. A few regional stakeholders said they were still unsure what the term 'additionality' meant and how to interpret this within the RIC.

3.19 One national stakeholder indicated that it would have been useful to clarify that 'additionality' could be achieved through rolling out good practice within collaboratives, building on existing very strong performance of some partner authorities.

3.20 There were also specific challenges for different RICs. For example, in one area it was felt that priorities were very high level during phase one, and limited in scope. Conversely in another area it was felt there were too many priorities, and a need to reduce the number of workstreams.

"The scope and ambition of phase one was quite tight."
Regional stakeholder

3.21 Regional stakeholders felt that the workstreams and priorities they were developing for phase two were stronger. In some cases, further research and consultation had strengthened the rationale for focusing on existing workstreams. In other cases, the research and consultation identified a need for new workstreams, which had been developed for phase two plans. Most indicated that their phase two activity was more informed by research and consultation than their phase one activity.

Connection to school, local, regional and national priorities

3.22 Overall, regional and national stakeholders felt that RIC plans were very well connected to national priorities, as set out in the National Improvement Framework (NIF). Regional stakeholders highlighted that they focused strongly on the priorities within the NIF and felt that their plans were well aligned with NIF priorities. Headteachers also highlighted positive connections with the National Improvement Framework and closing the poverty related attainment gap.

3.23 However, most regional and national stakeholders felt that plans were not so well connected to school priorities. Regional stakeholders felt that more needed to be done to focus on the priorities that are relevant to the region, and the individual schools - but that the timescales for producing phase one plans made this very challenging. Regional stakeholders largely felt that the approach to the RIC plan had been top-down rather than bottom-up, due to the timescales. Most had tried to do some analysis of school priorities, as best they could in the time available.

3.24 Most school staff felt that there was a lot of similarity between school, local authority, regional and national priorities, with everyone working towards the same broad aims. Some highlighted that these were flexible, and could be tailored to meet needs at school level.

"The focus on improving attainment in literacy and numeracy meant it was natural for us to get on board. This was very relevant and could have a positive impact on our school."
Headteacher, primary

"It is good to see how the priorities translate from the national level to local level in a streamlined way."
Headteacher, secondary

"There is a clear link between all of the plans. It all ties up."
Headteacher, early years

3.25 However, a few headteachers felt that there were too many plans. One said that the RIC priorities covered everything that was relevant to schools, and a few felt it would be very interesting to see what was prioritised in terms of areas of focus or activity within RIC workstreams.

"The collaborative must ensure that the priorities and workstreams do not remain too high level, and bring this into the classroom so that it impacts on class teachers."
Headteacher, secondary

Example: Gathering headteacher views

In one area, the RIC looked at school plans and discussed common themes with headteachers. The RIC plan was based on what schools wanted, and needed, to improve.

3.26 While one national stakeholder had expected more alignment with school priorities, another felt that this was too much to ask within the phase one RIC plans, and the short timescales for developing these.

"They are not as connected as we would have hoped, particularly at school level."
National stakeholder

"It would have been a small miracle if they could have done that."
National stakeholder

Use of improvement methodologies

3.27 Most regional stakeholders indicated that improvement methodologies were used as part of the planning process. This included using driver diagrams and tests of change. In some cases, these approaches were used as part of the planning process, or in workstream development, but were not included within the final RIC plan. Regional stakeholders also highlighted other methods such as action research, population segmentation and outcomes focused planning.

3.28 A small number of regional stakeholders expressed concern about a focus on using improvement methodologies, feeling that other approaches to planning - including those already used within authorities - could also be effective. One felt that they already knew what needed to change, and did not need to test approaches.

"We know what needs to be done at systems level. It is about consistency."
Regional stakeholder

3.29 One national stakeholder felt that using improvement methodologies was challenging in view of the tight timescales for phase one plans, and the challenges around capacity with RIC partners also already doing their 'day jobs'. Another national stakeholder indicated that it would take time before monitoring and evaluation systems were in place for all RICs.

Use of data and data sharing

Use of data in phase one plans

3.30 All regional stakeholders indicated that they used data to inform the RIC plan. The extent to which data had informed phase one RIC plans varied. While a few areas said that they experienced few challenges gathering, sharing and analysing the suite of data they needed, most experienced some challenges. Most felt that they had used data, but in an "adequate" or "limited" way.

3.31 In some cases, RICs found it challenging to access data at regional level, using robust and reliable data sources. A few found it hard as the data they needed was not produced at RIC level. A few felt they needed support from the Scottish Government in accessing the relevant data at regional level.

"National data needs to be available at the regional level in a frictionless and timely way."
Regional stakeholder

"They have analysed the evidence at the level they can, but they want to go further and deeper."
Regional stakeholder

3.32 Most said that they did the best they could within the timescales for producing phase one plans. This included:

• using existing data - focusing on the data they already held, including publicly available data; and
• using data which could be easily shared - including high level or local authority wide datasets.

3.33 Regional stakeholders reported that approaches to using and sharing data had continued to develop and were a key focus for phase two plans. This included analysis of data on attainment, attendance, exclusion, school leaver destinations and performance inspections. Regional stakeholders felt that Education Scotland regional advisors had provided access to substantial analysis, to inform phase two plans in some areas.

Example: Focusing on data during phase one planning

In one area, the RIC established a data group led by a local authority data officer. This group looked at data around curricular achievements and school performance. It also looked at local authority plans, children's services plans and other local and national plans. In addition, it analysed findings from a survey of 1,300 staff, and explored findings from a discussion group with children and young people.

Establishing baselines and monitoring systems

3.34 In some cases, the identification of baseline data was seen as a real strength, enabling the RIC to set clear, measurable indicators and targets for each workstream.

3.35 However, most regional stakeholders indicated that more work was needed to develop a meaningful baseline and way of measuring progress against this. Generally stakeholders felt that this was "work in progress" which would evolve and be refined over time.

"It is still early days, but there is a real focus on how to measure progress and impact over time."
Regional stakeholder

3.36 In some cases, RICs had lots of data, but found that their baselines and monitoring systems were drawing on a data suite which was too large, had too many indicators, or that their indicators were not smart enough.

"The question is where you stop. The bigger you make the package of data, the harder it is to see the priorities."
Regional stakeholder

"We need to get a bit slicker. We have a lot of data."
Regional stakeholder

3.37 There was some concern from regional stakeholders that in some areas the outcomes set were not capable of measurement, because of a lack of data available at RIC level.

Data sharing

3.38 Most regional stakeholders felt that the RIC had helped them to share data and talk about the themes emerging across the region. Data sharing was reported to work well when:

• there were data sharing agreements in place;
• data was shared at the appropriate level - for example sharing local authority level data rather than school level, to reduce concerns about data protection; and
• authorities within the RIC area used the same systems - meaning that collating data was reasonably straightforward.

"We have created a collaboration and had the conversation, so expect that we will share information. A year ago this would have been an individual request to another director, and they would maybe have asked what we would do with it."
Regional stakeholder

3.39 In one RIC, regional stakeholders felt there were no challenges to data sharing. However, most RIC areas experienced some challenges.

3.40 A key challenge for phase one planning was sharing data in a way which enabled comparison or collation. Often local authorities recorded information in different ways, and some found it laborious to get data into a similar format across authorities. It also took time to explore key issues emerging where authorities had very different profiles - for example very varied levels of deprivation. In some cases, there was a reluctance to compare authorities - particularly if authorities were performing at very different levels, and worried about being portrayed in a negative light.

3.41 A few regional stakeholders found it challenging to access data from wider colleagues - such as children's services or health.

3.42 Some of these challenges arose because the RIC was not a legal entity, with clear organisational responsibilities around data control and processing. This necessitated the development of information sharing agreements.

"There were challenges about different views on the level of data we could share. Was it appropriate to share school level data, or at a higher level."
Regional stakeholder

3.43 A few regional stakeholders indicated they managed to share some data, to enable them to develop the plan, but that further work was ongoing around data sharing. One national stakeholder indicated that it was unrealistic to expect the phase one plan to be underpinned by regional level data, within the timescales.