Reducing Reoffending Change Fund Evaluation of Year 1 - Public Social Partnership Development

5 Co-production

5.1 As explained in the introduction, co-production - which involves public and third sector bodies co-designing services or interventions to deliver agreed social outcomes - is a key feature of the PSP model. This chapter sets out partners' views on the different approaches to co-production and the key features of the process. The chapter also focuses on the study research questions to assess whether partners feel that co-production is having a positive, negative or no impact on the quality of the services that have been developed, and what challenges they have faced through the co-production process.

5.2 The key findings are:

• In Year 1 of the RRCF, co-production primarily focused on third and public organisations working together to design interventions with service users' views fed into the process following consultation.
• Two approaches to co-production were identified - refinement of an approach proposed by the lead organisations, and a more fundamental approach to the design of a new service from scratch. Feedback on the latter process was very positive. Views on the success of the two approaches varied and preclude conclusions about which is more effective.
• The co-production process typically involved a number of tasks with a wide range of partners to identify gaps and evidence of need, consider how schemes linked with existing and developing interventions, and define objectives and outcomes. The process varied between PSPs and some challenges were faced.
• The process of co-production was one of the elements of PSP development that partners found most valuable, in terms of learning about services, sharing knowledge and experience and fostering a system of close joint working. The majority of partners felt the proposals were co-designed and the process has had a positive impact on the quality of services developed.

Approaches to co-production

5.3 Co-production, also described as co-design or co-planning, is increasingly important in the delivery of public services in Scotland. Definitions from other policy areas^[12], highlight the role of service users and providers jointly designing services which can include the service user expressing some choice over the services they use. In the Reducing Reoffending Change Fund PSPs' co-production primarily focused on third and public organisations working together to design interventions with service users' views fed into the process following consultation (as described in Chapter 4).

5.4 The importance of the process to the PSP model is highlighted in Ready for Business' PSP Guide^[13]:

'PSPs are based on a co-planning approach, through which the public sector can connect with third sector organisations (voluntary, charity and social enterprise organisations) to share responsibility for designing services based around service user needs rather than the current suite of standard services which users can access'.

5.5 Two approaches to co-production were identified during interviews with partners:

• Most commonly, lead organisations started with a service model in mind, which in some cases was an existing service, and refined this model with input from partners throughout the development process.
• It was less common to find that PSP partners had met with "a blank sheet of paper" at the outset and designed a new programme from scratch.

5.6 Feedback on the co-production process was very positive among partners involved in designing a new programme from scratch. They welcomed the opportunity to contribute in this way to the design process and felt they had played a significant part in the PSP development process. They also highlighted the strength of partnership working that this process involved.

5.7 Views on the success or otherwise of the two approaches identified above varied between PSPs and preclude conclusions about which method is more effective. Some partners felt that initial clarity of vision from the lead organisation was helpful because it provided focus and a basis for refinement whereas other partners felt that this gave lead organisations ownership and limited their ability to influence design.

Key features of the co-production process

5.8 Partners described a range of aspects of the co-production process which typically involved the following:

Where it was felt necessary PSPs undertook initial work to ensure all partners understood the PSP model, to establish clarity about the development process and partners' roles and responsibilities.

Many lead partners developed a PSP timetable which identified clear milestones. In some cases project management techniques were also adopted, for example Prince 2 principles.

Evidence gathering activities and service user consultation was undertaken (the extent to which findings were shared with partners varied)

The majority of PSPs held some variation on a co-production workshop with partners to facilitate service design. If lead organisations had a service model in mind they presented this to partners, invited them to share their initial thoughts; design work stemmed from there. Typically the lead partner led these workshops, where the agenda would include a discussion of the RRCF criteria, evidence of need and service user involvement.

Some of the larger PSPs including the national bids had wide reaching discussions with partners and undertook extensive consultation. To facilitate this process they established themed workstreams to consider different elements of the programmes being developed

Once an agreement in principle was reached, much of the detailed planning work was undertaken by the lead partner. Partners inputted through ongoing email and telephone exchange, with documents circulated for comment and approval. Where necessary, face-to-face meetings were arranged to discuss and resolve issues.

5.9 There were variations in the extent to which each of the steps outlined above involved all partners, and were viewed as successful. There were also variations in timing and the process was not a linear process as summarised above. The majority of PSPs began designing their service before the evidence gathering had finished. Many partners felt that there was not enough time for research to be completed before design work began and therefore adopted a simultaneous process of information gathering and designing.

5.10 The following were identified as key tasks in the co-production process:

• Identifying gaps and evidencing need.
• Considering how schemes would link with existing and developing interventions.
• Defining objectives and outcomes.

5.11 These issues and the associated challenges identified by interviewees are discussed below.

Identifying gaps and evidencing need

5.12 All PSPs undertook work to identify service gaps although this varied between the partnerships. Some PSPs undertook detailed mapping, survey work and consultation with partners, while others used publicly available information. In some areas PSPs relied on public sector partners' local knowledge of provision rather than undertaking a systematic approach to identifying gaps. In both approaches, partners reported that they were generally satisfied with the information gathered and that it was sufficient to identify gaps.

5.13 One PSP attempted to assess how effective existing mentoring services were instead of just identifying that they existed. The lead organisation acknowledged that this was a difficult task as it relied on open information sharing in a short period of time but they felt it was important as they sought to develop high quality evidence-based mentoring services as outlined in the Guidance.

5.14 A small number of interviewees highlighted challenges related to the scope of the mapping exercise. For some the challenge was clearly defining what services were to be included and in one PSP this led to a public sector provider initially not providing information because they did not see their services as mentoring - the lead organisation clarified their requirements and the information was provided.

5.15 All PSPs also made efforts to assess and quantify the need for their proposed service although how this was done also varied between partnerships. PSPs approached this activity differently, some relied on statistics (usually provided by the CJAs), others gathered qualitative evidence such as case studies of service users or samples of Criminal Justice Social Work Reports, and some incorporated the findings from evaluations of their own existing mentoring services. All lead organisations highlighted that the evidencing of need was sufficient for their needs.

5.16 Evidencing need was described by several partners as one of the most challenging elements of the PSP development process. Different reasons were given for this, including a perception that there was limited information. One PSP was working in a field they described as "under researched" - undiagnosed mental health issues - and found there was little existing evidence for them to draw on; they attempted to address this knowledge gap by undertaking their own research with service providers and users but found it difficult to engage service users. Another PSP had hoped to undertake a Social Return on Investment approach but found that there was not sufficient data to do this. In another PSP, a public sector partner reported that the lead organisation's data requests were time consuming because they necessitated the establishment of new procedures for gathering information from existing reports. Some partners were unsure about what relevant data existed, and those involved in more than one PSP commented that larger, more experienced lead organisations were better at gathering evidence and knowing how to use it. Some partners suggested that this element of PSP development could have been undertaken centrally, for example by The Robertson Trust, the Scottish Government or the CJAs, to make sure that there was a level playing field in terms of all partners having access to the same data and to prevent overlap.

5.17 Some interviewees highlighted that there may have been duplication of effort in identifying gaps and evidencing need where similar exercises were being undertaken by a number of PSPs locally and nationally. Some interviewees were of the opinion that this might have been perceived as a nuisance by the services they were trying to get information from. A more co-ordinated approach to service mapping was suggested by a number of stakeholders including lead organisations, public sector partners, and the Scottish Mentoring Network which stated they could have provided detailed information on mentoring services, including justice services across Scotland.

Considering how schemes would link with existing and developing interventions

5.18 Discussion took place in all PSPs about linking proposals with existing and developing interventions and they were mostly viewed as successful. In a number of cases PSP partners provided a direct link to existing or developing interventions, for example:

• In one regional PSP, two large third sector partners which were also leading their own Reducing Reoffending Change Fund PSPs agreed that their bids would take account of the proposals in the region so there was no duplication between the proposals.
• In Glasgow key organisations in the City - Glasgow Community and Safety Services, the Police and CJA - instigated discussions with a national PSP bidder to ensure their proposals took account of existing services in the City and importantly adopted the 'One Glasgow' approach to service provision.

5.19 A number of partners highlighted challenges in taking account of the programmes proposed by other PSPs, particularly how the proposed local and national bids fitted together and whether cross-PSP referral pathways should be considered during the development phase. There were issues gathering information on some services when some partners were reluctant to share information, and uncertainty about how to then plan services to take account of services that may or may not come to fruition. Some partners suggested that the competitive nature of the development period prevented PSPs from sharing information about their proposals - some described "an air of secrecy" between PSPs brought about by the competitive element of the process. There was experience of other providers who were not partners in PSPs being reluctant to share detailed information about their service provision, feeling that the information was commercially sensitive.

Defining objectives and outcomes

5.20 Each PSP specified the intended outcomes of their programme within the MoU or work plan although detail varied across PSPs.

5.21 There were variations in the extent to which PSPs approached the process of defining outcomes and these differences link to the model adopted for co-production. PSPs that began with a specific service in mind often had a clear idea of the outcomes that service model was based around. In these cases partners focused more on the practical process of agreeing to or refining the finer detail of the model more than undertaking work to define outcomes. In areas where partners approached PSP design with a "blank piece of paper" the identification of outcomes was generally more prominent and an important first step with service design built from there.

5.22 In interviews, the majority of partners within PSPs described a shared vision and understanding of the high-level outcomes their services was designed to achieve. Most commented on longer term outcomes linked to the RRCF including most explicitly reducing reoffending or described broad impacts such as engaging harder to reach offenders, increased provision of tailored services and adding to the evidence base on mentoring. Some partners reflected on the outcomes for mentors such as learning, skills development, improved self esteem, and others identified outcomes for partners for example larger networks, closer joint working and skills developed throughout the PSP development process. A minority of PSPs investigated methods of measuring the impact of their proposed services although many did not go into such a level of detail and some suggested that the independent evaluation of Years 2/3 would achieve this.

5.23 In discussions about outcomes there were mixed views about the flexibility of service design. Some interviewees felt that there was flexibility within the operational period to review and adjust services once their impact was clear, however others suggested that the time for reflection would come at the end of Year 3, when services where being put out to tender.

Views on co-production and its impact on the quality of the services that have been developed

5.24 Overall, the process of co-production was one of the elements of PSP development that partners found most valuable, in terms of learning about services, sharing knowledge and experience and fostering a system of close joint working. In most cases co-production represented a significant shift in the dynamics between third and public sector partners because it involved both parties having a clear stake in the programme design process. This was very different from the majority of stakeholders' previous experiences, which tended to involve the public sector designing the service and the third sector seeking to deliver it to a prescribed service specification. The majority of partners within PSPs felt the proposals were co-designed and have had a positive impact on the quality of services developed. They welcomed the opportunities to influence service design even if they chose not to. The survey of PSP partners found that 82% of respondents agreed that the PSP will have a positive impact on the quality of services being developed, 80% agreed that the PSP will be able to identify the social benefits delivered, and 76% agreed the PSP will be able to prove the social benefits delivered (see Appendix 7).

5.25 In some of these cases, partners suggested that the timescale limited the extent to which they were able to scrutinise the final versions of proposals - with bids being worked on up until the very last hour before the deadline - but they felt this was an issue with time rather than an issue with the co-design process. Some public sector partners were generally of the view that there was little consideration of the demands placed upon them by the PSP development process alongside their existing responsibilities. This was most relevant where the scale of the proposal was relatively small and relates to the lessons learnt from the earlier PSPs that inputs should be proportionate to the expected outcomes.

5.26 Several interviewees described the enthusiasm and "can do" attitude of the partners involved in co-design activities. When probed, they used terms such as "motivated", "engaged" and "innovative". Some suggested that the economic climate has reduced the number of opportunities for public sector staff to develop new approaches; that instead they often required to focus on finding ways to increase efficiencies in existing services. These interviewees felt the RRCF represented both an opportunity and resources which allowed partners to create something that they might not have been able to otherwise.

5.27 A small number of public sector partners described instances of "feeling in the dark" and being asked to support programmes they had not helped to develop. The reasons given for this varied - a minority suggested that the PSP lead misunderstood the development process, others suggested the PSP lead was not interested in partners' views, and one suggested that the PSP lead was ultimately unclear about the service they were developing, undertook too much consultation, and found it hard to articulate their ideas to partners or facilitate co design in a meaningful sense.

5.28 Some public sector partners suggested that the PSPs led by large national third sector organisations possibly had an advantage when it came to co-production, in that they were probably more experienced and comfortable in the use of tools such as logic models.

5.29 Partners interpreted their roles in the co-production process differently. Some partners believed their role was to input into the design process, question proposals and suggest alternatives - where this occurred they tended to be supportive of the co-production model and where it did not occur they tended to have some reservations. Other partners felt their role was to provide support for a bid, not to question it, and generally they had no strong views on the co-production model - this suggests the principles of co-production and the PSP model were not fully understood by these partners.

5.30 A small number of partners felt that the legal aspects of co-production were unresolved, in terms of how intellectual property was to be managed and who could claim ownership of the model developed at the end of Year 3, when it goes out to tender.