Scottish Rural Network: evaluation

Stakeholder Engagement and Communications

Key Findings:

What has gone well:

• The SRN has supported stakeholder engagement through the development of and participation in events and workshops.
  
• The SRN has informed the public about rural development funding.
• The SRN has developed and grown a range of communication channels, including a mailing list and social media profiles.The SRN has produced a range of case studies highlighting the benefits of the RDP.
• The SRN has been able to assist in facilitating engagement in consultations with rural stakeholders.

What the challenges are:

• While it is positive that communication networks have grown, it is not always possible to link this development to the achievement of core SRN aims.
• Stakeholder engagement is hard to objectively measure, making the effectiveness of the approach challenging to evaluate. Similarly, it is challenging to evaluate the impact of non-systematic interventions, like events and filmed case studies.

Policy Priorities

The responsibilities for stakeholder engagement and communications are reflected in the first and third priorities assigned to the SRN. The first priority is:

"Get more people from rural communities, businesses and the wider public involved in policy developments that affect them."

This is further subdivided into the following requirements

• establish and maintain a central "network of networks" relating to rural development;
• organise and facilitate national, regional and local initiatives and events to inform policy development and programme implementation

At a general level, this requires the SRN to increase the number of stakeholders and participants involved in policy developments that are relevant to them. Methods for doing this are then specified. They include maintaining a central 'network of networks' and organisation initiatives and events. For ease of reference, the above goal will be called 'stakeholder engagement'.

The second priority is:

"Inform public and potential beneficiaries about policy and funding opportunities (as per the Information and Publicity Strategy)"

This, again, is broken down into different, more specific goals:

• promote best practice in rural development through the use of case studies, best practice competitions and peer-to-peer learning;
• gather information, photos and videos showing good examples of projects covering all priorities of the SRDP;
• provide a new or refreshed NRN website;
• disseminate information to the public though newsletters, website articles, videos, social media and public events.

How has the SRN Pursued These Goals?

The SRN has pursued the above goals via the organisation of and contribution to events, the production of case studies to share best practice, and communication across a range of mediums. These are discussed in detail below.

Events and Event Support

Events were mentioned by a number of participants as being a particularly valuable part of the SRN's contribution to the RDP. Throughout the 2014-20 period, the SRN has developed, contributed to and facilitated a range of events concerned with rural policy development. In particular, these have offered opportunities for rural networking among attendees, learning, and policy discussion. Where appropriate, they also generated some outputs from events, including reports, summaries and short films. Key events which the SRN has played a major role in shaping and delivering include:

• 2016-2018: Support for both Scottish Rural Parliaments to date.
• 2016: LEADER conference
• 2018: Rural Youth Festival of Ideas
• 2018: Rural Transport Convention
• 2018: The 11^th OECD Rural Development Conference
• 2019: LEADER Cooperation Workshop with the Republic of Ireland
• 2019: Rural Enterprise Futures Conference

As this list indicates, the relevant events are varied in scope. Restricted to SRN events, the number of attendees over the period has been:

• 2016: 401
• 2018: Approximately 600, including 400 at the OECD event.
• 2019: 247

It was emphasised by participants that the SRN had a useful capacity to feed into event development, undertake the complex logistics of organisation and promote events to their network. As an example of logistical support, participants mentioned the SRN's input to the 2016 Rural Parliament. It was emphasised that the SRN's work here permitted the primary organisers – the SRA – to focus on event content and workshops, with the SRN taking on the considerable logistical burden of event organisation, providing assistance to the planning of workshops and the development of outputs.

Participants in the evaluation also emphasised that, in the context of events, the SRN had contributed to the planning, organisation and facilitation of meetings and workshops, owing to their experience with rural policy. This has allowed them to play a useful role in developing event programmes, co-ordinating with speakers, identifying appropriate themes and facilitating discussions designed around maximum stakeholder impact. The following quotation indicates the kinds of activities event organisation involves, in relation to the 2019 Rural Enterprises Futures event mentioned above:

"[The SRN] worked with representatives of the LAGs and FLAGs^[15] chairs group. As part of the event working group they worked closely with them to develop the event programme, source venue, and assist with identifying speakers. Additionally, as there was a Minister speaking, they liaised with ministerial offices, produced briefing materials and wrote speech content."

(Anonymous Evaluation Participant)

Another example concerns the 11^th Organisation for Economic Co-Operation and Development (OECD) Rural Development Conference. Here, the SRN acted as a local partner for the OECD event. This involved developing the agenda, inviting speakers, organising the venue and managing the event and facilitating discussions.

Engagement with events has also been used to raise the profile of Scottish rural policy in a European context. For example, in 2019, the SRN co-ordinated the submission of 10 Scottish nominations to the Networx Rural Inspiration Awards, two of which made it to the final round. This work involved considerable co-ordination and organisation, and is a valuable example of how the network has helped to link Scottish rural policy with European rural policy more broadly. The form of this engagement, typically, involves quarterly meetings with ENRD colleagues to develop rural policy via workshops, which the SRN has contributed to. For example, recent developments around smart villages and short supply chains have received substantive contributions from SRN colleagues.

Another mechanism through which the SRN has contributed to stakeholder engagement is by directly using their network to feed into specific policy challenges. An example of this was noted by an evaluation participant:

"There was a consultation with communities who use rural airports regarding Air Departure Tax (ADT). [The SRN] were asked by the lead policy to publicise the consultation. They promoted the consultation through an article in the SRN newsletter and also through our social media channels. As the target audience was quite a specific sector and demographic. [The network] went directly to partners, like Crofting Federation Scotland, Scottish Island Federation, all the various LAGs that were in those physical locations, and asked them to share the messaging through their channels and to their networks in order to broaden engagement. The survey generated a lot of interest and as a result the response rate far exceed the expectations of the academics undertaking the analysis of the results and the lead policy area. The feedback [the network] received was that the results were superb, they had no expectation of the high volume of the responses that they would receive."

(Anonymous Evaluation Participant)

While this is an example of an ad hoc use of the network's capacity, it shows that the maintenance of a wide rural network may have considerable benefits for policy development. Another example of effective, but ad hoc work involving the network was with the Wester Ross community. Here, the SRN was approached by the Wester Ross Biosphere group with a view to improving the area's capacity as a tourist destination. The SRN worked with the group to design and facilitate a workshop, the outputs of which will be used to inform a community action plan to maximise the area's natural assets and heritage to increase sustainable tourist engagement within the area. While, in this particular case, the benefits may have been somewhat lessened in light of the Covid-19 pandemic this is a useful example of how the SRN can apply their skills to pursue Scottish Government objectives while supporting community led development.

Key Finding: Overall, the SRN's capacity to provide ad hoc event support in this way was generally perceived as useful, and they have contributed to a valuable programme of events. However, it should be noted that the scheduling of specific events is largely driven by stakeholders outside of the SRN, rather than necessarily reflecting an agenda directed by the SRN. Overall, however, it can also be noted that these sorts of ad hoc forms of support, where the agenda and purpose of the activities is largely driven externally, may be less effective in terms of generating measurable changes than a programme of events designed by the network themselves. While ad hoc responses are valuable and are likely to always be necessary, there may also be potential in any future network for a more regular process of consulting and engaging with rural communities in line with overall strategic goals.

SRN Newsletter

Since 2014, the SRN has provided a weekly e-newsletter. The newsletter provides information on rural news, funding opportunities and information about ongoing projects. The mailing list has grown from an initial membership of 700 in 2014 to 1, 812 at the most recent count (October 2020). The focus of the newsletter is on rural news, funding opportunities, consultations and events information. Evaluation participants emphasised that the newsletter sought to share best practice and inspiring examples of rural development while promoting good news stories, upcoming events and sources of funding. The SRN also produce a LEADER newsletter, which currently has 266 subscribers. While it may, in principle, be valuable that this communication network has been developed, there are broader questions about the 'added value' of this newsletter, in a context where there are multiple sources of information on the topics above e.g. ARE Comms, the Farm Advisory Service, Scottish Rural Action. Pointing out the crowded landscape is not intended to diminish the efforts of the SRN here, but simply to highlight the issue for policymakers making decisions in the future on this matter.

Case Studies

The SRN has produced a range of filmed and written case studies, often accompanied by text and images, which document successful examples of community led rural development. As of June 2020, there were 50 case studies available on the SRN website, 24 of which included short films produced by the SRN. Since 2014, the number of these produced each year has fluctuated from a maximum of 23 to a minimum of 5. The case studies consider a wide range of relevant rural topics. Recent examples include videos exploring examples of community led rural housing development, a recent visit of Swedish fisherman to a LEADER project and a video documenting the recent 'Smart Villages' event in Stirlingshire.

On the SRN website, the focus of case studies are somewhat weighted towards examples of LEADER funding. Of these case studies, the majority – 27 – were based on LEADER funded projects. The next most common subject of case studies related to the Food, Processing, Marketing and Cooperation grant, with seven case studies. As was observed by the evaluation participants, this could reflect both the fact that LEADER had specifically sought out this sort of help, while other schemes may have been less forthcoming, and the fact that LEADER projects are very amenable to the case study format and provide interesting content. This is discussed further in chapter 4.

The importance of these case studies was emphasised among the evaluation respondents. Among other things, they helped to make developments in rural Scotland visible and pertinent in high profile contexts:

"Having a body of case studies is really helpful for Ministers. If Ministers are already going out somewhere to do something…is there some case study of best practice out there that they can go and visit, and enrich the picture of rural Scotland?"

(Anonymous evaluation participant).

Case studies were also perceived to potentially provide inspiration and information to those considering similar projects. They were also reported to have additional value for demonstrating the value of the SRDP and had been praised by the European Commission to this effect:

"The [EU] Commission love them, and they quite often share them across Europe….They see it as a real positive and are very keen that [these are there] for communicating the programme in that digestible, easy to use format."

(Anonymous evaluation participant)

In some cases, stakeholders had specifically sought out the SRN to develop case studies. This was reported with regard to many of the LEADER case studies, as well as the Highland and Moray Fisheries Local Action Group, who requested that the SRN film a case study of the Ullapool Jetty project they had developed.^[16]

The soliciting of these case studies is a useful indication that these are perceived as adding value to local projects, and it is valuable to have visual examples of some of the SRDP schemes for both communication and promotion. At the same time, while the case studies may well have influenced applications for funding and rural projects, it is not straightforward to demonstrate this. While this may simply reflect the limited scope of this evaluation, it appears that there is limited evidence of specific initiatives to produce case studies with a view to, for instance, encouraging applications for specific funding opportunities or the funding of work in a specific area, with accompanying monitoring to measure the effects.

Videos

The SRN also maintains a Vimeo channel, which hosts a total of 159 videos.^[17] The videos hosted here are the case studies described above, alongside a range of videos documenting LEADER events, highlights from events that the SRN participated in, short interviews/vox pops and information about projects. Between 2015 and August 2020, SRN videos have been viewed 23, 217 times and finished 6,376 times. The most popular videos, when last checked, were the Scottish LEADER programme (3,008 views), the Scottish Rural Development Programme (2,218 views) and an animation about LEADER (2,025 views).

SRN Website

The SRN website plays an important role in the network's communications and stakeholder engagement. It hosts a range of useful resources and case studies for those seeking to develop rural projects. It also played an important role in hosting the LEADER Expression of Interest forms at the beginning of the 2014-2020 SRDP. These forms constituted the first part of the LEADER application process, without which initial enquiries could not be made. In 2016, 2017 and 2018, this resulted in over 1000 LEADER Expression of Interest (EoI) forms being submitted (1,520, 1,270 and 1,316, respectively). The number sharply dropped in 2019, to an estimated 292 over the course of the year. However, this is likely to reflect the natural decline in applications as the current LEADER funding cycle winds down.

Chart description:

This graph displays the website sessions and unique visits to the SRN website from 2016 to 2020 (up until October). Website sessions increase from 22, 036 in 2016 to a peak of 79, 326 in 2019 and a further 73, 326 in 2020 (although this may reflect the incomplete data for this year). Unique visits increase from a base of zero in 2016, to 31, 110 in 2017 to a peak of 62, 465 in 2020.

The development of sessions, unique visits, and use of the funding search portal over time can be seen in Figure i. As this demonstrates, website traffic in all cases has tended to trend upwards over the period as a whole (with the exception of the most recent year, although this is likely to reflect the fact that the year is not yet complete). This indicates both an ongoing and potentially increasing need for these resources, while evidencing the SRN's effectiveness in engaging their user base. So far in 2020, there have been 73, 326 website sessions as of October 2020, indicating this demand is being maintained.

Social Media

The SRN has also maintained an active Twitter profile, Facebook page and Instagram account In all cases, the trends indicate growth and, by proxy, an increase in demand from the public for SRN services (see Figure ii). As of the most recent data in June 2020, there are 2, 666 Facebook likes (+586), 6, 718 twitter followers (+298) and 1, 065 Instagram followers (+491).

SRN Funding Search

The SRN website also provides a funding search engine. This is delivered by the Scottish Council of Voluntary Organisations (SCVO). Based on the numbers using the service, this appears to be valuable. The service launched in 2017, and its use over time can be seen in Figure iii. While use has not increased substantially since 2017, the consistency of use indicates that demand is being maintained. The funding search came online in 2016 (hence the low use in this year and 2015). The use of this service indicates that, irrespective of future policy decisions, this may be a valuable resource to maintain.

Key Finding: The SRN has, as demonstrated above, developed their communication capacities considerably and now offer a wide range of outputs to support rural development. In particular, the website and the funding search facility both show high levels of engagement and have grown over time.

Have These Activities Contributed to the Goals of the Network?

It is clear that the SRN has contributed to rural stakeholder engagement and the provision of information to relevant parties. As discussed above, it is not possible to infer clear and measurable impacts that have followed on from these approaches. At the same time, the consistent increases in the social media presence, website visits, mailing lists, and other engagement should provide evidence that, at the least, there is a demand for these services.

As many of the events involving the SRN involve collaboration with other organisations, the SRN's contributions cannot be assumed to be uniform. Numerous participants emphasised, however, that the logistical support and experience that the SRN could offer to smaller, grassroots organisations could be extremely valuable in supporting rural actors.

One challenge that becomes apparent from looking at the range of SRN activities is that they are operating with an extremely wide remit, i.e. 'rural issues', which has, in practice, included a wide range of policy areas. In this sense, a concern is that that there will be far more opportunities for engaging in rural issues than there is capacity to do so. However, this also creates potential opportunities to develop more innovative ways of working and leveraging digital technology to expand the capacity of actors within the network, including but not limited to the network support unit.

Finally, the above findings suggest two clear senses in which the SRN has added value. First, the SRN has acted as a platform for collaboration between rural actors. This is most apparent in the above instances where the SRN has facilitated events with rural actors and Scottish policy officials, such as the Rural Enterprises Future event in 2019. Second, it is clear from the discussion above that the SRN can play a useful role in developing narratives from rural Scotland, specifically in terms of the effects and consequences of rural policy and the support it can provide for community led projects. Particularly when combined with the funding information provided by the SRN, this may act as a powerful tool in encouraging ongoing development work. However, in both cases, more explicit goals for the network and a clearer strategic plan would make it easier to identify the impacts of this engagement on the broader goals of the rural development programme.

In the event that a network structure is used in the pursuit of rural development goals going forward, there are two primary steps that would contribute to the broader goal of delivering the rural network and making it visible and accessible to others. First, any future network should engage in steps to make the broader rural 'network' more visible and easier to define. This will support future monitoring and evaluation, as well as making the network easier to work with and to understand for those working internally to Scottish Government and those working with it from outside. As detailed in the recommendations section, one approach to this would be adopting a formal network membership structure that would allow organisations and individuals to 'sign up' to become members. This would not involve a financial cost, but it would make it far easier to determine the extent, range and nature of those who benefit from the network.

This would have various benefits. It would mean being able to clearly demonstrate demand for the network, it would provide a clear view of the nature of the priorities and needs of members and it would make it possible to make inferences about the nature of and changes to this demand over time. It would also provide a format for more regular consultation with rural stakeholders for the development of priorities and feeding into the policy process, as well as identifying rural priorities.

Second, a future network should develop an online map that makes it possible to view the geographical distribution of rural community groups that have worked with the network and are actively supporting the network's goals. This would mean, at a glance, those external to the Scottish Government and internal to it would be able to see the extent and range of network supported projects. Such a service would be maintained by the network, but allow external groups and organisations to register their presence so that those local to them would have the opportunity to learn about their activity via the network.

Third, as will be discussed in the next chapter and conclusion, there may be value in clarifying any future network's intervention logic to reflect their success in relation to both developing narratives about rural Scotland and in facilitating rural collaboration.