Rural affairs, food and environment research programme 2016 to 2022: evaluation report

4. Findings

4.1. Overview

As part of the findings, Section 4.2 provides an overview of the Programme's performance, beginning with an assessment of the Programme against the vision in the Research Strategy. It moves on to consider the extent to which the Programme has helped deal with challenges faced by the Scottish Government. It then looks at the contribution of the Programme towards national outcomes. Section 4.3 reports on the Programme's outputs, including ways of working. Section 4.4 goes beyond outputs to discuss the impacts of the Programme for Scotland, including an assessment of the estimated economic impact.

Finally, Section 4.5 reports on the resources put into the Programme (financial and human resources) as well as the delivery model and procurement arrangements.

4.2. Assessment of the Programme against the vision and principles in the Research Strategy

4.2.1. Delivery of the research in line with the vision

This section draws on information gathered and analysed for evaluation question I1: Has the research been delivered in line with the vision?

The vision (as extracted from the Strategy (Scottish Government, 2015, p3)) states:

We will have delivered research that is relevant, respected and responsive

to Scotland's communities, its people and to the rural economy in 2021 by:

• Having a Strategic Research Programme, which has interdisciplinarity at its core and has a single clear identity that is recognised nationally and internationally for its excellent science.
• Demonstrating increased levels of collaboration with researchers from other institutions through leverage of our investment in research.
• Having evidence of increasing innovation activity associated with the Programme with a range of non-commercial and commercial funders.
• Ensuring we demonstrate the impact of our research undertaken on the communities, businesses, public sector and the economy of Scotland in a clear and measurable way.
• Making the significant data holdings we support through our funding more visible and accessible.

Creating Centres of Expertise at points of significant demand in the system for the translation of scientific understanding from across Scotland into solutions to critical questions that will emerge over time.

Interdisciplinarity of the SRP

Of the 1,244 policy outputs produced by the Programme, 56% (691) were linked to at least one of the Programme's three themes, of which 85% (588 outputs) were related to one theme, 12% (82) linked to two themes and 2% (21) covered all three themes. This demonstrates a limited level of interdisciplinary research. However, some of the 553 policy outputs which did not have any theme information may also have been interdisciplinary.

Feedback from interviewees reinforces the view that there was interdisciplinary working during the 2016-22 Programme, with one interviewee noting that "when research is established, there's a push toward interdisciplinarity". Another reported that the Programme performed well at finding relationships between different types of work and bringing expertise together. However, there was a desire for more opportunities to bring complementary areas of expertise together, with communication within the same theme seen as easier and more common than cross theme working, with one interviewee noting that "there was less communication across the themes".

Programme identity

During engagement, interviewees were asked about their views on the reputation of the Programme. There was a mixed response. Interviewees spoke about RESAS in the context of other funding Programmes, such as UKRI or EU research. Four interviewees explicitly stated that the Programme had a good reputation. One interviewee said that the SEFARI portal was a great positive for the Programme's reputation, as it brought the various institutes together under one portal and brand, therefore allowing the Programme to build a coherent identity. However, this same interviewee did say they believed the UKRI had a better reputation but did not expand on why. Another interviewee agreed that SEFARI made the Programme less fragmented than it had previously been.

None of the interviewees spoke negatively about the Programme's reputation, but said instead that numerous stakeholders were more aware of specific institutions than the overall Programme, evidencing potentially a limited visibility of the Programme. For example, an interviewee noted that if a stakeholder carried out Programme funded work with one of the institutions they tended to remember the institution and not the Programme. Interviewees also tended to speak more about the reputation of their own organisation than the overall funding Programme. One interviewee said that they would have liked the Scottish Government to promote their funding of strategic research more prominently.

Demonstrating increased levels of collaboration

Collaborative working was apparent, with 30 collaborations between CoEs, 979 instances of CoEs working with other organisations, and 2,329 undertakings where MRPs collaborated with others. Examples included work on willow population genetics, which involved a collaboration between RBGE, a private estate and two non-governmental organisations, and a project on antimicrobials which was undertaken by Hutton and involved two universities. Trends in collaborations depend on the type of organisation involved. The total annual amount spent on subcontracts by CoEs increased year-on-year except for a dip in 2020-2021.

The annual number of MRP collaborations was more variable, ranging from a low of 356 in 2016/17, up to a high of 414 in 2017/18. Numbers then went down and up again, before dropping to 375 collaborations in 2021/22. Collaborations therefore occurred, but not necessarily in increasing numbers as the Programme progressed.

Interviewees reported collaborations, noting that "there was a lot of synergy amongst the projects" when they were aligned within a research deliverable, and that "scientists were encouraged to work together". Such collaborations between the researchers resulted in those in director level roles also working together. Collaborations led to studies with a global reach, for example, work on identifying greenhouse gas emissions from global aquaculture involved SRUC, the Food and Agriculture Organisation of the UN (FAO) and others, and was published in the scientific journal Nature^[9]. Collaborations enabled research to go beyond the lab and connect with industry. For instance, researchers working on integrated pest management practices collaborated with the National Farmers Union and the Voluntary Initiative (an industry led Programme) to develop a pest management plan for use by crop producers^[10].

However, not all interviewees attributed collaborations to the Programme. One expressed the view that cohesion was "due to a long history of the individual scientists working together", thus suggesting that existing relationships are equally as important for collaborations as a Programme that facilitates working together. Another felt that the Programme "wasn't quite as cross institutional as it had been before", although they were positive about collaborations where these occurred.

Having evidence of increasing innovation activity

£1.0 million of the Programme's £279 million of funding was spent directly on innovation – equating to just under 0.4% of total spend. SRUC employed 1.2 FTEs (full-time equivalents) in 2017/18 within innovation, with 0.7 FTEs on technical staff and 0.5 FTEs on natural scientists. Across the 2016-22 programme, £500,000 was spent on innovation within 2016/17, £400,000 during 2017/18, and £100,000 in 2018/19. However, there is limited quantitative or qualitative information within annual reporting from MRPs and COEs on how this funding was utilised. EPIC reported holding an innovation summit to explore the role of organisational culture with respect to knowledge seeking and sharing behaviour. This aimed to codify a framework to ensure sustainable, resilient and efficient science-policy communication.

Wider innovation activity is assumed to capture research collaborations and the involvement of organisations outside the Programme in funding, contributing to, and/or taking up research. Income from industry brought in over £70 million, as companies and other organisations took advantage of commercial opportunities resulting from research and innovation under the Programme. An illustrative example of this is the James Hutton Institute obtaining royalty income from soft fruit cultivars. This was achieved through their soft fruit breeding Programme covering both raspberry and blackcurrant, which is important for both the institute but also internationally. The breeding Programme allows exchange with researchers around the world to enable varieties that are bred with the traits necessary to meet the demands of the soft fruit industry and the changing environment. This demonstrates innovation from the Programme feeding directly into production by industry. The issuing of patents is also relevant to innovation activity, with 39 patents granted over the course of the 2016-22 Programme. The majority related to Moredun.

Non-commercial funding was also leveraged by the Programme, with money from research councils, the European Union and Defra all obtained. For instance, a project on the control of black leg (which affects potatoes) attracted funding from two research councils and Defra, in addition to Scottish Government input^[11].

Demonstrating the impact of research undertaken

The main arena for showcasing research was SEFARI. Interviewees were particularly positive about the SEFARI Gateway, noting that "the development of SEFARI gateway was a really positive highlight…". The Gateway allowed effective knowledge exchange between various stakeholders, helping to bring them together, collate evidence and inform policy. Interviewees commented that "engagement with the gateway has come on greatly". Twitter was also used to disseminate information. Anecdotal evidence noted that industry contacts saw outputs on SEFARI and then contacted researchers to follow-up, demonstrating the relevance of the research to the economy.

Individual projects also undertook their own knowledge exchange and dissemination. SRUC wrote a report on Scotland's Agritech, Animal Health & Aquaculture (AAA) outlining its capacity, capabilities and potential. This led to the organisation of an international conference. In another example, the Moredun Institute undertook work to analyse the impact of medicated grit on the health of red grouse. This project led to dialogue between grouse moor owners, managers, and gamekeepers that resulted in many more Scottish Estates improving their practices and therefore creating a more sustainable sector.

Work by CREW created the open access tool 'Dynamic Coast', which offers reports and maps of the entire Scottish coastline. The website achieved 13,000 hits from 8,000 users globally by March 2019. This tool has been used to support numerous policies and plans for organisation ranging from the Scottish Government to local councils. The James Hutton Institute also developed a web-based platform to facilitate access to data generated as part of a long-term sustainable cropping experiment at Balruddery Farm. The platform was regularly accessed by numerous stakeholders from RESAS and EU funded projects and numerous PhDs students.

Making data holdings more visible and accessible

Responses from interviewees suggested that the underpinning capacity was not promoted as strongly as it could have been. However, there was evidence of data sets being used by stakeholders outside of the Programme. For example, the James Hutton Institute responded to two requests in 2021/22 for samples from the National Soils Archive. These were used for investigating the distribution of microplastics in Scottish soils and identifying sulphur isotopes for use when determining the provenance of food stuffs^[12]. There also appears to be increased usage of Hutton's live insect collection service. Moredun's collections of pathogens, host tissues and related materials also continued to be used with 32 papers utilising the collections in 2019/20 alone^[13].

Creating Centres of Expertise at points of significant demand

The Plant Health Centre (PHC) was set up in 2018 to improve resilience to plant health threats in Scotland by connecting science to application to inform policy, planning, responses and solutions^[14]. Within the PHC's first 18 months, five of its projects have directly informed policy, showing a demand for the research^[15]. Contracted projects included a study on the impacts of withdrawing the molluscicide metaldehyde, which fed directly into Scottish Government decision making^{Error! Bookmark not defined.}. The PHC also commissioned work to assess critical biosecurity risks to Scotland. This included researching risks associated with non-specialist (e.g. supermarket) and online horticultural sales^[16]. The research was needed because the sector is important but understudied and difficult to reach. The work resulted in recommendations and best practice options to reduce risks to plant health.

4.2.2. Extent to which the Programme has helped deal with challenges

This section reports evidence collated for evaluation question I2: To what extent has the Programme helped deal with challenges faced by the Scottish Government?' It considers the spread of funding and interviewee opinions on what was funded and what they thought was missing. It then discusses how the Programme has helped deal with challenges, through identifying examples where research has resulted in community benefits, contributed to net zero, fed into policy and resulted in scientific benefits.

Spread of funding and interviewee opinions

Of the total £279 million funding for the Programme, the Scottish Government allocated around 69% (£190 million)^[17] to the Strategic Research Programme (SRP), 16% (£43 million) to Underpinning Capacity, 13% (£35 million) to CoEs, 3% (£8 million) to knowledge exchange and management and less than 1% (£1 million) to innovation. Respondents from MRPs tended to suggest that in their view, the Programme funded the right balance of topics across the three themes of food, health and wellbeing; productive and sustainable land management and rural economies; and natural assets. It was noted by one interviewee that the land management theme "was the same size as the other themes combined, in terms of cost and personnel". This is a view reflected by data on Programme outputs. This theme had a total of 738 scientific outputs across the original Programme and extension year, compared to 742 outputs for the other two themes combined.

One MRP stakeholder commented that "in terms of the scope and for the amount of money available, it [the SRP] covered most of what was required", whilst another noted they did not particularly like the titles of the themes, "because they were too broad", which could lead to misinterpretation of what kind of work could be covered. However, it was also seen as a positive that the Programme could accommodate a wide range of research. It was acknowledged that "on the day the projects were funded", the balance was "absolutely" correct but that the determination of the correct balance changed over the duration of the 2016-22 Programme. For example, one interviewee highlighted that the direction of research priorities would be adapted, noting that "even as soon as two years ago we were not thinking of the food insecurities that we are seeing now". As the Programme progressed, the initial priorities became less relevant to the issues highlighted in emerging policy areas. As such, whilst the Programme provided positive contributions to policy areas covered by the Rural Affairs, Food and Environment portfolio at the Programme's outset, this might have waned as the Programme progressed and new challenges arose. This was echoed by another interviewee who added that the Scottish Government has included an area (air quality) in the 2022-27 Programme that they thought was missing, as well as reduced emphasis in other areas, such as human nutrition. Biodiversity was also suggested as being "an obvious hole in the [2016-22] Programme", whilst another interviewee said more research could have been funded on renewable energy, although it was noted that there was now action to grow this area.

Community benefit

SRP-funded research has informed decision-making on areas such as flooding, energy and adaptation, as well as encouraging engagement and providing recommendations for particular sectors e.g. the farming community. CXC research has informed a number of climate-related strategies. CXC helped to formulate and implement the Scottish Government's property flood resilience action plan. CXC also produced three reports on district heating, with some work cited in a Heat Networks Delivery Plan. CXC also worked to support the Scottish Government to consider the impact of changes in the energy policy landscape on domestic consumers. A report on the potential impact of regulation in the transport system on certain groups was presented at a Scottish Transport conference, and shared with Edinburgh City Council and the South East of Scotland Transport Partnership. CXC produced the first carbon and greenhouse gas balance of restored bog, and enhanced engagement with the local community through collaboration.

EPIC research activities have had direct benefit for stakeholders working with livestock. The CoE has participated in three disease preparedness exercises and engaged in stakeholder knowledge exchange events such as the Royal Highland Shows, with industry professionals, scientists, vets and farmers. Veterinary risk assessments were developed or updated for a number of livestock diseases, to enable rapid policy decisions in the event of outbreaks.

CREW work has informed a number of water-related strategies. This has included assessing the long-term impacts of flooding on communities. CREW has a Sustainable Rural Communities theme which supports private water supplies and rural water management. CREW worked with stakeholders to identify priority questions for the Catchment Management Modelling Platform to answer. CREW produced an open-access web-based tool Dynamic Coast, which allows identification of vulnerable coastline assets. The project contributed to evidence leading to funding of £12 million for coastal change adaptation. Other examples include the Valuing Your Soils projects and Rural Sustainable Drainage Systems practical 'design and build' guide for farmers and landowners; a review of climate change and risk to water resources for water industries; facilitating discussions in the development of the North Glasgow Integrated Water Management System; ongoing benefits from a CREW surface water flood forecasting project, and a Handbook of Catchment Management.

The PHC researched and reported on potato cyst nematode control, producing recommendations for industry and research, thereby providing potential benefits for growers. The Scottish Government has committed £2.3 million over five years to fund a project that will implement the recommendations from the potato cyst nematode working group. PHC has jointly run Scotland's Plant Health Conference.

The SEFARI Gateway education group contributed to the provision of education during the Covid-19 outbreak, collating freely available online educational resources for teachers and home-schooling parents.

Net Zero Contribution

Research by CXC under the Programme has informed various government strategies: the Heat in Buildings Strategy; the Energy Strategy; Scotland's Energy Efficiency Programme; the Heat Network Delivery Plan; the Clean Air Zone policy; the 2018 Climate Change Plan; and Net Zero Nation, the public engagement strategy for climate change. Research on ultra-low-emission vehicles was cited in the Just Transition Commission's final report. A report on agricultural mitigation featured in a Scottish Government consultation and in the 2021 Climate Change Plan Monitoring Report. Work on property flood resilience was cited in the 2021 Climate Change Plan Monitoring Report. A CXC report on the contribution of investment in infrastructure to emissions reduction was widely cited, including in Scottish Government evidence to a parliamentary committee and in a March 2021 budget review. Work on the Climate Challenge Fund was welcomed by the Cabinet Secretary for Environment, Climate Change and Land Reform at the Climate Challenge Fund Annual Gathering 2020. CXC work has included producing a Scottish Transport and Air Pollution model, which can be used to analyse alternative transport scenarios to meet goals related to carbon emissions, air quality, etc^[18] and so progress towards net zero.

Scientific benefit

MRP research advances have included: an innovative method for monitoring and conserving genetic diversity; a suite of approaches for soil health; promotion of conservation in multi-use landscapes; informing woodland expansion and soil carbon sequestration; supporting peatland restoration; informing farming to achieve net zero; integrated pest management for reducing pesticide use; work on pre-natal food preference for health; and long-term work supporting UK folate fortification. New potato cultivars and barley products were developed, and research on pulses and hemp delivered environmental and commercial benefits. Livestock advances included new methods for control of adenocarcinoma in sheep, performance indicators of cattle informing reductions in greenhouse gas emissions, and Qualitative Behaviour Assessment delivering improved animal welfare.

A CREW project helped to inform policy makers on implementing a prevention-led approach to improve drinking water quality. Other research outputs included recommendations for the development of Scotland's shellfish water monitoring Programme; an assessment of the impact of climate change on the water quality of Scottish standing waters; and policy briefs on topics including antimicrobial resistance and the potential influence of beaver activity on the natural functioning of rivers and streams in Scotland.

EPIC has produced mathematical models of the spread of African Swine Fever (ASF), the potential for Bluetongue virus spread in Scotland, the utility of a new diagnostic test to reduce sheep scab spread, and Chronic Wasting Disease spread in deer. Economic analyses identified factors affecting Scottish farmers' uptake of animal health measures, consequences of animals persistently infected with Bluetongue, indirect costs of foot and mouth disease and African Swine Fever, and economic impacts of the avian influenza housing order. Stakeholder workshops identified incentives to improve disease control on Scottish dairy farms.

Research by PHC research has improved knowledge of Xylella fastidiosa vector ecology and also understanding of how those involved in habitat restoration or creation view the risks posed by plant pests and pathogens.

Policy contribution

Policy contributions were identified as part of Programme reporting. In addition, interviewees were asked if they were aware of any instances where research under the Programme has underpinned policy decisions or led to the creation of strategies. Numerous examples were given ranging in topic from peatland restoration to flood prevention.

Five stakeholders from the Scottish Government answered this question. Two interviewees mentioned contributions to research related to the greenhouse gas inventory. Interviewees mentioned general topic areas or organisations without giving specifics, such as the Common Agricultural Policy, Food Standards Scotland and peatlands restoration. They were often aware of the work but had not been directly involved in it themselves. One interviewee gave two clear examples:

• Research into coastal erosion and flooding over time, specifically how large an impact there would be from climate change. This led to the creation of models that were used in the creation of local coastal management plans by local authorities; and
• The monitoring of sewage to gauge the amount of viral Covid matter which was undertaken with SEPA and Scottish Water. This helped monitor the spread of Covid in the community and fed into government research and policy.

These varied examples highlight how research under the Programme has underpinned policy decisions and led to the creation of strategies.

Stakeholders from the MRPs were asked the same question as those from the Scottish Government. A total of 17 interviewees answered this question. All interviewees stated that they were aware of instances where it had taken place. Of those 17, five stated that they were aware of it but could not give concrete examples. There was a reticence from certain interviewees to directly ascribe research to specific policy changes. For example, one interviewee stated that Scotland had very progressive climate change policies which was due to confidence in the science from the underpinning research particularly with regards to peatland restoration and planting trees. However, they could not point to specific projects leading directly into policy changes. Two stakeholders did point to specific work undertaken on rural depopulation that led to the creation of a map of Scotland showing where this was taking place. This ultimately fed into the creation of the National Island Plan by the Scottish Government's Island Policy team. One interviewee spoke about outputs that were used by industry, such as agro-ecological land management practices that aided farm systems in becoming carbon neutral and climate positive. It must be noted that the majority of stakeholders could only speak for their own institutes and could not speak for the overall Programme.

A total of eight stakeholders from the CoEs and the ARPs were asked this question. One interviewee answered no, as they said they were at the basic end of the research that would not have required a policy change. Another interviewee found it difficult to attribute specific pieces of their research to a policy outcome but they were aware of plenty instances when their research was referenced by Scottish Government strategy papers and policy commitments. This was repeated by another stakeholder who said that their research informed the development of policy and added concrete evidence but did not directly shape it. Other stakeholders gave concrete examples, such as the Dynamic Coast project and the covid sewage analysis which were also mentioned by interviewees from the MRPs. Three stakeholders who were aware of research stated that the best examples could be found in the published annual reports or on their respective websites.

Many stakeholders were reluctant to directly ascribe their research to changes in government policy and instead stressed that they provided the underlying evidence for the government to use. Overall though, it is clear that there are numerous examples where research has informed policy decisions particularly in flooding and monitoring of Covid-19 levels in the community. This demonstrates that the Programme has helped to deal with many of the policy challenges faced by the Rural Affairs, Food and Environment Portfolio.

4.2.3. Extent to which the Programme has contributed towards national outcomes identified in the Strategy

This section reports on evaluation question I3: Which of the national outcomes identified in the Strategy has the Programme contributed towards, and to what extent? The following box sets out the relevant national outcomes. The text then discusses the extent to which the Programme has contributed towards these.

National outcomes that the Programme aims to contribute towards (as extracted from the Strategy (Scottish Government, 2015, p4))

• We value and enjoy our built and natural environment and protect it and enhance it for future generations.
• We reduce the local and global environmental impact of our consumption and production.
• We are better educated, more skilled and successful, renowned for our research and innovation.

Valuing and protecting the natural environment

Table 4-1 shows the breakdown of the 1,480 scientific outputs produced according to the three Programme themes.

Table 4-1: Summary of scientific outputs by theme and underpinning capacity

Outputs by theme	2016/17	2017/18	2018/19	2019/20	2020/21	2021/22	Total
Natural assets	37	89	116	130	112	50	534
Productive and sustainable land management	20	136	142	203	118	119	738
Food, health and wellbeing	3	40	33	55	40	37	208

Around 36% (534) of scientific outputs produced over the six years related to the natural assets theme, showing that around a third of research linked to valuing and protecting the natural environment. Another approximately 50% (738) of outputs related to productive and sustainable land management, which is also relevant. Some examples of work against this outcome include a 2019-21 project to support woodland planting which SEFARI Gateway co-designed with the Loch Lomond and The Trossachs National Park Authority. Another is a SEFARI report which provided recommendations for the new Environmental Standards Scotland body in defining environmental regulation data needs. A SEFARI Gateway project with the Water Industry Commission for Scotland (WICS) and Scotland's Hydronation Initiative, in collaboration with CREW, provided insight on frameworks to value various different benefits of water to Scotland. Climate-themed work notably informed a number of Scottish Government strategies. A considerable proportion of work under the Programme therefore contributed towards the outcome of valuing and protecting the natural environment.

Reducing the local and global impact of consumption and production

Linked to consumption and production is the Programme's theme of food, health and wellbeing. However, the food, health and wellbeing theme was only responsible for 14% (208) of scientific outputs, suggesting that less research was carried out in this area than others. This reflects a comment by an interviewee, who highlighted the Programme's apparent inability to tackle food equality and poverty. Despite this view, some examples of work contributing to food security were identified: a model of international nutrient trade was developed, which showed that removing obstacles to trade, increases food and nutrient security. It was adapted in an Economic and Social Research Council Rapid Response grant to simulate the impact of Covid-19 on global food trade. Initial work was carried out to identify parts of Scotland with poor provision of retail services, to assess availability of healthy food. Research also improved understanding of local and locality-identified foods.

The productive and sustainable land management theme is also of relevance for the national outcome of reducing the impacts of consumption and production. As already noted, this theme accounted for 50% (738) scientific outputs, suggesting that nearly half of the outputs developed linked in some way to land management and improvements to this. Examples include work that supports: achieving net zero in farming, conservation in multi-use landscapes, woodland expansion and soil carbon sequestration; peatland restoration; and integrated pest management for reducing pesticide use. The Programme did therefore contribute to the national outcome on consumption and production, but perhaps more on the production side (linked to the land management theme) than the consumption part (linked to the food, health and wellbeing theme).

Being better educated, skilled and successful, and renowned for research

The Programme has contributed directly towards upskilling and educating individuals through fully funding 160.5 full time and four part-time PhD students (with additional PhDs partially supported with Programme funding). The SEFARI Gateway has also contributed to this outcome, with the organisation of knowledge exchange, training and development events, as well as the publication of case studies and research summaries to disseminate the knowledge gained.

Participation in Government advisory group bodies, inquiries or committees by researchers could be viewed as indications of the relevance and importance of the research commissioned under the Programme, as well as the expertise of individuals. In total, there were 833 representations by 419 researchers, suggesting each researcher is a member of two groups on average. Groups reported are varied and reflect the different themes covered by the SRP, for example, the Scottish Government Flooding Policy Stakeholders Group, the UK HSE Pesticides Forum and the European Plant Science Organisation Committee. Researchers also contributed to non-government advisory groups, with a total of 1,325 representations by 573 individual researchers. Examples included:

• Natural assets theme: Edinburgh Living Landscape Project, Member of the International Peat Society Climate Change Panel and International Union for the Conservation of Nature (IUCN) Red Book Committee;
• Productive and sustainable land management and rural economies theme: Dundee Community Growing Working Group, European Association of Potato Research and International Science Advisory Board of the Southern African Centre for Infectious Disease Surveillance; and
• Food, health and wellbeing theme: Cairngorms Community Development Network, Quality Meat Scotland, and the European Food Standards Agency.

Peak involvement was in 2019/20 for both government and non-governmental groups. This suggests it was the work being carried out under the 2016-22 Programme that was of interest to the groups, since by the middle of the Programme, early projects would have been completed and outputs disseminated. The involvement of Programme researchers in expert groups that are unconnected to government shows that the knowledge and expertise of the researchers is valued and utilised outside the Programme, thereby demonstrating the reputation of the researchers. The number of representations show that these are not isolated incidences related to particular individuals, but a feature common to many researchers. The Programme is therefore thought to have made a considerable contribution towards this national outcome both in terms of increasing skills but also raising awareness outside the Programme of those undertaking research.

4.3. Outputs from the Programme

4.3.1. Overview

Section 4.3 presents the findings for evaluation question I5 (What 'ways of working' did the research programme enable?). More specifically, it presents the outputs generated under the programme, including the dissemination of these, based on the analysis of programme data and the evidence gathered during interviews.

4.3.2. Collaborations

Collaborations were identified as organisations working together on a piece of research. Recorded collaborations included: (1) collaborations between CoEs; (2) collaborations between CoEs and other organisations; and (3) collaborations between MRPs and other organisations. Numbers of interdisciplinary projects were also recorded.

Over the six years of Programme implementation, there were 30 collaborations between CoEs. CREW recorded the most collaborations (17), followed by EPIC (7) and CXC (6). PHC did not record any collaborations. It has to be noted however, that the organisation was formed part way through the Programme.

Figure 4-1 shows collaborations between CoEs and non-CoE organisations. The total number of collaborations in a year (unique collaborating organisations) was highest at 210 in 2016/17, and fell to its lowest at 116 in 2019/20, before increasing in 2020/21 and 2021/22.

MRP collaborations peaked in 2017/18 at 414, then declined in 2018/19, before rising again to 410 in 2020/21.

The number of interdisciplinary projects showed a year-on-year increase from 199 (2016/17) to 323 (2021/22), with a minor dip in 2018/19. Cross theme working therefore increased over the Programme's duration.

4.3.3. Publications for trade

Publications for trade include outputs such as articles for online journals and trade websites. In total, 500 publications for trade were recorded between 2016 and 2021, with an additional 107 publications recorded in the 2021/22 extension year.

The 2019/20 reporting year saw the greatest number, with 115 publications recorded, whereas the 2016/17 reporting year saw the fewest with only 41 publications recorded. Low numbers in 2016/17 may reflect the early stages of the Programme when research was getting underway. SRUC was the MRP with the greatest number of publications for trade in both the main 2016-21 cycle (223 publications), and the 2021/22 extension year (68 publications).

When studying the main themes, a significant number of publications focused on the 'Productive & sustainable land management and rural economies' theme, in particular the sub-theme of 'Livestock production, health, welfare & disease control', with this sub-theme discussed in 418 (84%) of the 500 recorded publications between 2016 and 2021. In contrast, the least discussed sub-themes were 'Healthy diets & dietary choices' and 'Water Resources & Flood Risk Management', discussed in one and five of the 500 recorded publications, respectively. This situation reflects interviewee comments, where it was noted that the land management theme effectively had the budget of the other two themes combined.

4.3.4. Consultancies for industry

A total of 212 instances where researchers at MRPs undertook consultancy work for industry were recorded between 2016 and 2021. The largest number of consultancies took place in 2017/18 (56, 26% of the total), and the lowest in 2021/22 (14, 7%). Data were recorded for BioSS, Hutton, MRI, RINH and SRUC. Organisations contracting researchers included private companies, industry associations, and special interest groups/organisations.

4.3.5. Funding from industry

The expertise and track record of researchers bring in revenue from commercial organisations who want to use the knowledge gained. Industry income is recorded as coming from contract research funded by industry; revenue from intellectual property; revenue from Innovate UK^[19]; and revenue from other commercial activities. Total revenue starts with a low of £9,470k in 2016/17, peaks at £13,316k in 2017/18 and then declines for two years. There is a slight increase in revenue for 2020/21, then a further increase for 2021/22 (the extension year).

The breakdown by MRP across the main Programme time period (2016-2021) is shown in Figure 4-2.

This illustrates that contract revenue was the greatest source of industry income for four MRPs (Hutton at £8.5m, Rowett at £2.0m, SRUC at £9.8m and BioSS at £330k). Examples include Hutton's work on breeding, germination and plant propagation, with the institute managing 12 such contracts in 2021/22 relating to soft fruit (e.g. Raspberry Breeding Consortium)^[20]. SRUC's contracts included a project for AHDB Beef and Lamb in relation to nutrition standards for improving beef production efficiency and work for an estate trust on fungicides and winter barley^[21].

For Moredun, revenue from other commercial activity (£5.8m) made up the largest amount of industry revenue, followed by contract research funded by industry. Where Moredun provides biological materials to another organisation, this is governed by a material transfer agreement, with charges for commercial companies. Examples of materials exchanged in the 2018/19 Programme year include herpesvirus to a South African company and a sheep viral disease to a Belgian public and animal health research body^[22].

Revenue from intellectual property was only relevant to Hutton and Moredun. RBGE received no industry income.

4.3.6. Patents

Three MRPs, namely Hutton, Moredun and Rowett, were granted patents over the course of the six year Programme. Of the 39 granted, Hutton received three, Moredun 35, and Rowett one patent, with this being a primary patent for 'Coated food product for controlled release and improved performance'. Patents were granted for research into topics including equine parasite detection, a vaccine against Teladorsagia circumcincta (a nematode parasite in sheep and goats) and for nanoparticle synthesis using plant extracts and viruses. However, only three of the 39 patents were outputs of the 2016-22 Programme, indicating that 36 of the patents resulted from research prior to 2016.

4.3.7. Licence agreements

Programme documentation provided data on the numbers of income generating licence agreements or material transfer agreements. Over 2016-22, there were 88 agreements. The highest number (19) was granted in 2020/21. Considering the split by MRP, 71 were made by Hutton (of which 62 related to the 2011-16 cycle whilst nine were from the 2016-22 cycle. The remaining 17 were made by Moredun of which nine linked to the 2011-16 cycle and eight to the current one.

4.3.8. Policy outputs

Over the five years of the main Programme (2016-21), there were 1,048 policy outputs. Most of these outputs (962 or 92%) drew on inputs from the 2016‑21 funding cycle. The greatest number of outputs (253 or 24%) was produced in 2020/21, reflecting work done in earlier years and the lead-in time needed to carry out research prior to producing outputs. The lowest number of outputs (181 or 17%) was in 2018/19, closely followed by 2016/17 (183 or 17%). Data on outputs were provided by 28 different institutions, although those that featured most commonly in the dataset were SRUC (331 outputs or 32% (including a combined entry with the University of Edinburgh) and Hutton (257 outputs or 24%).

Considering the extension year 2021/22, there were an additional 196 policy outputs (with SRUC responsible for 77). This took the overall total to 1,244 for the six years. For many (but not all) of the 1,244 policy outputs produced during the 2016-22 Programme, data are available on the themes to which they are relevant. Figure 4-3 illustrates the number of policy outputs related to each theme (in dark blue) and sub-theme (in lighter blue). The most common sub-theme for outputs was 'rural industries' within 'land management and rural economies', whilst the least common sub-theme was 'crop and grassland production and disease control', also within 'land management and rural economies'. Note that outputs may relate to more than one theme.

Figure 4-3: Number of policy outputs related to each theme across 2016-22

4.3.9. Scientific outputs

Over the full six years of the Programme (2016-22), there was a total of 1,480 scientific outputs against the three themes. In addition, there was 69 outputs that specifically related to Underpinning Capacity. Excluding peer-reviewed articles (see next sub-section), there were outputs ranging from conference proceedings to book chapters, and from datasets to disease alerts. 'Other outputs' (e.g. newsletters, films and magazine articles) formed the biggest overall category across the five year Programme, with a total of 728 outputs, followed by peer-reviewed conference proceedings with 454 and book chapters with 196.

4.3.10. Peer-reviewed publications

In total, 2,674 peer reviewed publications were produced. The highest annual number was 535 (2016), and the lowest was 337 (2021). Hutton consistently had the highest annual number of publications, ranging from 148 in 2021/22 to 283 in 2016/17.

4.3.11. Trends in measure of esteem

For the CoEs and MRPs, measures of esteem indicate the number of prestigious activities or titles associated with researchers in the organisation. Figure 4-4 shows the numbers of measures of esteem for each CoE/MRP by year.

Hutton had the highest numbers of measures of esteem except for in the 2017/18 year, when it was exceeded by BioSS (note that Hutton has the highest number of FTEs of all the MRPs so could be expected to have a higher number of measures of esteem). Total measures of esteem peaked at 362 in 2020/21 before declining slightly to 311 in 2021/22. This could reflect fewer activities taking place within the extension year due to the Covid-19 pandemic.

4.3.12. Spin-outs

A spin-out is a company that is formed to commercialise intellectual property where there is no existing business to take up the ideas generated, but there is potential for the innovation to feed into products and services^[23]. The total number of spin-outs remained static with five in existence each year from 2016-2022^[24]. Of the five spin-outs, three were produced by Moredun, one by Hutton, and one by Rowett. In the 2019/20 data, one Hutton spin-out was removed and replaced by a newly formed one (MycoNourish). However, neither this new spin-out, nor the other spin-outs recorded were outputs of the 2016-22 funding cycle.

4.3.13. PhD students

There were 1,862 full-time PhD students across the MRPs and CoEs during the 2016-2022 Programme, with 160.5 of these fully funded by the SRP. There were 95 part-time students during the same period, with four fully funded by the SRP. Note that PhDs that were not funded by the SRP are likely to have been partially supported by the Programme due to their location at MRPs and CoEs.

The number of PhD students generally declined with each reporting year. Figure 4-5 provides a breakdown of PhD students per annum.

4.3.14. Interviewee feedback on outputs and their dissemination

Interviewees were questioned on how their research had been disseminated and to what type of audiences. Interviewees generally cited a variety of dissemination channels and suggested there was a strong degree of interaction and uptake of research with audiences. Only two out of 30 interviewees believed further interaction with external stakeholders should have been achieved.

Interviewees generally stated that research outputs had been published to the general public via institution websites, information events (workshops, outreach buses, science festivals, lectures, etc.), and social media (e.g. YouTube, blogs). They also commonly cited via high profile academic journals as a source of wider dissemination to the academic community. Again, interviewees highlighted the SEFARI Gateway as one of the primary mechanisms for disseminating outputs to wider communities, all of which believed SEFARI to have worked effectively in sharing data, reports, and case studies. More targeted dissemination was undertaken for: industry organisations and associations with vested interest in research (e.g. regarding agriculture and livestock); NGOs and lobbying organisations capable of informing the Scottish Government on the development, value, and application of research; and internal stakeholders within the Scottish Government who can influence policy legislative decision making. Finally, interviewees also highlighted collaborations with international governments (e.g. Ireland and New Zealand). In this respect, stakeholders generally felt research outputs had underpinned policy decisions and been taken up by industry.

Interviewees were also questioned on the extent to which outputs had been utilised by those not directly involved in the research. Examples provided by the Scottish Government included (but are not limited to) Food Standards Scotland (FSS), as well as lobbying agencies and NGOs (agriculture and farming led groups who inform national policy makers, such as members of the National Farmers Union of Scotland (NFUS) on the transition to future condition agricultural support). Other examples include research on barley that has been picked up by the Scottish whisky industry, research on animal health in the farming and animal husbandry sectors resulting in prevented spread of disease. Another specific example includes the development of a new diagnostic test for bovine respiratory disease complex (use of this test enables faster diagnosis and intervention, reducing veterinary costs, and allowing better herd management). Welfare assessment methods for livestock were developed, leading to more humane methods of culling chickens.

Another interviewee spotlighted the creation of a mobile phone app used to monitor the wellbeing of farm animals in commercial use (which was underpinned by SRP research), which subsequently won the BBC's 2021 farming for the future award (although the visibility of the SRP may not have been recognised). SRP research has also underpinned technological developments, such as ultrasound screening for preclinical ovine pulmonary adenocarcinoma (OPA) which is now commonly offered by vets to clients in Scotland.

Interviewees also highlighted SRP research that has underpinned policy outcomes with public authorities and service provides. For example, Scottish Water, Nature Scot, NHS, Irish Government & New Zealand. Examples of research that affected policy outcomes include research related to climate change which informed the Scottish Government's climate change plan, veterinary risk assessment methods adopted by the Scottish Government, research into social and behavioural drivers of biosecurity practices (livestock) which underpinned published guidelines, and peatland restoration as informed by the Programme's research.

4.4. Impact to Scotland from the research undertaken

4.4.1. Value to Scotland and the Scottish Government from the Programme

This section considers evaluation question I6: What is the estimated economic impact of the Programme? It uses information from Programme documentation to estimate the value that Scotland and the Scottish Government have gained from the research. The monetary estimates should be seen as an indication of the economic impact, rather than a full estimate since many of the individual project impacts are not monetised here, for several reasons. Not all impacts of the Programme are immediately visible. Some may take several years to materialise, so are not currently apparent. It may also be difficult to determine to what extent an impact should be attributed to the Programme, as opposed to another factor. Finally, impacts may be qualitative or descriptive and difficult to quantify in a robust manner. This means that the evaluation has made various assumptions in order to determine the estimated value of the Programme. Any such assumptions used are described alongside the estimated economic impacts.

Different methods have been used to monetise the different types of impact. These have included drawing on published Government statistics (e.g. gross value added) but also other sources. For example, a report by the National Foundation for Educational Research has provided a value for monetising the benefits to the exchequer of an individual gaining a qualification. Whilst the evaluation has attempted to identify the most appropriate values to use to monetise the benefits, use of different values would result in a different estimate of impacts. This report presents the monetary estimates to two significant figures as a reflection of the level of uncertainty associated with the estimates.

All monetary value are presented in £2022 prices with values occurring in the future discounted using the social time preference rate of 3.5%^[25]. Additional details on the calculations, assumptions and sources can be found in Section 6 of the Technical Annexes.

Economic benefits – jobs and gross value added (GVA)

Programme data show that the number of jobs supported directly by the Programme ranges from 354 in 2016/17 to 267 in 2021/22, and totals 1,891 across the six years. Applying multipliers enables the numbers of indirect and induced jobs to be estimated. Such jobs may be in the supply chain or result from the income being spent by those employed by the Programme. The Scottish Government's input-output tables (latest version available is for 2019) provide multipliers for SIC code 72, Research and Development^[26]. Applying the multipliers to the number of directly supported jobs produces estimates of indirect and induced jobs. The total estimated number of jobs supported by the Programme is calculated as 3,971 (with each job counted as lasting one year).

GVA per head figures for Scotland can be used to estimate the GVA generated by these jobs. GVA data are available for five years of the Programme (2016 to 2020)^[27]. For this assessment, the value for 2021 assumed to be the same as that for 2019^[28]. Applying the relevant GVA per head figure to the number of jobs in each year of the Programme results in a total estimated GVA of £100 million (£2022, to two significant figures). These benefits relate directly to the Programme in that the Programme has provided funding for jobs. However, there is no information on whether employees moved into their positions from being unemployed or from other jobs. Some of the jobs may be displaced from elsewhere in Scotland rather than being new or additional (as noted in the Green Book, interventions may divert resources from places that are nearby or with similar characteristics^{Error! Bookmark not defined.}). This means that a proportion of the GVA benefits may be local benefits rather than national.

Economic benefits - Spin-outs

Whilst there were both spin-outs in existence and newly formed during the 2016-22 Programme, these all related to research carried out under previous Programmes. However, it could be assumed that over the next five years, spin-outs will be developed that relate to the 2016-22 Programme. Evidence from Hutton reinforces this assumption, with the 2012/22 annual report noting that a spin-out is in development for commercial pest and disease forecasting^[29]. The company will be launched in Africa and smallholders are lined up to test the forecasting service.

Annual metrics data indicate that three spin-outs were formed during the 2011-16 period that were still in existence during the most recent Programme (2016-22). It is assumed that another three spin-outs will be formed in 2022-27, but due to research carried out in the 2016-22 Programme. The duration of a spin-out is likely to be variable, as is the number of staff employed. However, if each of these spin-outs is in existence for three years, and initially employs four people^[30], then the potential GVA benefits resulting from these jobs can be estimated. Assuming a spin-out is formed every other year for the next five years, this could result in an additional £890,000 of GVA generated (present value, £2022).

Economic benefits - Licence agreements

There were 17 licences granted in relation to the 2016-22 Programme. These included seven for raspberries, two for maps, four for sheep scab, two for bovine serum from BVBD infected animals, one for calf jackets for faecal sampling and one for supplying a test for a nematode that affects sheep and goats. Revenue was received by both Hutton and Moredun in relation to these licences (and potentially existing licences related to research under the previous Programme). Total revenue for intellectual property equated to £4.6 million (present value, £2022). This revenue is assumed to result from the Programme, since without the underlying research, the products for which the licences have been developed would not be available.

Economic benefits – Animal diseases

Various animal diseases have been researched during the 2016-22 Programme. Work on sheep scab has continued, building on research undertaken in the 2011-16 Programme. EPIC has undertaken modelling to determine that where farmers use the diagnostic test for subclinical sheep scab, they will reduce prevalence of the disease and improve sheep welfare with no additional cost^[31]. Sheep scab was also a focus of knowledge exchange activity carried out by Moredun, which brought together biosecurity information on sheep scab and other diseases (e.g. bovine tuberculosis) for farmers, vets and policy makers^[32]. The work was launched at the Royal Highland Show in 2016. Moredun has also carried out work to keep farmers and vets up to date with research on sheep scab following previous work to eradicate the disease in Mull^{Error! Bookmark not defined.}. Furthermore, work has continued during the 2016-22 Programme to develop and test a plant based system for producing a vaccine^[33]. The results suggested that plant based systems might provide a cheap and scalable way of producing vaccine components. The ongoing efforts to counter sheep scab indicate that the annual benefits estimated under the previous Programme are likely to continue or even be enhanced.

Sheep scab notifications in Scotland have been relatively stable from 2014-19^[34] with data from the latest full year (2021) showing a similar number of notifications to previous years^[35]. Costs for individual flock infestations vary from £1,000 to £2,400 per 300 ewes for lowland animals and from £1,000 to £2,100 per 300 ewes for highland ones^{Error! Bookmark not defined.}. However, impacts from research are expected to take time to be realised. BiGGAR Economics (2017)^[36] estimated that SRP funded research from 2011-16 (including work on a sheep scab diagnostic test) could reduce the costs of sheep scab and hence benefit the Scottish economy by £1.8 million per year^[37]. These benefits were not expected to be felt until 2026^{Error! Bookmark not defined.}. Given the additional research carried out under the 2016-22 Programme, it is assumed that these benefits would start earlier than previously suggested. Awareness raising and strategic use of the diagnostic test (as demonstrated by the modelling work) could lead to different management practices and therefore save costs. These avoided costs (i.e. benefits) are expected to be felt sooner than 2026. This assessment assumes the annual benefits begin directly after the conclusion of the 2016-22 Programme, and continue for five years, as research on disease control and testing is put into practice. Uprating this annual figure to £2022 prices and discounting it for five years beyond 2022 results in present value benefits of £9.3 million.

Environmental impacts – Avoided carbon emissions

Many of the funded projects have the potential to decrease carbon emissions and contribute towards net zero. Avoided carbon emissions are a benefit that can be monetised using carbon values from BEIS. One example project is the Food Swap tool developed by BioSS and Rowett, which has the potential to enable more sustainable and healthier diets through identifying alternative foods. An earlier tool (Number Muncher Diets), also developed by BioSS and Rowett, provides some figures on carbon emissions of an example diet. It shows that decreasing the preferred minimum amount of red meat consumed from 10g per day to 5g, and white meat from 15g per day to 5g could reduce the CO₂e associated with a non-vegetarian diet by 0.1kg over one week^[38]. Multiplying this out over a year means a reduction of 5.2kg. The population of Scotland is estimated as 5.46 million^[39], with 4.31 million being 20 years old or more, and so assumed to have some choice over what they eat. It is not clear how many of these individuals might decide to use the tool. However, the Food in Scotland Consumer Tracking Survey^[40] undertaken in 2020 found that 78% of those surveyed agreed with the statement "In Scotland we have to make significant changes to what we eat to be healthier". Meanwhile, 48% of respondents thought it was very important or quite important to eat less meat^{Error! Bookmark not defined.}. These figures suggest that a considerable proportion of people think their diets should change. However, the extent to which people will take action and make changes is another issue.

The Tracking Survey also found that 25% of respondents thought it was very important to look at the front of pack traffic light or colour coded labelling on calories, salt levels, etc. in packaged food^{Error! Bookmark not defined.}. This suggests one quarter of respondents do take action to try and choose healthy food. It could therefore be assumed that 25% of the adult population would use the tool to make dietary changes to reduce the amount of meat they eat. This would be consistent with the proportion who already think it is very important to check food labelling, but lower than the proportion who think they should eat less meat. It would also be in line with figures from Australia which indicate that 22% of survey respondents believed that eating less meat would have environmental benefits^[41]. Assuming 25% of the adult population use the tool and make dietary changes that save each individual 5.2kg CO₂e each year, this could result in annual emission savings of 5,600 tonnes.

These emission savings can be monetised using the carbon values from BEIS (2021)^[42]. Assuming the dietary changes are made for 10 years, this would mean that 56,000 tonnes of carbon emissions would be avoided. The monetary value of these avoided emissions is £13 million (present value, £2022)^[43]. This value should be seen as an indication of the potential benefits of reduced emissions from one project, should the outputs from that project be widely disseminated and acted upon. Across the Programme, other projects are likely to have similar benefits, should the assumptions hold.

Case Study: Food, Health and Wellbeing theme

Project title: Food Swaps for healthier, more sustainable and affordable food choices

Organisations involved: Rowett Institute and BioSS (these organisations also developed the previous model, the Number Muncher. The latest food swap tool takes this a step further)

Summary of project: This project (unclear timeframe and cost) developed the FoodSwap tool which draws on the National Diet and Nutrition Survey (NDNS) Nutrient Databank to enable better food choices. Results from the tool were published in two academic papers. The tool was runner-up at the 2021 Innovate UK KTN Food Industry Innovation event, and presented at the 2021 Foods of the Future Colloquium. Funding from the Aberdeen Grants Academy, and collaboration with NHS Grampian, allowed the use of the NDNS Nutrient Databank to develop personalised dietary recommendations for individuals with Type 2 diabetes

Economic impacts: The project aims to equip people with the tools to have more affordable 'shopping baskets'. Further, any resulting health improvements could potentially alleviate pressure on the NHS through enabling people to make healthier choices when food shopping

Environmental impacts: Sustainable food choices are a key aim of the tool, which factors in the carbon impact of food. Use of the tool could potentially lower carbon emissions through encouraging individuals to choose foods that have lower carbon footprints and so change their diets over time. The SEFARI blog page notes that food consumption relates to up to 30% of the environmental impact of household consumption

Social impacts: This tool aims to make food healthier, more affordable, and more sustainable. This will potentially lead to wider societal benefits

Future outputs: The tool formed the basis of a PhD studentship. It has been taken forward as a demo model with money from SEFARI Gateway Responsive Opportunity Funding. The next step will be to develop an interface offering a more personalised experience. Another aim is to develop an e-health app to motivate dietary change. The researchers have already been in touch with those at Scotland Food & Drink and the NHS as part of continuing to test and validate the tool and ensure it is consumer friendly.

Source: SEFARI (2022): Strategic Research Programme End of Programme Report 2016-2022, p16. Report by Scottish Environment, Food and Agriculture Research Institutes. SEFARI (2020): Innovative New Tool: Food swapping for healthier and more sustainable diets, accessed at: innovative-new-tool-food-swapping-for-healthier-and-more-sustainable-diets

Social impacts - social value of having a secure job

Over the six years, the 2016-22 Programme has directly provided 1,891 jobs. The Social Value Bank (HACT, 2021^[44]) identifies the value of a secure job as £12,034 per year (£2021). This figure relates to the social benefits to the individual of having the job, rather than the economic benefits, which are already captured within the GVA figures estimated above. Assuming each of the jobs is provided for one year, this value can be applied to the total number of jobs to estimate the social benefits of those jobs. Uprating the value to £2022 prices and applying it to the 1,891 jobs results in benefits of £24 million (to two significant figures).

Social impacts – benefits of having new skills (PhD students)

Across MRPs and CoEs there were 1,862 full-time PhD students and 95 part-time students. Of these, the SRP directly supported 160.5 full time and four part-time students. However, some of the other PhD students may also have been supported, at least partly, by funding from the Programme. There is expected to be a link between the number of PhD students and the number of staff, since PhD students will require supervisors in their areas of expertise. To estimate the number of PhDs that received some support from the Programme, the ratio of the number of SRP supported PhDs to SRP staff was calculated for each year. This ratio was then applied to the total number of staff supported by the Programme in each year to estimate the number of PhD students supported by the Programme^[45]. This approach resulted in an estimate of 222 PhDs supported.

The benefits of the additional skills gained through PhDs can be valued using figures identified by the National Foundation for Educational Research (Lynch et al, 2015^[46]). This highlighted returns to the Exchequer from change in tax revenues associated with several vocational qualifications. It identified the greatest benefits as being associated with apprenticeships, with a one-off benefit to the Exchequer of £56,000-£81,000 (£2015). This figure is viewed as appropriate for valuing the benefits of PhDs because these qualifications also take several years to achieve (as per an apprenticeship^[47]) and are specialised in nature. Updating this value to £2022 and applying it to the 222 PhDs gives benefits of £18 million (to two significant figures).

Social impacts – social return on investment

Georghiou (2015)^[48]notes that some research would not be performed without public support, since factors such as uncertainty mean that private companies may be unwilling to invest. Public research is seen as complementary to private input and a driver of business investment. This view is consistent with what has occurred under the 2016-22 Programme, in that the research carried out has resulted in investment by industry in licences and other commercial outputs, but also participation in research contracts (as demonstrated by figures for subcontracting).

The value resulting from public research is thought to be around three to eight times the initial investment over the full life cycle of impacts^{Error! Bookmark not defined.}. This equates to around 20-50% in terms of an annual rate of return. For the 2016-22 Programme, some of the return on investment has already been captured through the impacts discussed earlier e.g. skills. However, there will be other benefits that have not been monetised because they are very specific to an individual project, are difficult to quantify and hence value, or have not yet occurred. The average rate of return on public investment can be used to estimate the extent of these benefits. It is therefore viewed as picking up any impacts that are not specified individually.

For each year of the Programme's investment, it is assumed that five years of benefits will result. Some benefits are expected to occur beyond five years, but since the length of a typical RESAS Programme is five years, it is likely that there would be some additional investment in research at this point. Alternatively, the research may have been developed further or in a different direction (in line with the latest research strategy). The five year timeframe is therefore chosen to avoid accounting for future research investment now, and also to bear in mind that research fields may move on with the result that something that is five years old may be less relevant. However, this timeframe is not expected to capture all benefits from the 2016-22 Programme.

As a starting point, the low rate of return (20%) is applied to avoid any potential double counting given that some impacts have already been monetised separately. Applying 20% to each year's investment in the Programme (with figures uprated to £2022) and discounting for benefits expected beyond 2022 results in potential benefits of £290 million (to two significant figures).

In order to estimate a range for the benefits from investment, the mid-point rate (35%) is also applied. Note that the upper estimate of 50% is not used since this would overestimate the total benefits when combined with the benefits from the other impact categories. Applying a 35% rate of return gives potential benefits of £510 million (to two significant figures)^[49].

The benefits from the social return on investment are therefore estimated as ranging from £290 million to £510 million. This range is a broad-brush estimate of the return on investment and provides an indication of the order of magnitude of the benefits.

Total estimated value

The sum of the above benefits is £470 million in 2022 prices if the low estimate for social return on investment is used. Note that if the higher benefits estimate for social return on investment is included, the total benefits rise to £680 million.

Table 4-2 presents a summary of the monetised benefits. It demonstrates that the monetised impacts are greater than the Scottish Government's investment of £279 million. This is despite some impacts not being explicitly included in the valuation. Non-monetised impacts cover economic benefits from work on plant diseases, benefits to properties due to avoided damages from flooding (expected to result from work on natural flood management, and the Living with Flooding' action plan), and environmental benefits linked to protection and enhancement of natural assets (e.g. work on oak trees fed into woodland conservation). Also excluded from valuation are the social benefits of improved health and wellbeing. Whilst various pieces of research related to health (e.g. SRUC's research that fed into the National Rural Mental Health Forum), the benefits are difficult to quantify with any level of certainty, Many assumptions would be needed to attempt to monetise such impacts, so they are not included within this valuation.

As noted by the Magenta Book, social cost benefit analysis is reliant on data for costs and benefits to be monetised^[50]. There is a wealth of Programme documentation on outputs from the MRPs, CoEs and Scottish Government. However, the full extent of how these outputs result in impacts is yet to be felt. This assessment makes some assumptions, but the full value of the Programme is likely to be greater and include impacts that are not yet explicit.

For comparison purposes, the economic assessment of the previous programme (2011-16) identified estimated benefits of £468.9 GVA from operational impacts and £151.8 million GVA from economic benefits^[51]. This gives a total of £620.7 million, which is higher than the lower estimate for the 2016-22 Programme of £470 million, but close to the upper estimate of £680 million. However, the figures are not necessarily comparable because different approaches were used for the two assessments. For example, the assessment for the 2011-16 Programme monetised more individual impact categories (including benefits for plant health) than the 2016-22 assessment.

Table 4-2: Summary of monetised benefits (£2022)

Impact type	Value	Details	Assumptions and caveats
Jobs and GVA generated from jobs	1,891 jobs under the Programme, with 1,513 indirect jobs (in the supply chain) and 567 induced jobs (linked to spend by those in direct jobs) GVA of £100 million	Indirect and induced job numbers estimated through application of Type I and Type 2 multipliers (Scottish Government, 2022). GVA based on ONS (2022) values for GVA per head for Scotland	Some jobs may be displaced from elsewhere rather than being additional jobs so a proportion of GVA may relate to local benefits rather than national benefits
GVA generated from spin-outs	£890,000	Draws on GVA per head values for Scotland (ONS, 2022) with values discounted using standard rate of 3.5% as per the Treasury Green Book	Assumes three new spin-outs formed over next Programme (as during 2011-16 Programme) with each spin-out having four staff members and a duration of three years. A larger spin-out or those lasting more than three years could generate additional benefit
Licence agreements	£4.6 million	Relates to income received by Hutton and Moredun for licences	Some income may relate to older licences rather than just the 17 identified as being related to the 2016-22 Programme. Income that may be received beyond the Programme has not been captured
Economic impacts – animal diseases	£9.3 million	Based on BiGGAR Economics (2017)	Sheep scab benefits continue at rate estimated for previous Programme (£2.1 million annual benefits in £2022)
Environmental benefits – avoided carbon emissions	£13 million	Estimates of carbon emissions saved based on Number Muncher Diets (Rowett Institute and BioSS, no date) Carbon values from BEIS (2021)	Example benefits based on one project. Food Swap tool is used to make dietary changes to decrease carbon emissions by 25% of the adult population of Scotland; carbon savings are made for 10 years
Social value of having a secure job	£24 million	Based on HACT (2021) value of having a secure job	Each job is valued on an annual basis; applied to jobs directly supported by the Programme only
Social benefits of new skills	£18 million	Based on Lynch et al (2015) and the increased value to the Exchequer from an increase in skills	Estimated 222 PhDs assumed to be supported by the Programme over the six years (calculated from the ratio of SRP supported PhDs to SRP supported staff and multiplied across all staff)
Social return on investment	£290 million (£510 million if a moderate rate of return of 35% is assumed)	Based on Georghiou (2015) and 20% annual rate of return for public research	Benefits are felt for five years after each investment period (so 2016/17 investment results in benefits for 2017/18 to 2022/23)
Total monetised benefits	£470 million (£680 million if a moderate rate of return of 35% is assumed)	Values to two significant figures in £2022 (figures presented here have been rounded so may not sum)	Not all impacts are monetised

4.4.2. Impacts from the research undertaken

Evaluation question I4 (What environmental, economic and social impacts to Scotland did the research themes achieve?) provides information to illustrate some of the non-monetised impacts of the Programme.

Economic impacts

RESAS' research has contributed to numerous economic impacts. For example, Scotland's Agritech, Animal Health & Aquaculture (AAA) sector represents a significant opportunity for the growth of the Scottish economy and increased trade. However, there is no platform that outlines the capacity and capabilities of the sector. To overcome this issue, a SRUC led a SEFARI project to create an inventory of the sector's capabilities and resources^[52]. The report found that there are over 210 companies and stakeholders operating in the AAA sector in Scotland and one of the largest concentrations of expertise in animal and veterinary science in Europe and the world at Easter Bush Campus. The potential of the future AAA sector was also explored, especially the need to define the gross value added of the sector. This report was vital in the A3 2020 International Conference and has also led to an enhanced focus on the AAA sector from Highland and Islands Enterprise. Access to a comprehensive overview of the sector will allow for better planning in the future and the opportunity to maximise potential economic benefits.

Case Study: Productive and sustainable land management and rural economies theme

Project title: A framework for benchmarking greenhouse gas (GHG) emissions intensity in Scottish farming

Organisations involved: CXC, SRUC, and James Hutton Institute

Summary of project: This project was commissioned for Scottish Government and cost £49,687. It developed a framework methodology to assess GHG emissions intensity data on Scottish beef farms. It resulted in a public report published in 2019

Economic impacts: No explicit economic impacts are noted

Environmental impacts: The method allows the estimation of the emissions of the Scottish beef industry, informing the development of steps needed towards net zero. It is expected that implementing the framework, and expanding it to other food production sectors, could help to lower the emissions of food production

Social impacts: The greater availability of information on emissions has the potential to inform a just transition

Future outputs: The researchers recommend that the tool is implemented for the beef sector in Scotland. Their paper notes that use of the tool will inform decision making related to the Scottish Government's commitment to reduce the carbon footprint of food consumed

Source: ClimateXChange (2019): Annual Report 2018-19 Wiltshire J, Martineau H and Jenkins R (2019): A framework for benchmarking greenhouse gas emissions intensity in Scottish farming. Report by Ricardo Energy and Environment for ClimateXChange. Available at https://www.climatexchange.org.uk/media/3648/a-framework-for-benchmarking-greenhouse-gas-emissions-intensity-in-scottish-farming.pdf on 17 November 2022

Another economic impact includes EPIC's work on the creation of a computational tool that better assesses disease risks in which knowledge of the population risk is incomplete^[53]. This tool will be used by scientists to improve responses to disease outbreaks in wildlife populations that are currently not well mapped and also diseases in livestock in which details of all holdings are difficult to accurately obtain. This in turn will help policy makers make decisions that will help control disease outbreaks in the future and therefore reduce any potential loss of livestock.

EPIC also undertook work that has had beneficial impacts for land management. Specifically, they have analysed pig movement data from England/Wales and Scotland to see how movements of live animals within production pyramids are structured, and the impact that novel movement restrictions have upon the pig industry^[54]. This research has led to ongoing industry interest and has been shared with numerous government agencies around the UK. This links to EPIC's other work, especially the formation of a Scottish Poultry Group that aims to foster closer relationships between the poultry industry and policy actors^[55]. This led to the creation of the Scotland's Poultry Hub, which illustrates the impact EPIC's work.

Moredun undertook work to see how grouse moors could be managed in a more sustainable way and ensure compliance with the law. This was achieved by analysing how medicated grit (taken up by red grouse), would help reduce their susceptibility to parasitic worms especially in the more densely populated grouse moors^[56]. The project led to a far greater knowledge exchange between grouse moor owners, managers, and gamekeepers that has resulted in many more Scottish Estates improving their practice with regards to the use of medicated grit.

Environmental impacts

CREW developed a guide for farmers and land managers to help them create the most suitable sustainable drainage systems for their farms and also how to look after the system when it has been created^[57]. A sustainable drainage system prevents diffuse pollution that can often come from rainfall runoff after fertiliser application or chemical spraying is used on the land. As well as helping to reduce the amount of harmful runoff, the guide also promotes the wider benefits of rural sustainable drainage systems, such as helping to cope with extreme weather-related climate change, localised flood prevention and enhanced biodiversity.

Under the Programme, the James Hutton Institute has undertaken a long-term sustainable cropping experiment at Balruddery farm^[58]. Samples of soil, weeds, crops, harvested materials and invertebrates are collected every growing season then stored. Samples and data are made available to all contributing projects on the site. A web-based platform has been created to allow easier access for contributors, and requests for access can be made from outside groups. During 2018/19, there was a total of 30 requests for data from the Programme, PhDs, and EU and Innovate UK funded projects. The resulting projects have advanced scientific knowledge relating to arable sustainability over a range of disciplines. Furthermore, there were six events organised, three peer-reviewed papers published, and the further development of links with industry and EU farm platform networks. There has also been greater acceptance from both growers and industry bodies of environmental farming practices during the life of the Programme. CXC has undertaken numerous pieces of work that have had positive environmental impacts. For example, they have guided and evaluated the Scottish Government's substantial peatland restoration efforts which are an important part of their climate goals. This is because of their potential to sequester large amounts of carbon dioxide and therefore act as a carbon sink. Linked to this, CXC carried out work on agroforestry, which describes the integrated use of trees on a farm or small holding. This could have the double benefits of increasing woodland cover and also help to lower emissions^[59]. The study identified the main agroforestry options available to Scotland and compared the benefits and challenges of these options. It will act as a useful resource for the practical deployment of agroforestry in Scotland, and ultimately have positive environmental impacts.

SEFARI's SRP and externally-funded collaborative research has shown that over 500 species associated with oak may decline^[60]. Work also identified possible replacement species. The findings were used by Defra within the Action Oak initiative and in an Action Oak Knowledge Review and are informing woodland biodiversity conservation, demonstrating how the Programme can feed directly into protection of natural assets.

Case Study: Natural assets theme

Project title: Development of an in-house method for the separation, identification and analysis of microplastics from solids and sediments

Organisations involved: James Hutton Institute

Summary of project: This completed seedcorn project ran from 1 Oct 2018 to 30 Sept 2019 (12 months), with a budget of £19,928 and the involvement of six staff members. It led to the development of a new method for the determination of microplastics in environmental samples. Outputs included a PhD studentship using the method, presentations at international conferences, collaborations with India, and interaction with other researchers. Funding to create a microplastics pollution map was applied for but this was rejected

Economic impacts: The separation technique is cheaper and faster than the traditional method, therefore it could potentially be rolled out on a wider scale. The method provides more efficient extraction, as well as reduced process times and involves fewer toxic chemicals

Environmental impacts: The research could potentially lead to more effective environmental monitoring, due to the ability to record more data on microplastics, and could improve understanding of the pathways of microplastics transport into the environment

Social impacts: Microplastics have an unknown but potentially negative impact on human health, so this new method could improve monitoring and potentially lead to better policy to combat relevant issues

Future outputs: A Notification of Interest for a NERC Grant was submitted; if successful, it would involve further collaborations with Hutton staff

Source: James Hutton Institute (2020): Annual Report on Underpinning Capacity 2019/20 James Hutton Institute (2021): Separating Microplastics from Solid Matrices, News article from 27 August 2021, accessed at https://www.huttonltd.com/news on 13 January 2022

Social impacts

Research from RESAS has had clear social impacts. SRUC undertook an exploratory data analysis of the first rural mental health survey carried out in Scotland^[61]. The analysis was policy focused and the researchers worked with the Scottish Government to ensure that key findings would feed into the government's Mental Health, Isolation and Suicide Strategies. This analysis influenced the formation of the new National Rural Mental Health Forum, a partnership of 30 organisations (both rural and mental health) which are taking forward a collective national wellbeing campaign. Furthermore, the report was cited in the Scottish Parliament Chamber in relation to mental health provision in rural Scotland.

Another social impact stemming from the Programme was the creation of Scotland's National Coastal Change Assessment (NCCA) by CREW^[62]. The aim was to create a shared evidence base to support more sustainable coastal and terrestrial planning decisions in the context of a changing climate. This led to the creation of Dynamic Coast, an open access web-based tool that offers reports and maps of the entire 21,000 km of Scottish coastline, allowing easy identification of potential areas of erosion. The web resource achieved 13,000 hits from 8,000 users globally by March 2019 and won a prize at the 4^th Scottish Knowledge Exchange Awards 2019. The tool has been used to support numerous policies and plans from the UK and Scottish Governments, such as the UK Climate Change Adaptation Plan and Marine Planning Policies.

4.4.3. Impacts under the extension year (2021/22)

Two evaluation questions considered impacts under the extension year. This section brings them together to report on:

• The extent to which research funded through the Programme extension looked at the impacts of Covid-19, the recovery from Covid-19 and data emerging from the Covid-19 pandemic (C3); and
• The extent to which research funded through the Programme extension looked at the impacts of EU-exit (C5).

Covid-19 research

Annual metrics data for the extension year (2021/22) indicate that there was a relatively limited amount of research undertaken on Covid-19. This may reflect the aims of the extension year, in that whilst Covid-19 work was included, there was also a need to catch-up and conclude other research. Covid-19 outputs covered:

• Policy outputs: of the 196 policy outputs from the extension year, three were identified as being specifically related to Covid-19;
• Peer reviewed publications: five of the 237 peer reviewed publications related to Covid-19;
• Scientific outputs: four outputs (one by Hutton, one by EPIC and two by BioSS) were recorded as relating to Covid-19 from a total of 226 for the year;
• Publications for industry: one publication from a total of 107 for the year was produced on the impact of Covid-19 on women in agriculture;
• External collaborative projects: eight Covid-19 related projects were reported by BioSS from a total of 375 for the year. These collaborations were with universities in Scotland, the wider UK and other countries; and
• Membership of advisory groups or boards: four researchers (three from BioSS and one from Rowett) attended non-government committees/boards related to Covid-19, whilst one researcher from Moredun was a member of a government group linked to Covid-19.

Members of the Scottish Government were asked if the extension year led to projects related to the Covid-19 pandemic, leading to five responses. Three interviewees pointed to specific examples of work undertaken in response to the Covid-19 pandemic. This included the use of some labs to undertake Covid-19 testing and analysis of sewage content to estimate viral load. Programme reports show that CREW worked on a suite of analyses on the "early warning" potential of wastewater-based epidemiology in the context of Covid-19 surveillance. It was also noted by one stakeholder that they developed new methodologies and models for the pandemic that were not used, but did advance their techniques.

Stakeholders at the MRPs were asked the same question. Similarly to the Scottish Government stakeholders, the use of laboratories for testing and creation of models was mentioned most often (by five of 15). One stakeholder expressed their disappointment that one of the projects related to Covid-19 had to be cancelled as it could not be designed, implemented and reported on within one year. They believed there was a missed opportunity as their work could have been valuable. There was similar feedback from stakeholders from the CoEs and the ARPs, who pointed to their work on risk assessment and modelling required for policy decisions related to the pandemic. An example of this has been identified from the Programme documents. CXC undertook research for Transport Scotland investigated employer perceptions of Covid-19 impacts on travel, including barriers to increased home working, and helped to shape Scotland's Clean Air Zone policy.

EU-exit

Extension year outputs related to EU-exit included:

• Seven policy outputs from a total of 196. Rowett (two outputs) and SRUC (five outputs) were responsible for these outputs;
• Six publications for trade from a total of 107 related to EU-exit. All of these publications were produced by SRUC and related to the theme of 'Productive and sustainable land management and rural economies';
• Other EU-exit related outputs (not elsewhere classified) included a podcast on rural policy produced by SRUC and a presentation at a RESAS seminar, again by SRUC, on trade agreement scenarios; and
• One researcher (from Moredun) was a member of a government group linked to EU-exit. None of the representations on non-government boards and groups were related to EU-exit.

Interviews suggested that EU-exit themed projects in the extension year were not undertaken to a substantial degree, although interviewees from the Scottish Government were aware of 'Brexit-themed projects' focussing on impacts to trade. However, interviewees were less frequently able to comment on the nature of EU-exit related research as they were not part of the undertaking. One interviewee stated that the research under RESAS had a scientific remit and was less relevant for strategy/policy outcomes related to the EU-exit. Rather MRP interviewees suggested the majority of research in the extension year related to the continuation of previous years' work, with some studies related to Scotland's Covid-19 response.

Of those that commented on EU-exit related projects, one MRP thematic lead stated that such projects aligned with the topic of a 'green recovery' and explored pathways to achieve such a recovery; although the interviewee did not believe any of their projects explicitly focussed on the recovery. They continued to state that other studies focussed on business resilience issues resulting from the EU-exit. Another MRP interviewee felt that the extension year was key for the undertaking of EU-exit related research, which otherwise would not have occurred.

4.4.4. Benchmarking: comparisons with other programmes

The 2016-22 Programme was praised by one stakeholder for "taking good, and often world-leading, science, and using it for practical outcomes", as well as for being "longer-term and big picture" compared to the "short-term and reactive" UK Research and Innovation (UKRI). UKRI is funded through the science budget of the Department for Business, Energy and Industrial Strategy (BEIS) and is a significantly bigger Programme, in terms of money spent per annum, compared to the SRP. The nine councils that comprise UKRI had annual budgets ranging between £140 and £2,377 million in 2019-2020, whereas the RESAS Programme funded £279 million worth of research between 2016-2022.

The scope of the Programme was acknowledged as being different to research funded through comparable schemes, but the level of expertise in the Programme was acknowledged as being "on par". UKRI, for instance, funds a much wider body of work, in fields from adolescent mental health to waterway engineering, or from the generation of synthetic chemical energy carriers to electrochemical science. Whilst work funded under the 2016-22 Programme was seen as "positively received"; one stakeholder suggested that "UKRI funded work might have more of a higher-standard reputation".

Interviewees found the RESAS Programme applications "a lot more straightforward than UKRI", which typically "have a lot more information collated at the application stage". One interviewee noted that UKRI proposals require "justification for resources, the pathway to impacts, and animal use statements", as well as "six pages on project objectives and methodology and two about your track record", compared to the four page limit for Programme proposals, which made things "not so easy". The longer page limits in UKRI allowed for people to "get their scientific concepts down on paper" and develop the "scientific reasoning within an application". However, this need for more detail is likely to be behind the comment that other funders are more prescriptive than the RESAS Programme about the way in which research should be carried out.

It was also noted that reporting requirements for the 2016-21 SRP was much heavier than for UKRI. This was reaffirmed by another stakeholder, who noted that "much more reporting was involved compared to UKRI awards of comparable value". A RESAS stakeholder noted that the aim was to have a lighter application process for academics compared to UKRI processes, but acknowledged that some of the choices made were "more burdensome or more challenging". This was corroborated by the experiences of one MRP-based interviewee, who said they were new to the most recent SRP Programme, but expressed surprised that "four pages went into the system and a decision was made; it was very different to a UKRI grant of this value where there are many other things to complete via the Joint Electronic Submission system (Je-S)". One applicant appreciated that the application process for the new 2022-27 Programme made them feel like they were writing a "more connected piece of research", commenting that the process had become more like that of UKRI.

4.5. Programme inputs and delivery

4.5.1. Overview

This section covers Programme inputs. It discusses the advantages and disadvantages of the delivery model, the procurement arrangements and the Research Delivery Framework (RDF), drawing on data collected for the process evaluation questions (P1 to P4). It considers Programme delivery and performance of MRPs and the Scottish Government.

It also discusses the extension year, specifically its design, administration and implementation (question C1). Evaluation question C2 provides information on the impacts of EU-Exit and Covid-19 on the extension year's administration. There is then discussion on the extent to which the extension year supported research disrupted by the pandemic (C4) and whether it was effective (C6).

4.5.2. Delivery team structure and operation of the Programme

The governance structure for the 2016-22 SRP is shown in Figure 4-6.

The SRP had strategic oversight from the Strategic Advisory Board (SAB), which was directly fed information on impacts from an Operational Group (OG) consisting of Directors' Executive Committee (DEC) and RESAS members within the Scottish Government. The DEC provided operational management of the SRP, reporting to the SAB/OG. The DEC consisted of MRP Directors, the Knowledge Exchange (KE) Director, Programme advisors, and the DEC secretariat. Alongside the DEC, a Strategic Advisory Group (SAG) existed to advise the DEC on progress toward impacts and KE work. Within the MRP institutions, Theme Coordinators coordinated all research within each theme, monitored progress, and reported impacts to the DEC and RESAS' science advisors. The Theme Coordinator role was the main interface between researchers and strategic operators.

Within each thematic area, research was split into distinct work packages facilitated by Work Package (WP) Coordinators. WP Coordinators were responsible for the overall delivery of research to budget and timeline. And finally, Research Deliverable (RD) coordinators were responsible for ensuring the individual projects were undertaken and responsible for monitoring and reporting. RD coordinators would routinely upload information to the Research Delivery Framework (RDF), which was a set of reporting spreadsheets hosted on Huddle.

Reports, data, and provisional impacts were recorded and passed through to the Centre for Knowledge Exchange and Impact (CKEI) to support dissemination and impact generation. Underpinning Capacity (UC) provided financial support for operational costs and assisted with promotion of research to underpin policy and strategy decisions.

Overall, Scottish Government, MRP, and COE interviewees felt the RDF worked effectively although many interviewees highlighted areas for improvement, particularly regarding monitoring and reporting. These were seen as too frequent and required too granular data from the outset of the Programme.

Communications

Interviewees generally felt the mechanisms in place to communicate were effective. Some MRP interviewees, however, thought it was quite difficult to communicate outside of their institutes with some partners (particularly Principal Investigators (PIs)) proving difficult to get in touch with. It was suggested that it would be beneficial in future to appoint a single point of contact within the institutions to coordinate and chase on behalf of the RD leads.

MRP interviewees saw the use of Excel spreadsheets for reporting as a major issue, with one RD coordinator stating they had designed their internal communications to be as minimalistic as possible, as too much time was already spent on reporting via spreadsheets which meant less research was conducted.

Relationships

Whilst MRPs stressed the benefits of co-productive approaches, such as ensuring minimal knowledge gaps and the production of cross-cutting, robust research, they also identified drawbacks in terms of dependencies on strong working relationships. Interviewees noted that staff losses throughout the Programme were problematic as it required delivery teams to rebuild relationships with new partners, who may have different subject knowledge. In addition, some partners were difficult to motivate to deliver work, particularly from partner organisations outside work package coordinators and RD Coordinators' institutes. It was also felt by some that there was a lack of accountability for those who did not deliver. Personal relationships with individuals at other research institutes were often labelled as key in ensuring work was delivered in a timely manner. For instance, when communication was occurring effectively across institutions, it was acknowledged this was due to historic relationships between the researchers involved.

Science advisors at the Scottish Government were praised as one of the mobilising factors of co-production and relationship building, noting that MRPs were particularly appreciative of feedback loops and assistance in identifying key stakeholders. MRP interviewees felt that there could have been further harmonisation between the thematic areas to encourage even more collaboration.

Project Delivery

Some interviewees felt that the RDF did not allow for flexibility for a project to change what was being delivered, how, and when. There was concern amongst interviewees, including DEC members, that there may not have been recognition within government that strategic research addresses long-term issues, and that research began to focus on meeting more short term policy goals. They believed the direction of research should be scientifically rather than policy led.

Interviewees from the Scottish Government also highlighted that some staff struggled to keep up with workloads, which was potentially linked to a staff shortage. Some noted that those responsible for work packages often became responsible for additional and unfamiliar work packages resulting in less quality work being completed in both areas.

4.5.3. Effectiveness of resources in facilitating the Programme's delivery

Staffing (2016/21)

Table 4-3 provides a breakdown by employment mechanism of the Programme's original five years. In terms of role, there were 225 researchers (191 natural scientists and 34 social scientists) and 130 technical staff (figures rounded up to nearest whole number).

A much larger number of FTE jobs were supported through SEFARI than through the HEIs. Throughout the original five years of the Programme, FTE numbers gradually decreased, with the 2020/21 figure of 289 FTE positions equating to 82% of 2016/17 positions.

Table 4-3: Breakdown of FTE jobs employed through the Programme, by job provider and by year

Year	Total FTE jobs supported	Employed via SEFARI	Employed via HEI	Percentage change to previous year's total FTE employment
2016/17	354	334	20	N/A
2017/18	354	330	24	No change
2018/19	336	312	25	5% reduction
2019/20	291	266	25	13% reduction
2020/21	289	264	24	<1% reduction

Funding from the Scottish Government

Total Scottish Government funding for the rural economy and environment portfolio was just over £279 million for 2016-22. This is in comparison to the £246 million invested for the 2011-16 programme (which was for five years). The portfolio covers the SRP, CoEs, Underpinning Capacity, innovation, and knowledge exchange and management. Annual amounts varied slightly, with the highest amount being £48.5 million in 2016/17. Annual funding then declined each year to £44.9 million in 2019/20 before increasing again to £46.5 million in both 2020/21 and 2021/22.

External funding for research

External funding from non-industry sources covers funding for research and consultancy that is related to RESAS funded research or that builds on previous Scottish Government funded research. Considering the original Programme (2016-21), external funding derived from non-industry sources totalled £85 million. Of the six MRPs, Hutton obtained the most non-industry external funding at £29 million with SRUC second at £27 million. The lowest amount of external funding was obtained by BioSS, which brought in £2 million.

Figure 4-7 disaggregates the data by MRP to show the different types of non-industry external funding received by each organisation for the 2016-21 period. All MRPs received money from five different sources, but in varying amounts.

Interview data

Interviewees from the MRPs, the CoEs, the ARPs, and the Scottish Government were asked if they thought that sufficient funds had been allocated to enable the delivery of the Programme. The responses were generally positive, however numerous respondents stated that it was difficult to answer or stated that there could always be more funding for scientific research.

Focusing specifically on the MRPs, 17 interviewees were asked this question. Most interviewees felt that there was sufficient money to achieve the objectives, however more money could have allowed better overall scientific research. Two interviewees stated that the funding decreased in real terms due to inflation during the Programme which made project management progressively difficult year on year. Linked to this issue, one of the interviewees praised the Scottish Government's compassion when cuts needed to be made. Three interviewees said that they had spent more money than they would have liked on the administrative side of the project management compared to the actual scientific research.

A total of eight stakeholders were spoken to from the CoEs and the ARPs. Five of the interviewees agreed that there were sufficient funds to enable delivery of the Programme. One stakeholder stated they had not actually spent all their research funding due to not being able to carry out all the work because of staffing constraints. However, overall, the CoEs and ARPs gave largely similar responses to the MRPs acknowledging that the funding was sufficient for the allocated work, but they would have liked higher funding to achieve more.

Members of the Scottish Government were also asked if they thought that funding was sufficient to enable delivery of the Programme. There was a total of five responses to this question. Three of the five explicitly stated that the funding was adequate, with one stakeholder pointing to the excellent outputs that had been achieved by the research Programme. The two interviewees who did not feel there was sufficient funding were scientific advisors, stating they experienced difficulties due to too few staff. However, it was noted that this is being addressed in the current (2022-27) Programme.

Members of the Scottish Government were asked the reasons for the change in the funding allocation and if staff numbers fluctuated in response to changing budgets or other factors. Four stakeholders responded to this question, and they all said that the overall budget allocation had changed which had trickle-down impacts into the research budget.

4.5.4. Effectiveness of the funding application process

Feedback from interviews suggests mixed experiences with the funding application process. In general, institutions had a good level of knowledge about the Programme's remit and its expectations, due to involvement in previous rounds. They also spotlighted strong existing partnerships between institution members in which collaborative and open approaches to research were being applied. Interviewees stated that research proposals were written and submitted in a tender response format (written application).

Interviewees identified several issues with the application process. It was suggested that the turnaround time between the release of the project specification and the deadline for applications was too short. Application page limits being too short was also raised, which it was felt had a detrimental effect on the quality of submissions. It was also acknowledged as a strenuous process to navigate whilst projects under the previous SRP were finishing and required reporting.

From the perspectives of CoEs and ARPs, one principal investigator stated that there was a lack of clarity on what the Programme wanted to achieve and therefore their application may not have been as relevant as it could have been. Interviewees frequently fed back that the research questions outlined by the tender were not well defined and that it was difficult to write a proposal to meet the needs of the Programme.

Delays in awarding contracts for successful applications created tensions within teams. Staff had concerns regarding their job security, with one interviewee mentioning that people looked for employment elsewhere as a result.

4.5.5. Effectiveness of monitoring and reporting requirements

Overall feedback from interviewees was that Programme monitoring was excessive and led to too much time being spent on administration instead of the actual science. More specifically use of an Excel spreadsheet that had to be frequently updated was criticised by numerous stakeholders, especially the MRPs. This was viewed as burdensome and challenging to use.

A total of five stakeholders from the Scottish Government were asked how the Programme monitoring and annual reporting was carried out. Three of those who responded mentioned the use of monthly monitoring with an Excel spreadsheet, with one stating that it was not a great method for absorbing information. Another aspect that was mentioned by numerous interviewees was the use of regular milestone checks that were in the delivery framework. These were delivered quarterly or every six months. One interviewee noted the importance of frequent informal discussions with the work package leads, which helped to address issues before they could delay the progress of any projects. Of the five stakeholders interviewed, one stated they believed that the progress reporting worked well whilst the rest did not comment on its effectiveness.

All stakeholders across the Scottish Government, MRPs, CoEs and ARPs were asked what they thought worked well and less well in relation to monitoring and reporting. There were four stakeholders from the Scottish Government who answered this question. The general theme across interviewees was that the reporting improved in the lifetime of the Programme as small changes were made and researchers gained more understanding of the requirements set out by the Scottish Government.

A total of 17 stakeholders from the MRPs were asked what worked well and less well with regards to monitoring. Focusing on what worked well, three interviewees praised the use of SEFARI for the annual reporting. There was also praise for how the system evolved in the time of the Programme. One interviewee also said that the information sharing was effective, particularly the annual online updates that were shared via the SEFARI website. However, the overwhelming view from stakeholders from the MRPs was negative, with thirteen interviewees stating that reporting was difficult with particular emphasis being placed on how challenging it was to fill in the Excel spreadsheet monthly. Note that the new (2022-27) Programme uses ResearchFish, which was praised by several interviewees.

The views of stakeholders from the CoEs and the ARPs were more mixed. Of the eight stakeholders interviewed four stated that it went well. They particularly praised the open communication they had with whoever they needed to report and the amount of time they were afforded to carry out reporting. However, other stakeholders had a more negative view. One stated that they were not clear on what the data they were reporting back was going to be used for and felt that the Scottish Government was merely going through the motions of collecting the data.

4.5.6. Design, administration and implementation of the extension year

Insights used to answer this question have been extracted from five interviews with senior managers and science advisors within the Scottish Government who were responsible for the implementation of the Programme.

The extension year was designed to support institutions during the Covid-19 pandemic when research projects experienced significant disruptions and risked becoming undeliverable. This disruption occurred during the original final year (year 5). One Scottish Government interviewee stated that options were to simply shut down the Programme for year 5 or to support institutions to continue research where possible. The extension year aimed to fund four broad criteria: to ensure the recovery and finalisation of research projects; to aid the achievement of impacts; and to undertake short-term research related to the impacts of Covid-19, and/or the EU-exit. The extension year was designed as a simple continuation of funding for institutions, relying on the same application and reporting processes as before, albeit with lessened rigidity and scrutiny. The extension was designed to have a loose grip approach to account for the highly complex and nuanced issues experienced by research projects.

Two interviewees highlighted that the design and implementation of the extension year occurred at a difficult time. Particularly, the extension year design occurred concurrently to the design of the next RESAS Programme(2022-2027) which had also been delayed due to the disruption, suggesting increased workloads were experienced. Further to this, the design occurred during a time when RESAS staff (particularly science advisors) had been seconded to the Covid-19 response hub, meaning few were available to assess new applications.

One interviewee stated that extension year outputs and deliverables were different from previous years, stating that they were "snappier" and more "rapid". The reason for this being that the extension year occurred towards the end of the Programme and projects could only operate for a maximum of one year unless they could be carried into the following new Programme (which was uncertain).

4.5.7. Impacts of Covid-19 and EU-Exit on the extension year's administration

The twin issues of Covid-19 and EU-exit caused disruption to the SRP's administration but did not derail the Programme in a hugely significant manner. According to feedback from stakeholders, this is chiefly a result of the Scottish Government's deployment of an extension year to the initial 2016-21 Programme. Covid-19 was seen as highly disruptive, but stakeholders from MRPs in interviews routinely praised the ease of communicating with the Scottish Government throughout the pandemic, commenting that the move to online meetings and interactions did not negatively affect their ability to get in touch.

Covid-19: Communications and Relationships

The most commonly noted impact of Covid-19, amongst all stakeholder groups, was the shifts in communication methods that it necessitated. One RESAS stakeholder noted that "everything went remote" and that "there were challenges that everyone faced [internally at RESAS] with being virtual", but that "the team overall worked our way through that and coordinated really well". The "huge reduction in face-to-face meetings" was acknowledged, as well as the reliance on personal relationships when working online as opposed to in-person, particularly with stakeholders at the research providers.

Throughout the wider SRP, nearly all interviewees were keen to stress the impacts of Covid-19 upon the ease of communication and relationship-building. Whilst Microsoft Teams and phone calls helped mitigate the impact, they were frequently labelled as no substitute for in-person interactions. For instance, prior to the pandemic, EPIC had team members regularly hotdesk inside Saughton House, which allowed routine in-person interfacing with Scottish Government Office. This was lost during the pandemic. Where working relationships had already existed, it was acknowledged as being "a relatively easy transition to chatting online". But in cases where these did not exist, there was a "loss of informal catch ups", with CoE stakeholders in particular feeling the loss. Scottish Government officials were frequently praised in interviews for their continued availability throughout the pandemic by MRP, COE and ARP stakeholders.

Covid-19: Staffing and Resourcing

During the extension year, the RESAS administration team were imperfectly staffed and resourced, due to staff being seconded to pandemic-related teams within the wider Scottish Government. The ongoing makeup of the team was referred to as a "skeleton staff and operation" during "a unique set of circumstances". New science advisors were hired during this time, which was a positive step in fixing wider, historical staffing issues.

CoE stakeholders noted a lot of researchers "were diverted to working on the Covid-19 response, especially modellers". CoE staff worked on the Royal Society RAMP Programme, and joined the Scottish Covid Response Consortium (SCRC). EPIC staff in particular were reassigned, which led to "a bit of reprioritisation of work" with originally envisaged projects being delayed or cancelled to allow staff to work on Covid-19.

Covid-19: Inability to complete fieldwork

Nearly all MRP stakeholders, when interviewed, referred to the inability to conduct lab and field work as having a significant disruptive impact. Labs were closed for "more than six months, with all human interfacing work having to stop, and even for the next year, we had huge restrictions and lower occupancy in labs". Meanwhile, researchers also could not "get out onto farms and into the field", and these issues combined led to significant delays on numerous projects and to an operational slowdown. It is important to note that stakeholders indicated not all projects to be affected. For instance, one Scottish Government interviewee noted that "farm work and animal feeding had to continue", because the animals on site ultimately needed to be fed or they would perish, but that "lab work was more greatly affected, like with Rowett, who did other bits and pieces to get themselves through".

Covid-19: Project Monitoring and Reporting

MRP stakeholders perceived a high level of impact from Covid-19 on reporting. For instance, some reported being furloughed, many reported having projects delayed, and it was acknowledged that lockdowns and homeworking also meant researchers had "kids at home or family that were sick that needed looking after".

From the Scottish Government point of view, project monitoring and reporting during the extension year was seen as "relatively smooth for everyone". RESAS officials were praised for their understanding of the difficulties in conducting field and lab-based science during the Covid-19 lockdowns, with one RESAS interviewee acknowledging that "there was a general appreciation that peoples' lives were being turned upside down". Where work could no longer be delivered, RESAS "tried to not be punitive and tried to find a middle ground", whether through "not pushing [research providers] to deliver undeliverable tasks" or "pushing them in the direction of write-ups of other work when they are unable to do other tasks".

EU-exit: Communications and Relationships

EU-exit increased workloads across the Scottish Government, with one interviewee commenting that "EU-exit meant we had a lot of difficulty getting hold of people in policy [teams]". A RESAS official noted that their colleagues in policy teams "were busier than ever, and it hasn't slowed down; legislatively it's still busy due to EU-exit".

Previously constructed personal relationships were identified as being important; with one RESAS stakeholder acknowledging that it was likely harder "if you're a researcher who doesn't have a relationship [with the Scottish Government science advisors] already" to get in contact with the Scottish Government, due to the increased workload across the Scottish Government.

EU-exit: Staffing and Resourcing

Numerous MRP interviewees identified EU-exit as being a significant contributing factor to losing staff. One MRP stakeholder stated "a couple of EU nationals on our teams went home… EU-exit is a difficult one", whilst another noted that "we lost a few members of staff due to this [EU-exit]". The impacts of EU-exit were also acknowledged as being ongoing and likely to persist, with one interviewee stating "it is having more effects now; with fewer Europeans wanting to come to the UK". The loss of staff, and the unavailability of new staff, "affected morale, which in turn, affected delivery [of research]".

EU-exit: Further Impacts (i.e. EU grants and EU-exit research)

Another EU-exit impact identified by four stakeholders were challenges relating to potential problems with EU grants. One commented that there were "questions about whether we are still in Europe or not, and if we can apply for funding", whilst another noted that EU-exit "heavily impacted our externally funded projects [ones funded by the EU] and our ability to recruit". Another interviewee acknowledged they had some other "challenges over EU-exit, things such as importing goods, importing and exporting biological materials".

Of the seventeen MRP stakeholders interviewed, seven interviewees commented on whether they had worked on projects relating to EU-exit during the extension year, with three saying they had not, and four saying they had. For those that did not, it was stated that "studies regarding EU-exit did not really occur" and that "our research was scientific, and pointed at scientific outcomes, not policy outcomes… I don't think anything we did in year six was really relevant to, or impacted by, EU-exit".

4.5.8. Extent to which the extension supported the recovery of research and experiments that had been affected by the pandemic

All interviewees emphasised how fundamental the extension year was for the continuation of the research projects under the Programme; which without, projects would not have been able to be completed and staff retained.

Staff inputs in the extension year

The 2021/22 extension year employed less FTE-equivalent staff than previous Programme years, with a total of 267 staff. This is in comparison to 354 staff in the first two years of the Programme's original timeframe (2016/17 and 2017/18), and 289 in the fifth year of the Programme (2020/21).

Of the 264, 188 were employed via the SRP, 44 via the CoEs (of which 24 worked at HEIs), 34 via Underpinning Capacity and one through the SEFARI gateway.

External funding for research in the extension year

In the extension year, the six MRPs received £18m from such sources. For Hutton, 'other sources' (excluding levy boards) provided the most funding (over £2.1m) whereas for SRUC, Moredun and Rowett, the most important source was the Research Councils. For BioSS, the RESAS Contract Research Fund or another Scottish Government source was the most important, providing £0.38m (40%) of the organisation's £0.95m of external non-industry funding.

4.5.9. Effectiveness of the extension year

Scottish Government, CoE and ARP interviewees noted that research leads seized the opportunity to reflect on lessons from previous years and implement changes in the extension year. For example, research teams reviewed and enhanced their approaches in response to the disruptions, such as closer working with HEI champions and enhanced digital communications (e.g. website publications). One interviewee noted that the extension year played a pivotal role in retaining staff within institutions, particularly where funding was received on an annual basis (without extension year funding, roles may have been unfunded). Conversely, another interviewee highlighted that their CoE had previously funded post-doctoral roles as part of the Programme on an annual basis; applications for these roles in the extension year were few as there was no security of funding going forward.

Other interviewees highlighted that the extension year was effective as it prevented overlap between the 2016-2021 and following Programme. One interviewee noted that without the extension year they would have been unable to apply for funding in the next Programme due to condensed workloads. Several MRP interviewees did, however, experience difficulties when writing applications during the extension year due to the continued need to concurrently deliver the end of the 2016 Programme.

More commonly interviewees felt one of the key impacts of the extension year was the ability to continue research beyond the scope of the 2016 Programme; one interviewee noted work was utilised at the COP26 conference as a result. Other RD leads and coordinators highlighted that continued project work in the extension year resulted in strengthened relationships with wider partners which were brought forward into the following programming period (2022-27). This is key as the interviewee noted a lot of resourcing/experience had been lost.

Overall, stakeholders expressed how challenging it had been to work during the pandemic but said they were surprised how resilient their teams had been. One interviewee specifically highlighted something less tangible which was the pride that institutions felt in their response to the Covid-19 pandemic.