Environmental, natural, resources and agriculture research programme 2022–27: commissioning evaluation

1. Introduction

1.1. Background

The Scottish Government (SG) Rural and Environment Science and Analytical Services Division (RESAS) will invest almost £50 million a year over the next five years in scientific research in line with its Strategy for Environment, Natural Resources and Agriculture (ENRA) Research^[1] for 2022–2027. The Strategy was published in March 2021 and the research programme launched in April 2022.

As described in the Strategy, its main goal is to support policymaking within the SG with robust and relevant research and evidence. A wider intention is to enhance the continuity of long-term research programmes and science facilities. This will enable the Scottish academic base and research institutes to participate further in UK and international funding schemes.

The Strategy was built on findings from consultation activity with stakeholders such as the Strategic Advisory Board, universities, research institutes, non-governmental organisations (NGOs) and local authorities. The three main drivers for the Strategy development emerging from the consultation include:

1. Global climate change and crisis – linked to achieving net zero greenhouse gas emissions and decisions that will enable that in future, e.g., land use and changes in environmental and agricultural policies.

2. EU exit – transforming the UK's trading relationship with the EU which will impact on, among other sectors, Scottish agriculture and food and drink businesses, and therefore the research programme that is subject to this tender.

3. Sustainable economic growth and wellbeing – with the focus on the recovery from the impacts of the COVID-19 pandemic.

Considering the recent years' events, including the Net Zero Strategy publication, UK's exit from the EU, and the COVID-19 pandemic, the research programme seeks to be dynamic and adaptable to the current and anticipated conditions of the Scottish and wider UK economy and policies. It is important to note additional challenges impacting the economy, which the ENRA research programme funded projects are potentially addressing: supply-chain issues caused by the COVID-19 policies of China and the effects of the ongoing war in Ukraine.

There are five key themes within the research programme which are aligned to the SG's policies and national outcomes:

Most of the research funding (88%^[2]) has been granted to six Main Research Providers (MRPs)^[3] :

1. The James Hutton Institute;

2. Biomathematics and Statistics Scotland (BioSS);

3. Scotland's Rural College (SRUC);

4. Moredun Research Institute;

5. Royal Botanic Garden Edinburgh; and

6. The Rowett Institute.

The focus of this evaluation was the Programme Commissioning Process (PCP), which is the journey from Strategy publication (March 2021) to the research programme launch (April 2022) and which has involved six main steps:

1. Creation of research questions and strategy themes – projects within Strategic Research Programme are due to answer at least one of the research questions and fall under one or more of the specific Research Themes.

2. Contract creation and contract types –like in the previous programme, Strategic Research Programme and Underpinning National Capacity projects have been commissioned through Invitation to Grant Funding, however unlike the previous programme the Centre of Excellence projects have been commissioned through Open Competition.

3. Invitation to tender – guidance for MRPs on the application process and expectations.

4. Peer review process – research proposals were peer reviewed for relevance, excellence, and impact.

5. Principal Investigators (PI) appointment – MRPs were responsible for identifying the lead PIs within the funded proposals.

6. Project start up – meetings between the SG and PIs to discuss the project details with the aim of developing solid working relationship and shared understanding of the project goals.

More precisely, the evaluation focused on the following elements of the ENRA Programme:

• Strategic Research Programme (SRP),
• Underpinning National Capacity (UNC),
• Centres of Expertise (CoE), and
• A comparison of the latest and previous PCP.

1.2. Evaluation aims and objectives

The overarching aim of this evaluation was to explore and understand what has worked well and what has not worked well, what improvements could be made, and overall perceptions of the Programme Commissioning Process.

Specific objectives of the evaluation were to:

• Understand what can be learned from the delivery of SRP, UNC and CoE;
• Identify what has worked well/less well in the delivery of SRP, UNC and CoE;
• Identify what can be improved within the Programme Commissioning Process;
• Explore perceptions of MRPs and the SG topic and policy leads, as well as factual details from documentation the SG shared with Pye Tait Consulting at the start of the project, relating to the PCP process.

Additionally, the evaluation took into account changes implemented in the 2022–2027 programme, compared to the delivery of the 2015-2022 programme, e.g., CoE being commissioned through Open Competition as opposed to Invitation to Grant Funding.

Process evaluation research questions were as follows:

• Delivery in practice, i.e., what has worked well, were there any gaps, what improvements could be made in future?
• To what extent has the change in the way the research programmes have been contracted (compared to previously) made a difference to ease of engagement?
• How clear was the guidance provided to MRPs, could anything be improved?
• How well has the peer review process worked? Were there any gaps or improvements that could be made in future?
• What are MRPs views on PI appointment/what has worked well/could be improved?
• To what extent have the Scottish Government and PIs established good working relationship? What improvements could be made in future?
• What challenges/sensitivities are being felt and how are these being managed?
• How does the ENRA PCP compare to other commissioning and grant processes? What are similarities and differences?

1.3. Methodology

The methodology of this process evaluation comprises three key stages:

1. Evaluation scoping – to understand the process by which the activities of the PCP have been implemented; to understand the objectives of the PCP. This was done via a project inception discussion with the Scottish Government followed by a Rapid Evidence Assessment (REA). As part of REA, we assessed a number of documents, including tender guidance for MRPs, project proposals, the policy research questions and guidelines for peer reviewers.

2. Primary research – it entailed confirming the evaluation approach, evaluation questions, developing interview topic guide and undertaking in-depth interviews. We conducted a total of 35 interviews. Stakeholders we interviewed are SRP, UNC and CoE applicants, i.e., MRP Co-ordinators, and PIs, peer reviewers and the SG internal topic and policy leads.

3. Analysis and reporting – it involves qualitative analysis of REA findings and conducted interviews followed by producing a draft and final reports.