Farm Advisory Service - One to Many: evaluation

1 Executive Summary, Plain English Summary and Recommendations

1.1 Executive Summary

This report provides an evaluation of the Farm Advisory Service: One to Many service (FAS OtM). This report complements an evaluation published by RESAS of the FAS One to One service in 2019. The FAS OtM service was procured by Scottish Government as part of the broader Scottish Rural Development Programme (SRDP) 2014-2020, and seeks to improve the business and environmental performance of Scottish Agriculture through the provision of advice. The OtM component of the service comprises a range of features, including in-person events, providing information on the website, providing specialist support via the advice line, delivering a small farm and crofter subscription service providing discounted advice and a range of other outputs concerned with addressing a mass audience (i.e. podcasts, videos, social media, and downloadable technical publications).

The overall goal of the FAS service is: "Encouraging sustainable growth and broadening opportunities that will help create a more competitive dynamic agricultural sector that contributes to the long term viability of rural communities while maintaining high standards of animal health and welfare and environmental management."

The FAS OtM contract further specifies the following aims for the service:

• Knowledgeable, multi-skilled farmers and crofters capable of delivering business, societal and environmental benefits within a complex legislative and physical operating environment.
• Greater uptake of agricultural practices that deliver increased economic performance whilst mitigating against climate change and enhancing biodiversity status.
• Increasing numbers of dynamic young people successfully entering Scottish agriculture, thereby injecting vigour in the development of thriving farm and rural businesses.
• Easy access to up-to-date and relevant knowledge and information to all farmers and crofters through a network of advisory centres, on-line resources and a telephone advice facility.
• Land managers using key national performance metrics (benchmarks).
• Improved uptake of integrated methods of managing plant and animal health, including the safer use of and reduced reliance on pesticides and antibiotics.
• Improved water quality through a reduction in diffuse pollution and run-off.

A survey was undertaken that 148 FAS users responded to, and eight agricultural stakeholders and two participants from FAS were interviewed. Overall, chapter 2 indicates that SAC Consulting have fulfilled their contractual obligations with regard to the delivery of FAS. Highlights of the delivery have included:

• Delivering over 800 events over a range of geographical locations, with consistently high feedback. As many as 15, 656 people attended these events between 2016/17 and 2019/20.
• Consistently high satisfaction ratings and intention to change reported by 85% of event attendants.
• Provision of a small farm and crofter subscription service, providing subsidised advice to 2, 188 crofters and 287 small farms in 2019/20.
• Delivery of numerous Women in Agriculture events and events supporting New Entrants over the course of the contract.
• Providing technical information, including a Farm Management Handbook. Between January 2020 and August 2020, 108, 674 technical documents were downloaded
• Increased engagement with farmers through social media, website, podcasts, videos and newsletter over the course of the contract.
• An increase to 934 calls to the helpline in 2019/20 from 299 in 2017/18.
• Consistent achievement of Key Performance Indicators (KPIs)
• Our survey also found high satisfaction with the service and evidence of changes on farms as a result of engagement with FAS, including monetary savings, environmental improvements, improved profit margins and a reduction in the costs of inputs.

Overall, there is clear evidence that the FAS OtM has delivered a wide-ranging programme which, insofar as we have data, appears to be well-regarded by those who use it. Given the relatively recent development of this service, this is positive. However, a challenge for the evaluation has been that, while a number of KPIs are specified, the delivery of advice via the OtM service is not accompanied by impact related goals or outcome-based monitoring. Moreover, as we do not have population level data about the use of the FAS among farmers, it is challenging to compare those who have received advice to those who have not. This makes it difficult to demonstrate the impacts of advice directly, and the scope for addressing this in the future is discussed in the recommendations of this report.

In terms of structure, this evaluation provides an overview of the outputs from the project (chapter 2), a survey of users to identify benefits and challenges (chapter 3) and interviews with key stakeholders and the delivery partners about the service (chapter 4). This data allowed the evaluation to identify where challenges in the FAS delivery model have been observed and provides discussion around how these challenges might be addressed. The final chapter provides further discussion of how the logic model of FAS might be developed going forward.

It should be emphasised that the recommendations for developing FAS further should not be understood as a critique of the service provided. Rather, it is more a matter of identifying the ways in which the service could develop in the future and build on the work that has already been done. Both the quantitative and qualitative data suggests that, irrespective of how effectively information has been delivered, farmers may face substantial barriers in engaging with and following advice. There are also complex questions about how these changes can best be monitored and coordinated with a view to supporting sectoral transformation and a response to the climate emergency. The evaluation seeks to engage with these questions, and can be understood in this context as seeking to enhance the delivery of advice.

1.2 Plain English Summary

What is the Farm Advisory Service?

The first and current version of the Farm Advisory Service (FAS) was launched in 2016 and ran until December 2020. This was delivered as part of a number of European Union funded agricultural and rural development programmes in this period. The advice service provides free advice to farmers to improve their economic prospects and supports efforts to make farming more environmentally sustainable. Farming Advice of this sort is provided in European Member states, in part to support compliance with EU rules, although the nature and type of delivery vary.

How is FAS Delivered?

Delivery of Farm Advice is split into 'one to many' and 'one to one' components. One to one advice is concerned with farmers developing plans with advisors to implement changes on their farm directly and was evaluated by Scottish Government in [2019]. One to many (OtM) advice provides a range of services including in-person and online events, providing information and briefings online, responding to technical enquiries and producing content online to support the service's goals. It is the one to many service – delivered by SAC consulting - that is subject to evaluation here.

What is the purpose of this Evaluation and How was it Done?

The evaluation took place to fulfil the requirements of EU funding and consider how the Scottish Government should approach farm advice in the future. The key questions were: what has been delivered by the one to many service, what are the impacts of this, and what improvements may be needed going forward? To answer these questions, SAC's reports and data were analysed and a short survey of FAS users and ten interviews with key stakeholders were carried out.

What did the Evaluation Find?

The evaluation found that SAC Consulting had delivered a wide ranging service that fulfilled their requirements. Benefits included a large numbers of events, high satisfaction among attendees and increased use of FAS services over the contract period. However, the evaluation also notes that, at present, demonstrating the impacts of FAS OtM on farms is challenging, as there has been limited monitoring of the extent to which farmers use the service as well as limited monitoring of on-farm improvement. Going forward, identifying outcome based measurements to demonstrate the impacts of advice, improving the monitoring of FAS use among farmers and refreshing the 'mission' of the service are all recommended as methods to address these issues. In addition, it is recommended that techniques are developed to improve the engagement of those currently not making use of FAS and to support on-farm changes among those facing barriers to doing so.

Who is this Report for and What are the Next Steps?

This evaluation will inform ongoing policy discussions about how to take forward the delivery of farming advice in the post-Brexit policy context. While the main purpose of the report is to inform policy making, it may also be of interest to those concerned with how the Farm Advisory Service is delivered at present and in understanding the challenges and benefits of the current delivery method. FAS will remain co-funded by the EU until March 2021 and then will be fully nationally funded until December 2021.

1.3 Recommendations

In the main report, the recommendations below are restated in the main text in greater context. However, a summary of the eight main recommendations is here:

Recommendation 1: Review Monitoring Framework. Review the KPI arrangements for a future service and consider the possibility of developing outcome based KPIs. Review the options for establishing effective monitoring of farm improvements, ranging from centrally set and monitored goals to goals established and monitored by individuals or groups of participants.

Recommendation 2: Review Data Strategy. Consider potential mechanisms for regular, representative farm data collection to determine the extent to which FAS is used within the farming population as a whole and how the service is viewed. Consider opportunities for integrating monitoring of farm environments within a broader environmental monitoring strategy. Consider the possibility of a specific farm data strategy for monitoring and benchmarking environmental impacts.

Recommendation 3: Review Engagement Strategy. Consider contractual mechanisms that support the goal of engaging with those who are under-represented in FAS, and the possibility of requiring a detailed engagement strategy that examines subjective and structural barriers. Review whether future farm advice should develop an appropriate customer management system that allows monitoring that tracks additional advice and engagement from customers, to monitor crossover between the different components of the service (in the event that delivery continues to be separated into 'one to one' and 'one to many' components).

Recommendation 4: Review the 'mission' of the service. Consider the value of establishing an updated 'mission' for the service, using a participatory mechanism to ensure wide cross-sectoral buy in. This should be cognisant of the climate emergency and the need to support nature in farming.

Recommendation 5: Review Knowledge Integration: Review mechanisms for knowledge exchange to ensure there is a consistent approach to climate change and environmental practice both on and off farm, potentially incorporating knowledge exchange initiatives like SEFARI, the website that hosts the outputs from publicly funded research into food and agriculture.Similarly, consider the mechanisms for greater integration of FAS in relation to the broader farming advice context, and ensuring specialist knowledge is available and integrated into service provision.

Recommendation 6: Consider Scope for More Holistic Training Integration. Consider whether there is scope for more holistic integration of training with advice provision. Taking a long-term view, consider the scope for FAS to engage with longer-term training and advice mechanisms.

Recommendation 7: Ensure advice is inclusive. Consider the best mechanisms to mainstream the lessons of women-only training techniques, how best to ensure they are available and review barriers to participation that may exist for other equalities groups. For sensitive topics, for example, succession planning and mental health, ensure that FAS can provide an appropriate forum for discussing these sensitively.

Recommendation 8: Engage with Barriers to Following Advice. Consider developing mechanisms to cultivate small, facilitated groups of farmers which can collaboratively develop change over time. A common view among interview respondents was that achieving change is easier in the context of small groups of farmers, rather than individuals, and this should be considered as a mechanism for improving the take-up of advice.