Scotland Rural Development Programme 2007-2013: ex-post evaluation

3. Answers to Evaluation Questions

3.1 Introduction

Chapter 3 addresses all relevant Programme-level evaluation questions including the Common Evaluation Questions (CEQ) and draws on primary research and desk-based research findings.

The various topics are discussed on an axis-by-axis basis where appropriate (further detail can also be obtained by the appended Topic Guides for each Axis in Appendix C ).

3.2 Relevance of the Programme Interventions

3.2.1 Relevance - Axis 1

The rationale for Axis 1 interventions, particular with regard to the need to modernise and add value to agricultural and forestry products was maintained throughout the SRDP implementation period.

In terms of financial allocation, modernisation and adding value to agricultural and forestry products were the most dominant Measures in Axis 1 (together accounting for 92% of all Axis 1 public spend). This focus demonstrates the strategic relevance put on supporting transformational change and improving the sustainability of agricultural and forestry holdings. This emphasis on modernisation and restructuring was felt to have been vitally important in supporting rural businesses through difficult and uncertain times when confidence levels were low. In this context, stakeholders observed that supported rural businesses were able to develop better because of SRDP funding than those not supported.

Axis 1 Measures also sought to bring about positive environmental effects (water quality, slurry management, energy preservation and others), while relevant, it was difficult to attribute the extent to which the SRDP actions contributed to the positive developments that were experienced in view of other, more prominent support schemes (Feed in Tariffs (FiT)), none the less, in principle Axis 1 and Axis 2 interventions complemented each other in these areas.

Axis 1 was taken up and spent by more than the expected numbers of beneficiaries, demonstrating that the Programme was designed to meet demand. In addition, the relevance of a number of schemes such as CCAGS and SDS were commended for their demand-based and industry-driven design.

Nevertheless, many believed that the SRDP was still too driven by the desire to disburse financial resources and lacked focus to help rural businesses, farm and forestry holdings to adapt to change, increase their business viability and self-sufficiency. One of the biggest challenges was considered the difficulties in accessing the SRDP, particularly through one of the largest schemes, the RP. Because the application process was so difficult, stakeholders felt that mainly those rural businesses with the sufficient expertise and capabilities and/or financial means to pay for agents to do the application for them were actually able to access funding. This was widely reported and must have affected the Programme's strategic relevance regarding smaller farm and forestry holdings and other micro rural businesses in the Programme area.

Although Training (Measure 111) and Setting up of young farmers (Measure 112) were identified as key weaknesses in rural Scotland, the budgets for these Measures were drastically cut following the Government Spending Review and instead of 500 only 51 young farmers were set up thereby missing out substantially on the originally intended positive change. Thus the strategic relevance of the SRDP on encouraging young farmers was significantly reduced.

In the case of Training (Measure 111), the original rationale was based on delivering in-depth training presumably to achieve lasting impact on abilities and capacities. However, monitoring data show that the Measure was implemented with a significant number of beneficiaries accessing a minimal amount of training. It is questionable how this light-touch approach could have achieved lasting change. However, stakeholders did report positive impacts from the Monitor Farms and that relevant information was disseminated to a wide range of people.

Lower than expected up-take of some Axis 1 interventions, particularly by rural businesses was widely explained by economic factors, such as the recession, economic uncertainty and risk-averse financial lenders. Although the farm and forestry sectors were less negatively affected by this during the earlier years of the SRDP, during the later years product price volatility and rising fuel prices led to substantially falling income levels and increased need for support. While these developments emphasise the continued relevance and need for SRDP support, the positive change that had been generated by the Programme was reduced to some extent by global market developments.

3.2.2 Relevance - Axis 2

The objectives and Measures in Axis 2 were an appropriate response to widely recognised environmental problems in relation to rural land in Scotland.

There has been a long term decline in farmland biodiversity mostly as a result of more intensive farming practices and loss of fringe and semi-natural habitat, but occasionally as a result of abandonment, which could, for example, have influenced the decline of some moorland waders. More widely evident but less well targeted by the SRDP has been a diminution in landscape quality as a result of degeneration of e.g. shelter belts and poor field boundary maintenance, especially of drystone dykes. These landscape features also provide habitat connectivity.

With LFASS, the largest Measure of Axis 2 (€520m spend representing 75% of Axis spend), the SRDP continued with a long established and popular funding scheme. LFASS comprised an income transfer to farmers on poorer land but it was thought of as lacking in directing SRDP monies more directly towards intended activities. Without monitoring in place to inform about any effects of the investment, this is a distinct area of missed opportunity to monitor and assess change created. Grieve et al. 2016 highlights that a number of key stakeholders questioned the Scheme's utility as it was regarded as too blunt and ineffectual as an environmental policy tool. Notwithstanding this, LFASS was part of the support of High Nature Value (HNV) farming (Type 1 HNV regarding rough grazing).

Water and soil can both be adversely affected by farming and forestry practices. Remediation strategies are well known with respect to water and normally entail better protection of watercourses through fencing and buffer strips, reduced and more timely fertiliser application and better manure storage and more timely application. Manure storage was covered under Axis 1, but other water-related Measures were found in Axis 2. Soil quality has been compromised by intensive cropping and erodible soils are still used for intensive crops with resultant high levels of soil loss in some catchments. Water quality, driven by commitments towards water quality improvements in the Water Framework Directive, was much more effectively targeted in the 2007-2013 SRDP than soil quality.

Animal health and welfare Measures could only contribute a little to headline environmental problems associated with land use and the Scottish environment with the exception that better livestock health will almost certainly reduce greenhouse gas (GHG) emissions per unit of output.

In relation to forestry, there is an evident lack of a farm forestry tradition in Scotland compared to continental Europe and a legacy of rapid often mono-cultural afforestation by the state and private foresters in the 20 ^th Century when major state or grant-aided private afforestation occurred primarily for strategic reasons. The SRDP 2007-2013 provided substantial funds for afforestation on farm and other land and for environmental enhancement of forestry. Over the funding period and in response to major flooding events, some authorities argued for afforestation to mitigate flooding and pilot projects have been established.

The major environmental needs confronting the rural land use sector were largely recognised in the SRDP 2007-2013. The major gaps were with respect to soil protection and enhancement, flood mitigation where land use was implicated as a causal factor and in relation to [the lack of] any strategy in the SRDP to address farm level GHG emissions. Because the environmental problems associated with rural land use are highly varied over space, the use of targeting can be seen to enhance policy impact. LFASS and some biodiversity Measures were targeted, as were water quality-related Measures, but the degree of targeting as a whole was limited with consequential loss of impact.

To a considerable extent, many of Scotland's environment-related problems associated with land use were addressed by the SRDP 2007-2013. The principal changes have been with respect to successful engagement with recovery strategies for some bird species such as corncrake, chough and corn bunting and with addressing water quality issues in priority catchments. The desired changes in forestry were arguably already moving in the right direction. HNV land has been mapped in this policy period, but the procedures lacked the sophistication of procedures in many other EU countries and provided little support for associated targeted interventions.

The activities and outputs of Axis 2 were broadly consistent with the strategic objectives of the Programme, although there are still significant outstanding needs to be addressed in relation ensuring high nature values are maintained in HNV farming areas; supporting biodiversity enhancement, protecting water quality, protecting soil and addressing land-based GHG emissions.

3.2.3 Relevance - Axis 3

The main focus and primary objective under Axis 3 was to achieve thriving rural communities and support aimed to achieve this by diversifying rural enterprise, facilitating sustainable growth in the rural economy and generating employment opportunities beyond the land-based industries. The overall approach to the Axis was therefore integrative, encouraging applicants to address rural issues from different angles in support of each other. It was originally designed to pursue four identified priorities. These were to:

• add wider value to rural goods and services, including tourism;
• build capacity in local communities;
• promote public enjoyment and understanding of the countryside; and
• support rural services and infrastructure at a local level.

The central focus on rural tourism and other rural enterprises is reflected in the final budgetary allocations; Measure 313, Encouragement of tourism activities being allocated almost half of the final budget for Axis 3 (49%) with complementary business development support, Measures 311 Diversification and 312 Business creation accounting for a further 26% of the final Axis budget allocations.

The intervention logic underpinning this was to address the identified weaknesses in rural tourism by seeking to improve the attractiveness of the rural area and thereby encourage tourism activity. In so doing the dual focus of the Axis on diversifying the rural economy and improving the quality of life could be linked and addressed in tandem.

No significant changes in rural development policy affecting the logic for Axis 3 are recorded in the Annual Reports other than a renewed or strengthened focus at Scottish and European levels for supporting economic growth and sustainability.

The SRDP First Stage Review and the MTE confirmed the continuing policy relevance and strategic fit of Axis 3 with no major changes suggested. In addition, stakeholders and SRDP beneficiaries consulted during this ex-post evaluation strongly confirmed the ongoing relevance of the focus of support and stressed the importance of the availability of such support to rural businesses when facing the uncertainties and challenges which emerged through the Programme period.

The Spending Review and adjustments to Sterling following the 2008 financial crisis saw changes to the Axis 3 allocations by Measure and an overall Axis level reduction to 49% of the original budget. As noted in Chapter 2 the final total public expenditure for Axis 3 failed to meet the 10% minimum specified in the EAFRD Regulation and possibly suggests a lessening of the strategic importance accorded to the Axis.

The changes in budgetary allocations within the Axis demonstrated a proportional strengthening of focus on tourism and diversification through Measures 313 and 311 and the economic recovery outcomes sought. There were also significant shifts within elements of the Quality of Life objectives in particular in strengthening the focus on conservation and upgrading of the rural heritage.

The subsequent achievement of spend against the allocations suggests that Axis 3 was meeting demand from the targeted sectors. With the exception of Measure 313 (85%) all exceeded 100%. This suggests that programming had targeted clear development needs. The increases in the number of beneficiaries or interventions targeted under Axis 3 Measures were in line with the shift in Scottish Government policy towards targeting smaller businesses.

As a result, the way in which support was delivered perhaps did not reflect the limited effects of the economic crisis on land owners / managers, the influence of other sectoral factors and the changing realities of the rural tourism sector and market. Those able to access and absorb support were relatively well established businesses. This appears to be reflected in the uptake of Measure 313 in terms of number of activities supported which remained very low at 26% of its target although the Measure achieved 85% spend of its budget. There was a risk the approach to support under the Axis was demand, input and output led (and spend focused) rather than strategically driven by the outcomes sought.

As in Axis 1, the delivery mechanism, RP in particular, had difficulty in reaching the targeted beneficiaries whilst other, much smaller Schemes were much more popular in implementing activities in the area of conservation and basic services. Taken at face value these figures are contradictory with the allocation of the Axis 3 budget and could suggest that the logic was not appropriate to the needs of the rural area.

In retrospect stakeholders and beneficiaries consulted suggest that although the Axis intervention logic remained sound, the adjustments to allocations and targets were more a response to uptake and demand considerations rather than policy shift or proactive redirection to identified needs. The logic as designed was sound but not as delivered. In particular prioritisation through the RPAC approach could have been stronger.

In reality the key contribution required became more about sustaining, improving and extending existing enterprises and rural tourism provision, businesses and employment rather than creating new enterprises or activities. A number of stakeholders felt that the approach should have focused more overtly on contributing to the resilience and adaptability of the rural business sector and the contribution of this to thriving rural communities.

There was also some lack of clarity amongst stakeholders and beneficiaries over the respective roles and focus of LEADER and the Axis 3 Measures. This was also reflected by the Ex ante working groups for the new Programme which identified Axis 3 as overlapping with LEADER (Final Report of the Rural Economy Working Group p.10). There were 20 LEADER local development strategies all of which addressed the intervention logic of Axis 3 to some extent. However, rural stakeholder's awareness that LEADER funding was in fact SRDP funding was often low (primary research findings).

3.3 Effectiveness and Achievements

An overview of achievements are presented in Chapter 2, highlighting that more than the anticipated number of people benefited from accessing the Programme. One of the key challenges of the SRDP was target setting and monitoring to clearly assess the effectiveness in which the Programme's aims were achieved. Primary research undertaken throughout the Programme period has helped, but was usually narrowly focusing on the GVA results indicator and economic impact assessment. To improve the lack of data capture for agri-environmental interventions, a number of studies were commissioned (see below regarding Axis 2).

The Programme sought to achieve five principal outcomes:

• improved business viability;
• enhanced biodiversity and landscape;
• improved water quality;
• tackling climate change; and
• thriving rural communities.

Consultations with stakeholders found that the SRDP was perceived to have contributed positively to the five principal outcomes - albeit that for outcomes such as climate change and water quality it was impossible to identify the extent of the Programme's contribution due to a multitude of other initiatives taking place at the same time (climate change) and legislative regulations having a substantial impact on positive change (Water Directive). At the same time, it was felt that the SRDP investments played an important and relevant part in supporting and complementing other actions.

Findings further found that across the various Axes that SRDP investment had a positive effect on improving the business viability as well as the sustainability of rural communities (specifically through LEADER supporting better partnership working and integrated action).

In terms of supporting an improved regionalisation of SRDP interventions to enhance the focus on local needs and encourage collaborative action (water quality, waste management, and slurry storage), the SRDP made a positive attempt by the creation of RPAC regions working across the various Axes of the Programme. However, due to only limited decision making power, the RPACs were largely regarded as ineffective mechanisms and were not carried forward into the new Programme.

Regarding the Programme's efficiencies in meeting client expectations, Appendix D presents the various satisfaction levels of beneficiaries with the SRDP in more detail. For example the LMO application process was considered easier than that of the RP. Those schemes consistently scoring well included: CCAGS, FPMC, LFASS, LMO and SDS. ( Appendix D , p. 2).

The desire to create an online application system for all areas of the SRDP was achieved to only a limited extent and often thought as overly ambitious. This was particularly so in the context of RP where the sheer number of options plus the required computer literacy of applicants proved too demanding and potentially excluded some of the less IT literate potential applicants.

3.3.1 Effectiveness and achievements - Axis 1

The key outcome pursued by the SRDP relating to Axis 1 was to improve the business viability of farm and forestry holdings. The SRDP put substantial focus on the modernisation (75% of all SRDP investment of Axis 1 was dedicated to support restructuring and diversification). In this context, primary research findings confirmed that the schemes such as RP, FPMC, as well as CCAGS have made positive contributions in creating and safeguarding jobs. It was agreed by most stakeholders throughout the Programme period that SRDP was instrumental in sustaining employment in the agricultural holdings in particular.

Other aims identified for Axis 1 were substantially reduced during the Programme period and only attracted limited numbers (young farmers) or implemented through more light-touch engagement with participants (training).

Some of the services originally anticipated to be delivered by the SRDP (Advisory Service) were removed from the Programme. In the view of a number of stakeholders it might have been more effective to keep or re-model the SRDP advisory services particularly in view of up-take issues early on in the Programme implementation period.

A number of stakeholders felt that the SRDP could have been more effective particularly by linking training events better with business needs and explaining better how economic actions can relate or contribute to environmental actions. This would have supported the integrative approach in rural development better. Here, it was believed that either better advisory services or the requirement for a business plan could have improved the effectiveness and targeted approach of the Programme.

Although generally the effectiveness of the SRDP in targeting key sectors (food and drink) and business needs was regarded as good, there were continuing needs particularly in widening the definition of young farmers and to make future applications less complicated where interest rate relief was concerned.

3.3.2 Effectiveness and achievements - Axis 2

The objectives relating to Axis 2 were partially achieved. There were some areas where clear environmental benefits arose from interventions. However, the ability to assess whether the Programme objectives were achieved is compromised by the weak baseline data, the absence of counterfactuals and the highly interconnected nature of many of the Measures which could easily have had impacts on targets in several Axes.

Over and above the achievements with respect to individual Measures, key stakeholders were strongly of the opinion that the drawing together of SNH's Natural Care Scheme, with afforestation support and prior SRDP approaches into an integrated package created greater coherence in the delivery of environmental services with respect to rural land use.

As mentioned before, LFASS should have had a stronger and monitored environmental emphasis. The assertion that LFASS supports farming and thereby biodiversity and cultural landscapes in generally disadvantaged land types areas was reasonable, but over the period of the SRDP, the farmed area declined, as measured by the area in receipt of Measure 212.

It is extremely difficult to argue for the preservation of semi-natural habitat without acknowledging that land that passes out of agricultural production still delivers ecosystem services related to semi-natural vegetation cover (for a wider discussion of LFASS issues see Grieve et al. 2016) ^[10] .

The extremely large application menu of possibilities (options) regarding biodiversity and the highly variable uptake led to some species being significantly better protected, while others were little better protected. Indeed, a growing area of concern is the plight of upland waders that are dependent on farmland. ^[11] It would appear likely that a number of factors such as ease of engaging with the actions, prior engagement with agri-environmental Measures, presence of advisory support and actual levels of support mediated levels of uptake.

The number of holdings drawing down animal welfare payments (Measure 215) was less than half that of the target, though the number of contracts was much higher and exceeded targets. Enterprises' performance varies hugely as evidenced by Quality Meat Scotland. However, improved animal health and welfare is highly likely to be a contributory factor within this and such welfare and health improvements are thus essentially a productivity enhancing intervention. Given the narrower than anticipated uptake (in terms of numbers of holdings) improving animal health may never the less be considered a 'work in progress'.

The uptake by land area of first afforestation of farmland was below target but the Scottish Government remained committed to increasing forest cover on farmland. It sponsored a major inquiry into the topic in the Woodland Expansion Advisory Group ^[12] and has endeavoured to advance understanding of the barriers to afforestation of farmland ^[13] . Given the presence of other levers to enhance integration of forestry into wider rural land use, the separation of the SRDP effect from other confounding factors remained problematic.

The contribution of Axis 2 in the SRDP to EU rural development priorities was mixed. The LFASS scheme was regarded as a significant means of supporting rural employment. Biodiversity Measures have improved the condition of many protected areas and benefited certain target species and habitats, but overall the farmland bird index, the indicator of biodiversity impact, has declined over the period (having been in decline for some decades). Apart from the significant contribution to carbon sequestration from new afforestation, there were probably modest additional net emissions reductions arising from biodiversity interventions. A farm-based approach to emissions reduction did occur but out-with the SRDP. It has been argued that the SRDP could have done rather more in pursuit of land-based emissions reduction ^[14] . Some soils have been better protected as a result of inclusion of areas in biodiversity schemes, but large areas of arable soils remain with declining carbon content and compromised by erosion risk. Renewable energy output from farms has increased both through wind and water-based electricity production and through biomass for heating and grain drying. Although these have not been funded by the EC they help Scotland contribute to EU priorities.

3.3.3 Effectiveness and achievements - Axis 3

The EU priorities for Axis 3 are set out in Community Strategic Guideline 3 which states the overarching priority of creating employment opportunities which should in particular be used to promote capacity building, skills acquisition and organisation for local strategy development and also help ensure that rural areas remain attractive for future generations. Under Axis 3 of the SRDP this was expressed in terms of the objective to achieve thriving rural communities by diversifying rural enterprise, facilitating sustainable growth in the rural economy and generating employment opportunities beyond the land-based industries. Capacity building, skills acquisition and strategy development was addressed directly through LEADER, more so following the transfer of Measures 321 and 341.

In terms of achievement, Axis 3 Measures other than 323 underperformed to a significant degree in relation to the participant and activity indicator targets set. As noted in Chapter 2 there are questions over target setting. In addition, reported difficulties experienced by beneficiaries included accessing support due to the late start of the Programme, the lack of continuity in support from the previous period and the complexity of submitting applications under the RP as mentioned earlier. This was also influenced by the reported lack of clarity over the respective roles and focus of LEADER and the Axis 3 Measures.

As previously noted, uptake was also influenced by the impact of the economic crisis on non-agricultural rural business confidence and the changing realities of the rural tourism sector and markets. The high transaction costs of securing support was a particular challenge for the predominantly micro businesses targeted and might have limited the potential effectiveness of the SRDP.

The MTE survey of beneficiaries found that Axis 3 Measures generated many positive business effects in terms of the diversification of the rural economy, income, competitiveness and quality of outputs.

The ex-post primary research explored Axis 3 contribution both in terms of the relevant result indicators and through other Measures relating to the SRDP objectives. The outputs of this primary research indicate that Axis 3 was thought to have contributed positively to the objectives of the SRDP. The main contributions highlighted of relevance to Axis 3 were improvements to labour productivity and competitiveness (e.g. through the introduction of new equipment), safeguarding employment, (often self-employment or consolidating casual employment), increasing stakeholder capacities, creating economic growth (through quality and performance improvements), introducing innovation and improving the quality of life.

The beneficiary survey of Measure 321 (see Appendix E for the detailed findings) confirms the validity of this approach highlighting the surprisingly wide range of economic and environmental benefits achieved through targeting basic services.

Although these responses are not quantified other than in relative terms or directly attributed to Axis 3 overall, this suggests that the performance of Axis 3 was strongest in terms of contributing to the sustainability and competitiveness of businesses, employment and rural communities rather than new enterprises, additional tourists and tourism activities. Supported businesses were generally thought to have performed better in these respects than those not in receipt of support.

Respondents perspectives reflected the extent to which the key contribution required of Axis 3 had become more about sustaining, improving and extending existing businesses, helping them compete in a changing and more difficult market and sustaining employment. This appears to have been a function of the way in which the targeted beneficiaries were adapting to the changed circumstances and the way in which this affected the implementation of the support provided.

As Chapter 2 indicates, the performance indicators do not appear to provide a reliable measure of the real performance of SRDP achievement in delivering participation and development activity. The introduction of a suite of Programme specific result indicators would have enabled better reporting of the actual effects of the Axis in adapting and responding to needs, taking account of differences in demand (the nature of service sought) and the contribution to and delivering against the objectives and thriving rural communities.

It should also be noted that the performance of LEADER projects aligned with Axis 3 Measures was not reported against Axis 3 indicator targets. This missing contribution was identified throughout the Programme, by mid-term and ongoing evaluations, but remained unresolved.

3.4 Efficiency

Overall previous evaluation studies found that the SRDP was efficient in achieving results such as creating rural jobs and sustaining employment. The importance of the SRDP in sustaining and safeguarding jobs in difficult economic times was emphasised throughout the ex-post evaluation primary research as well. In terms of environmental results, previous research found that substantial areas of rural Scotland are now under positive environmental management regimes and progress is emerging in terms of cooperation and complementarity e.g. in developing links within and between holdings.

In many areas it is difficult to assess the extent to which results have been achieved, particularly regarding environmental improvement as monitoring systems were largely not in place at the time.

The positive factors influencing the efficiency of the SRDP are outlined as follows:

• The use of long established 'schemes' enabled applicants to relate to something known and already had the know-how for applying (LMO and CCAGS). These schemes were regarded as easy to access and apply to;
• Respondents were asked to gauge their satisfaction with twelve factors of the service offered by SGRPID and its partners ^[15] : The majority of respondents were found to be satisfied, with respondents most likely to report being satisfied with the helpfulness of staff, the speed and clarity in responding to enquiries involving contact with staff or RPID itself:

Negative factors regarding SRDP efficiencies included:

• The RP scheme was so complicated that many applicants required to commission consultants to apply on their behalf. (see Appendix D page 3, 90% of beneficiaries used an agent to apply for RP);
• Schemes such as LFASS (the largest single scheme within the SRDP) failed to apply further performance targets to the eligible beneficiaries, thereby missing the opportunity to at least report on the type of activities and outputs undertaken.
• The efficiency of the SRDP particularly in achieving its environmental impacts was impossible to assess for lack of monitoring and reporting.
• Frequent change of regulations or advice on reporting requirements made the SRDP in many cases a frustrating and expensive experience for applicants. Primary research findings identified this as one of the most negative side of the SRDP experience.

3.4.1 Efficiency - Axis 1

Primary research findings indicate that the aims and objectives of the SRDP, particularly regarding the introduction of innovative approaches and helping the agricultural and forestry industries to restructure and modernise were successfully achieved by increasing the business capacities and productivity.

Stakeholders (intermediaries as well as beneficiaries) also noted the positive impact of the SRDP investment on the supply chain, particularly the contractors to the farm and forestry businesses.

Findings from the ex-post evaluation stakeholder consultations indicate that the SRDP results impressed by the high number of people that benefited from support (as shown by the Results indicator tables in Chapter 2)

Evidence to assess whether results could have been achieved at lower cost is limited. A more detailed discussion of unit costs is outlined in Section 3.8.15.

3.4.2 Efficiency - Axis 2

The allocated resources have had discernible beneficial results in a number of fields. Questionnaire evidence from the MTE showed how LFASS contributed to job retention, although there is little evidence that LFASS provided beneficial environmental results and impacts (due to lack of monitoring). The perceptions of stakeholders is that the agri-environment options varied in their extent to achieve results. There has been a turnaround in numbers of some threatened species, although in the case of the corncrake there was a rather high level of expenditure to achieve those gains. Animal welfare gains have not been actively monitored and involved a much smaller number of farmers than the target. First afforestation involved more farmers but less land than was targeted with a resultant lower level of carbon sequestration. NPIs for water, coupled with slurry storage in Axis 1 contributed to water quality improvements in priority catchments, but attribution remains difficult, as other practices may well have changed as a result of SEPA advice.

Evidence to assess whether results could have been achieved at lower cost is limited and includes the following:

• LFASS operates like a direct income payment, and has low transaction costs. It might have reduced costs to have absorbed LFASS into the SFP/BPS;
• it is not possible to generalise about agri-environment Measures given the huge diversity of options/eligible operations. It is likely that the habitat improvements in some cases were relatively low cost. All support is based on income foregone on average farms, so any farm performing below the average should be a net beneficiary and anyone operating above the average is effectively under-rewarded. Rather than thinking in terms of additional cost for a particular field or part of the farm, other issues such as the overall functioning of the agricultural system and the extent to which there was rigorous policing of prescribed practices may also have influenced engagement. Overzealous attention to detail by officials was perceived as a deterrent to entry. Concerns were expressed about cost effectiveness in biodiversity related Measures with Austin et al. (2015) who asserted that there was "a need to focus not only on improving the cost-effectiveness of biodiversity conservation Programmes, but also to improve the robustness of cost-effectiveness assessments, in terms of data availability and accuracy and improved monitoring of the outcomes of interventions." ^[16] Further it is argued that in the absence of knowledge where highest levels of positive externalities were provided, there was no scope to steer resources cost-effectively in that direction;
• animal welfare gains were probably derived at reasonable cost although accurate data are lacking;
• NPIs for water quality enhancement were probably rather cost-effective;
• given the general reluctance of farmers to plant trees and the lack of rapid uptake of the Measure, it seems unlikely that the results could have been achieved at significantly lower costs, unless a more transaction cost heavy approach such as a tendering scheme was used. However, the loss of LFASS payments on afforested land means has created questions regarding policy direction; and
• forest environment Measures (225, 227) were probably achieved at a reasonably low cost.

The results tables show what has been achieved in relation to community priorities, but they fail to show clearly that the results arose directly and exclusively from SRDP interventions. Attribution remains problematic. LFASS has helped retain population in remote often disadvantaged areas at a time of economic crisis when unemployment rates were high. However, support was still not sufficient to retain people on the land in some remoter areas as the SRUC "Retreat from the Hills" evidence shows.

Including being informed by the results indicators presented in Chapter 2, areas in Axis 2 where we surmise that cost effectiveness is reasonable are:

• water related Measures;
• animal welfare;
• afforestation (although if LFASS was paid on afforestation it could be undertaken at lower cost); and
• some agri-environment Measures, but by no means all.

Areas where we surmise that cost effectiveness could be improved are:

• LFASS which could be embodied in BPS and achieve the same income supporting results;
• some open ended schemes agri-environmental Measures such as corncrake support;
• some agri-environmental Measures; and
• efforts to mitigate GHG emissions (should be win:win opportunities).

3.4.3 Efficiency - Axis 3

The main focus of the Axis was on rural enterprise and in particular rural tourism. At Axis level the focus and allocation of financial resources (see Table 2.9 in Chapter 2) was appropriate to the objectives targeted at that level (rather than the narrower Measure level). Resources were delivered proportionately against the priorities targeted.

Performance against those result indicators pertinent to Axis 3 shows a picture of apparently significant underachievement against all rural diversification economic indicator targets. In contrast to this the recorded contribution of Measure 323 to Population in rural areas benefitting from improved services saw this element of the target exceeded. Although Measure 321 appears to underperform the revised target performance here may actually be greater as the Measure was transferred to LEADER but that element of achievement is not reported in the total. This is evident from the results indicators in Table 2.10 presented in Chapter 2.

This apparently poor performance appears to a substantial degree to be a function of weaknesses in target setting and in particular the interaction between the upward revisions in monitoring indicator targets and the reductions in budget.

Nevertheless the primary research indicates a different perspective with respondents reporting strong contributions under the Axis to the sustainability and competitiveness of businesses, employment and rural communities all of which reflect the objectives of the Axis. Supported businesses were generally thought to have performed better in these respects than those not in receipt of support.

The survey conducted with Measure 321 beneficiaries shows high satisfaction in terms of improvements to the quality of life and the contribution to the attractiveness of the area, particularly from a tourism perspective (see Appendix E ).

Factors which influenced the ability of the Axis to produce the expected results include the reported difficulties experienced by beneficiaries in accessing support due to the late start of the Programme, the lack of continuity in support from the previous period and the complexity of submitting applications under the RP scheme in particular. This appears to have been a particular issue for Axis 3 and may have resulted in a reduced level of applications including certain categories of potential applicant being excluded, e.g. crofters and small farmers.

As noted previously noted, supported businesses were reported to be those able to access and absorb support were relatively well established businesses. The transaction costs involved in developing applications were thought to have favoured the larger investor or project. Consideration of spend output and result monitoring data tends to confirm this perspective and indicates that the average award under Axis 3 economic Measures was larger than anticipated. In contrast to this, Measure 323 substantially exceeded its output and result targets delivering greater than anticipated participation and leverage of resources within budget and thus at a lower unit cost.

The relative stability and capacity of these businesses and their ability to draw down support may predispose them to be more successful than those not in receipt of support.

Feedback from the MAPP workshops and the stakeholder consultations suggest that a stronger understanding of the SRDP and Axis objectives could have resulted in better applications reflecting the integrative objectives and approach desired and ultimately the outcomes achieved and overall cost effectiveness. Stakeholders and beneficiaries consulted suggest that the content of some application forms did not encourage them to shape their applications towards the higher level objectives sought. It is suggested that support for applicants through better advisory services or the requirement for a project business plan could have improved this and could also have extended the base of participation.

3.5 Results

3.5.1 Results - Axis 1

Considering the relevance of Axis 1 to beneficiaries needs is reflected in its intervention rationale and the financial focus on restructuring and modernisation of agricultural and forestry holdings through Measure 121 which helped diversify and increase income which was directly aligned to some of the key weaknesses identified in the SWOT analysis of the Programme. Uptake of the Programme increased over the period with very few 'recoveries'.

The monitoring records show that the SRDP has spent all its financial resources indicating that the funding met demand. However, only in the case of training (Measure 111) has this coincided with also achieving the targets set for beneficiary numbers supported. All Axis 1 Measures, but Measure 111, have under-achieved in attracting the anticipated number of beneficiaries. The SRDP therefore invested significantly larger amounts of funding per beneficiary than planned. A key question, therefore, is whether more agricultural and forestry holdings could have been supported if smaller awards would have been paid.

This should also be seen in connection with the RP Scheme through which a number of Axis 1 Measures have been accessed. In this context, primary research found that that the complexity of the application processes might have prevented smaller and less capable holdings from accessing support as it was seen as too complicated, costly, and un-accessible. This raises the question if the SRDP, under Axis 1, primarily supported larger, capable holdings and supported less those in greatest need of support in terms of improving business skills, diversification and modernisation?

There was agreement amongst stakeholders that the need for skills improvement in farms and forestry holdings changed over recent decades, shifting from the previous broad-scale of skills to a much narrower spectrum of skills due to the increased reliance on specialist expertise in many areas of agriculture and forestry. The extent to which this was reflected in the range of training provided is not clear.

There was also concern that the SRDP did not deliver strongly on business planning and linking SRDP investment with the requirement for a business plan.

At the time of the MTE, stakeholders felt that its strategic emphasis on facilitating transformational change (particularly through Measure 121) and improving competitiveness and viability of farms and forestry holdings was appropriate. However, the perception of stakeholders at the ex-post research stage was that SRDP could have potentially focused more on the delivery of business skills preparing holdings to adjust better to economic change.

Stakeholders observed that the more industry-driven the Programme was oriented in certain Measures, the more relevant the outcomes have been perceived by the beneficiaries. Particular examples related to the support provided to crofters through CCAGS (part of Measure 121), and SDS (Measure 111).

Measure 123 has had sector scale benefits in Scotland through supporting the renovation of the Brechin abattoir for pig processing (by the two Scottish pig producer cooperatives and Tulip) which arguably helped save the entire Scottish pig sector after the closure of the only Scottish volume pig processor.

Over and above the ongoing and mid-term evaluations, the Programme has also commissioned a number of studies, for example the First Stage Review to investigate and assure its continuous strategic fit and alignment with the needs of the sector.

Primary research findings show that SRDP funding was seen by most as a vital source of funding supported the sector through difficult times. An important function of the Programme was perceived to be creating, but mostly maintaining of agricultural jobs. Stakeholders felt that the RP, FPMC and CCAGS offered significant support regarding jobs also impacting positively on the supply chains. Many of the employment effects were noted to be for contractors of farming and forestry holdings.

The RP scheme supported diversification of farm and forestry holdings, however Axis 1 investment only was thought to have had little effect on more environmental results, such as water and soil improvements.

The Monitor Farms which were funded within SDS (Measure 111) were perceived to have had a positive effect on the know-how levels of the sector, including tourism. However it was also felt that these results would be felt more in the longer term.

The FPMC Scheme (Measures 123 and 124) contributed further to the improvement of the attractiveness of the rural area, creating improved access and amenities.

Measure 125 (Use of advisory services) was removed from the Programme due to provision elsewhere by the Scottish Government. However, stakeholders felt that there were not enough skills providers in rural Scotland. This was also identified as an issue in the First Stage Review, though the shortages were very regional - lots of advisers in the North East, not enough in the South West. The need for advisers was especially important in this SRDP as all applications were initially on-line and a proportion of farmers and crofters were uncomfortable with this approach. A more bespoke and longer lasting advisory service would have been preferred by many consultees and might have supported the uptake of the Programme by less computer literate beneficiaries.

The effectiveness of the SRDP to target sectors and businesses in need of support was perceived as 'fairly good', while a number of recommendations were made regarding improved facilitation and focus and the need to widen the definition of young farmers (to avoid unfairly restricting options for those over 40 years old who wish to enter farming). More up-front clarity as to what the Programme sought to achieve and to make future applications less complicated where interest rate relief is concerned were further recommendations frequently captured by the evaluation.

Finally, a one-stop-shop approach could have improved the up-take from a wider range of businesses.

In terms of adding value and improving the quality of agricultural and forestry products, the Working Group questioned if farm and forestry holdings should not be expected to improve their work practices and production methods to maintain their competitiveness by themselves, i.e. the market failure rationale of the SRDP intervention was questioned.

There was consensus that the SRDP supported and maintained the rural economy more than creating change and increase in self-sufficiency.

3.5.2 Results - Axis 2

Axis 2 of the Programme has addressed many of the needs of the direct Programme beneficiaries, probably to lesser extent through LFASS, as Axis 2 needs may have been more societal than the needs as perceived by beneficiaries. The aims associated with LFASS resided partly in the need to sustain incomes and partly in the presumed capacity of LFASS to maintain cultural landscapes and semi-natural vegetation to deliver public goods. With respect to LFASS, the absence of any degressivity in LFASS payments meant that a payment per hectare was paid regardless of whether the income of the beneficiary was compromised. This meant that large landowners could draw down significant income support.

But for many Axis 2 Measures, it is suspected that the direct Programme beneficiaries were often those who were not aware of biodiversity concerns, water quality concerns or land-based GHG emissions who needed to be made better aware of the societal needs. There is a significant sub-group of farmers with strong interests in sustainability and wildlife and these farmers were more likely to engage with Axis 2 Measures than other farmers. There is a danger that another cohort of farmers who are reluctant to engage with agri-environment measures are those that are creating the greatest negative externalities and the least positive environmental effects from their land use practices. Thus if societal needs were being articulated, the Axis 2 Measures of the SRDP 2007-2013 may not meet those societal needs especially effectively.

Further, the tendency for farmers to use agents/consultants in applying for schemes has led to farmers being removed somewhat from the prescriptions necessary for compliance and a clear sense of the societal needs driving policy formulation.

The Programme objectives have been largely achieved for Programme beneficiaries. LFASS has rewarded farmers on disadvantaged land with income supplements for retaining cultural landscapes, but obviously not enough to keep all hill and upland farmers on the land (See SRUC's Retreat from the Hills ^[17] ), a significant tranche of farmers in priority catchments and with land containing priority species and habitats and with environmentally designated land have been supported and compensated reasonably for their engagement with Axis 2 Measures. Uptake in some areas was rather low, so some target beneficiaries failed to take up options. In Measure 214 the low uptake of upland wader options has been noted. The results achieved do reflect community priorities at a general level. Axis 2 of the SRDP has made a real effort to engage with species and habitat protection, after good discussions with environmental Non-Governmental Organisations to determine priorities which broadly reflect wider European priorities.

In water quality enhancement, the driving policy is the EU Water Framework Directive, a keystone of European environmental regulation. The emphasis on priority catchments is a robust and appropriate response to EU desiderata. The large amount of support going to designated sites indicates congruence with EU policy. The high proportion of support going to LFASS was in line with EU priorities, even if the policy targeting left room for improvement.

The activities have delivered positive results as evidenced by the indicators, but overall impact assessment is rendered problematic by the absence of baselines or counterfactuals. Some experts have cast doubt on the utility and value of the water quality indicator ^[18] . Actual results and targets were often at variance, which suggests that a mixture of conservative target setting and lack of understanding of the drivers of uptake and likely response prevailed.

Axis 2 Measures and schemes in the SRDP 2007-2013 aimed at enhancing public goods and reduce negative effects, however, because there was no measurement of goods and 'bads' to act as a baseline or target, it is impossible to estimate gains quantitatively. Nevertheless, the direction of travel is generally positive with some evidence of species recovery (but not on all species) and some evidence of water quality improvements in priority catchments, though attribution to Programme Measures remains problematic. The major declines in avifauna in moorland areas suggest that the HNV designation and LFASS is not providing appropriate habitat for moorland waders, the decline of which merits more focussed attention from research and policy. Further, it is likely that a more focused approach to reducing soil erosion and GHG emissions might have generated significant net gains (reduced losses). The engagement of farmers and forest owners with renewable energy through other means has resulted in a significant contribution, albeit not attributable to the SRDP.

3.5.3 Results - Axis 3

With over 75% of the Axis budget allocated to enterprise support Measures the focus and intervention logic of Axis 3 and the associated Measure and resource allocation has addressed many of direct Programme beneficiary needs identified. Stakeholders and beneficiaries consulted confirmed the ongoing relevance of the Programme both at the MTE and at the ex-post stage. The Axis level logic was sound and not subject to any major revisions.

At the conclusion of the Programme Axis spend by and large met targets. As noted previously, there was some variation in spend at Measure level. This suggests that overall there was beneficiary demand for the type of support provided which appears in turn to indicate that needs were being met. Although in percentage terms the differences between Measures appears large, particularly the balance between Measure 312 (141%) and Measure 313 (85%) in terms of the actual financial resources utilised the difference is small. The uptake of resources by Measure and the feedback from stakeholders and beneficiaries consulted appears to confirm the validity of the integrated approach adopted to delivering the objectives of this Axis and meeting beneficiary needs.

Where the Axis performs less well is in terms of achieving the number of beneficiaries and development activities targeted. The majority of such output and result indicator targets have not been met. The Axis was therefore thought to have faced challenges in reaching the intended direct beneficiaries with Measure 323 being the exception here. This means that the average level of support provided to beneficiaries under the other Measures was greater than anticipated. This appears to reflect limitations in how support delivered and in particular in accessing the main scheme involved in delivering Axis 3, RP. Securing RP support is repeatedly highlighted as having high transaction costs, a considerable problem for small businesses or local communities seeking relatively modest support. This poses questions over the importance and strategic priority of the number of beneficiaries addressed in achieving the Axis objectives.

Despite these concerns it appears that the integrated approach adopted has afforded sufficient flexibility for the support provided to be able to adapt to the nuances of the way in which the needs have evolved e.g. in response to direct and indirect effects of the economic crisis. A common thread in many of the consultations and workshops with stakeholders and beneficiaries was the fundamental importance of support being available for the small rural businesses and rural communities targeted almost regardless of source.

The removal of the advisory services Measure as referenced above at Axis 1 may be seen to represent a missed opportunity. Beneficiary and stakeholder consultees highlight the contribution that improved support and facilitation for the process of participating in the SRDP might have made particularly for the small scale and micro businesses targeted under this Axis.

Although performance in terms of the CMEF indicators fell well short of targets, the beneficiary survey, stakeholder interviews and MAPP workshops all highlighted that when a wider range of performance Measures was considered significant levels of achievement relevant to the Programme objectives of thriving rural communities could be attributed to support under this Axis.

The MTE survey of SRDP beneficiaries found that Axis 3 Measures generated many positive business effects in terms of diversification of the rural economy, increased income, competitiveness and quality of outputs. This survey also found significant progress in creating rural jobs and, perhaps of greater importance in a time of economic difficulty, in sustaining employment.

The ex-post survey of Measure 321 beneficiaries highlighted significant achievements in terms of community participation, service provision, revenues and employment created and sustained. This had substantial effects in improving the quality of life in the rural areas and the attractiveness to residents and tourists. The workshops and other stakeholder consultations highlighted strong contributions under the Axis to the sustainability and competitiveness of businesses, employment and rural communities and to improvements in the quality of life. The main contributions to this arose through labour productivity and competitiveness improvements, employment creation and safeguarding, increasing stakeholder capacities, creating economic growth and introducing innovation.

In terms of the Community Priority for Axis 3 (and similar to achievements under Axis 1) the main achievements relate to supporting and sustaining rural businesses and communities rather than effecting significant change or growth. The Axis has created opportunities for creating employment but perhaps more opportunities for sustaining or consolidating employment, the employment situation as a result of Axis 3 interventions therefore appears to be better than it otherwise would have been. Supported enterprises are contributing to the wellbeing of the wider rural economy and community. There is also evidence (e.g. from Measure 321 beneficiaries) of improved community capacity and participation and a perception that the attractiveness of rural areas has been improved.

3.5.4 Results - LEADER

Funded by 5% of SRDP budget resources, there were a total 20 LAGs in Scotland, varying in area, population and funding size. In total, monitoring data state that 1,650 projects were funded covering a wide range of initiatives from basic community projects to high tech innovations. For around a third of the LAGs, this was the first participation in LEADER.

Chapter 2 already discussed the financial allocations and physical performance of LEADER, emphasising that the Axis 4 budget was reduced by 48% following the Spending Review, while more than the expected number of LAGs were funded (20 instead of 13). Monitoring reports state that there were 28,028 beneficiaries. Financial allocations were strengthened by the transfer of Measure 341 in November 2011 and fully spent by the end of the Programme.

In terms of monitoring data, SRDP processes gathered a limited amount of data from LAGs exclusively relating to Axis 4 performance indicators. Although each LAG had their own individual monitoring system in place collecting project specific data across their funded initiatives, these data were not aggregated at the SRDP level. Therefore, the extent to which LEADER projects contributed to Axis 1, 2, and 3 achievements is impossible to establish. This can be regarded as a considerable missed opportunity with regard to overall SRDP achievement levels, and more specifically in demonstrating the positive impact of LEADER projects across a wide range of rural development fields.

The thematic focus of the LEADER groups was considered appropriate. Overall, the essential success was seen in promoting improved local partnership working (partly embodied by the constellation and membership of the LAGs themselves, and partly through project activity and ambitions to collaborate more widely than before).

It was felt that LAGs were the prime promoter of building local capacities not only in implementing and delivering the local development strategy but also in facilitating the integrated local development approach in the relevant communities. Some of the LEADER areas achieved some excellent progress in integrated project delivery and creating synergies across projects from different sectors, such as combining biodiversity, food and drink and tourism initiatives, or training of hard to reach young people with landscaping projects (see Case Studies in Appendix F ). At the same time, many LAG areas interpreted LEADER primarily as a community development Programme treating 'the community' and 'rural enterprises and employment' as quite different things. However, an ambition to implement innovative projects and build relationships between community groups has been prevalent universally.

Overall, the SRDP LEADER Programme has successfully implemented the integrated approach in many LAG areas demonstrated by a number of good practice project examples. While better guidance on the integrated report and a more consistent requirement to implement the Local Development Strategies in an integrated manner might have achieved a more widespread success in this respect.

The extent to which an individual LAG linked the projects to its Local Development Strategy, depended to a large extent on the composition of LAG memberships and the respective LEADER support team. Here, the awareness of linking or embedding community based projects with employment generation was critical. This also impacted on the extent to which LEADER contributed to building local capacities for employment and diversification. Stakeholders and beneficiaries considered employment creation and safeguarding, business skills development and increases in productivity of rural enterprises as relevant outcomes creating positive changes in rural areas as a direct influence of LEADER.

Figure 3.6 in Section 3.8.5 shows that the majority of consultees felt that the SRDP scored highly in supporting innovative approaches. LEADER fostered partnership development, capacity building and improvements to quality of life, these were seen as the areas where LEADER made most positive impact in the participating rural communities.

Findings from primary research indicate that one of the most significant impacts of LEADER was improved the quality of life through creating improved partnerships linked to improved capacities and awareness of local governance.

Overall, the majority of consultees felt that LEADER was successful in making the rural area more attractive, which also related to the focus on tourism development in many LAGs.

LEADER had a strong and positive image across the Programme area including general recognition by stakeholders from other Axes for its achievements in rural community development and partnership working.

The implementation of LEADER was challenged by a number of difficulties, in particular a reporting and procurement regime which was generally regarded by LAGS to be substantially over bureaucratic and unhelpful, creating substantial administrative burdens particularly for small LAGs who were under-resourced and understaffed for this level of administration.

The following table presents some of the most frequently identified strengths and weaknesses of LEADER processes.

LEADER Processes - Strengths

LEADER Processes - Weaknesses

Passion and commitment of LEADER teams to make it happen Support of innovative and community-based projects Availability of support staff for project applicants Application process Devolved appraisal and decision making process Good collaboration between LAG co-ordinators Clear roles for LAG members Sharing best practice

Guidance notes lacked in clarity, changed a lot, and not in place at the outset of the Programme Eligibility issues arose due to misinterpretation of rules and regulations Frequently changing reporting and monitoring requirements created audit issues, particularly regarding State Aid Inconsistent process for claims and procurement Communication issues between Managing Authority and LAGs Too much time spent on administrating rather than facilitating project delivery

In early 2012, LEADER went through audits that caused considerable concern and resulted in a number of disallowances across many LAGs. On reflection, stakeholders believe that the extent to which audit issues occurred arose for a number of reasons, both on the side of the auditors as well as some administrative and interpretation issues of eligibility criteria on the side of LAGs. The extent to which domestic legislation was intertwined in the projects with EAFRD regulations created significant limitations for project eligibility and procurement. In total, £1.5m of project awards were disallowed by audits.

While the audits and disallowances caused considerable upset at the time, valuable lessons have been learnt helping to create a simplified landscape, much improved guidance material and improved monitoring systems for the new Programme period.

3.6 Impacts

This section presents the Economic Impact Assessment (further detailed by Appendix A ) and offers observations with regard to the environmental impact.

3.6.1 Economic impact assessment

The economic impact assessment is based on the gross impacts reported in a number of surveys conducted over the course the SRDP Programme. These were conducted using a variety of methods (telephone, online and postal) and included:

SRDP 2007-2013 Evaluation Surveys
2010 (466 respondents) 2011 (261 responses) 2012 (258 responses)	2013 (442 responses) 2016 (33 responses)

After duplicates were removed there were a total of 1,239 unique respondents.

Appendix A provides detail regarding the calculation model and related issues in determining impact.

We have conducted Economic Impact Assessment (EIA) at the Measure, Axis and Programme level. However, it should be noted that the findings of the EIA are only presented for four Measures (121, 123, 214 and 311), the findings for the other Measures are not considered to be representative for the totality of beneficiaries funded under these Measures at Measure level.

Gross impacts of the survey sample

In comparison to the GVA Result Indicator (reported in Chapter 2) which Measures change in economic growth in general, the Gross Impact indicates the increase and safeguarding of jobs and GVA which has been brought about as a direct consequence of SRDP investment. GVA impacts are higher than the result indicator GVA change, primarily due to the inclusion of safeguarded job impacts (generally safeguarded jobs maintain the same level of GVA), which are not captured in the result indicator for GVA measuring change.

The total gross impacts ^[19] that have been created by SRDP investment between 2007 and 2014 and are expected to be created in the future (as a consequence of the investment), captured through primary research of 1,239 beneficiary businesses are:

• gross additional impacts created/safeguarded:
  • 2,100 gross jobs created/safeguarded
  • £67m gross GVA per annum generated/safeguarded; and
• gross additional impacts predicted in the future:
  • 1,200 gross jobs created/safeguarded
  • £34m gross GVA per annum generated/safeguarded.

Net effects

In order to determine the extent of change that can be attributed to the Programme, we must move from gross to net impacts by applying the additionality factors of deadweight, leakage, displacement and multipliers. As with the gross effects discussed above, these are calculated and presented at the level of sample responses only at this stage.

Application of additionality factors

The additional effect of a Programme is the difference between what would have happened anyway (the reference case) and the benefits generated by the support (the intervention case), adjusted for displacement, leakage, substitution, and multiplier effects. Definitions of the various factors are outlined in Appendix A .

The additionality factors were determined on a case-by-case basis and therefore were identified only by those survey respondents that reported impacts.

Multipliers from the Programme were taken as an average multiplier for the wider agricultural sector (including elements of accommodation, retail and recreation) from the Scottish Annual Business Statistics.

The multipliers represent the indirect effects on the local economy due to the direct effects of the Programme. This can be through the income effect - the positive economic impacts created through the expenditure of wages and profits from newly created jobs and income - and the supplier effect - the positive economic impacts created through purchasing goods and services from suppliers.

Further details of the additionality factors (based on beneficiary feedback) applied in moving from gross to net impacts can be found in Appendix A ^[20] .

The impact assessment has identified the Programme has/will generate the following net impacts from the 1,239 respondents, outlined in Table 3.1 .

Table 3.1: Net Impacts

	Jobs		GVA
	At time of survey	Future	At time of survey	Future
121	650	390	£17.6m	£13m
123	1,040	480	£34.5m	£10.8m
214	10	-	£0.4m	-
311	140	120	£2.1m	£2.1m
Axis 1	1,800	900	£59m	£30m
Axis 2	70	-	£3m	-
Axis 3	240	190	£4m	£4m
Programme	2,100	1,100	£69m	£32m

Note should be taken that the figures at Axis and Programme level are based on the survey responses from all Measures.

The total net impacts ^[21] that have been created at the time of the surveys and are expected to be created in the future, captured through our survey of 1,239 beneficiary businesses are:

• net additional impacts created/safeguarded at the time of the surveys:
• 2,100 net jobs created/safeguarded
• £69m net additional GVA per annum generated/safeguarded; and
• net additional impacts predicted in the future:
• 1,100 net jobs created/safeguarded
• £29m net additional GVA per annum generated/safeguarded.

Grossed up impacts

Based on monitoring data, there have been 42,963 ^[22] individual businesses that have received support through the SRDP Programme

In order to calculate the impact of all businesses supported through the Programme, the results need to be 'grossed up' to reflect the wider population of businesses receiving support (42,963). Full details are present in in Appendix A .

Please note, outliers (i.e. those respondents who reported exceptional increases in employment) were removed prior to grossing up then added back in to avoid skewing or over representation of the data.

Table 3.2 reports the grossed up economic impacts along with associated margins of error (MoE), with a comprehensive sensitivity analysis presented in Appendix A . As noted earlier, although all findings are included at Axis and Programme level, only those Measures with a reasonable MoE are presented in the table below.

Table 3.2: Grossed up Impacts - by Measure (of 1,239 respondents)

	Jobs		GVA		MoE
	At the time of the surveys	Future	At the time of the surveys	Future
121	4,400	2,800	£123m	£107m	3.8%
123	2,700	1,200	£79m	£44m	10.1%
214	1,000	-	£30m	-	8.7%
311	300	200	£4m	£4m	7%
Axis 1	10,500	6,600	£394m	£262m	3.4%
Axis 2	8,800	-	£317m	-	6.3%
Axis 3	3,400	2,600	£61m	£40m	6.3%
Programme	22,700	9,100	£772m	£303m	4.7%

The total net impacts ^[23] that have been created and are expected to be created in the future by all recipients of SRDP funding (= the grossed up findings from the survey respondents) are:

• net additional impacts created/safeguarded at the time of the surveys:
  • 21,600 to 23,900 net jobs created/safeguarded
  • £735m to £809m net additional GVA per annum generated/safeguarded; and
• net additional impacts predicted in the future:
  • 8,800 to 9,600 net jobs created/safeguarded
  • £291m to £315m net additional GVA per annum generated/safeguarded.

Return on Investment

The value for money assessment is based on the cost per job and return on investment (RoI). The former compares the estimates of Programme impacts against public sector expenditure incurred, as presented in Table 3.3 (over) including the relevant range (Low, Mid, High) considering the respective MoE per Measure/Axis.

Table 3.3: Cost per Job

	Jobs	Costs	Low	Mid	High
121	7,200	£1.4m	£20,300	£21,100	£21,900
123	3,900	£153m	£8,100	£9,000	£9,900
214	1,000	£197m	£177,927	£194,900	£211,800
311	500	£25m	£46,500	£50,000	£53,600
Axis 1	17,100	£196m	£11,000	£11,400	£11,800
Axis 2	8,800	£1.03bn	£109,400	£116,700	£124,000
Axis 3	6,000	£119m	£18,700	£19,900	£21,200
Programme	31,900	£1.34bn	£40,900	£42,100	£43,300

Table 3.3 indicates that each of the estimated 31,900 created/safeguarded and future jobs directly attributed to SRDP investment in total have cost on average: between £40,900 and £43,300 at the overall Programme level. However it should be noted that this overall estimate includes substantial investment in initiatives which were not targeted at job creation or safeguarding.

Specifically, the Table shows significant differences between the cost per job by Axis with Axis 2 incurring the highest costs per job mainly due to the fact that the type of investment made by Axis 2 Measures, i.e. agri-environmental improvements were rarely impacting - or designed to impact - on employment. It is, therefore, important to recognise that Axis 1 and Axis 3 incorporated SRDP investment with a much more direct focus on creating and maintaining economic growth and job impact and have achieved this with an average cost per job between £11,000 and £11,800 for Axis 1, and between £18,700 and £21,200 for Axis 3.

Table A.8 in Appendix A details the Return on Investment, which provides an indication of how much value has been created with the investment of £1 public sector resources.

At the overall Programme level, support is estimated to have delivered a RoI of 2.30:1. This means that for every £1 invested in the Programme, a further £2.30 was generated in the Scottish economy in GVA.

Again, the significant difference between Axis 2 (£0.90 per £1 invested) with the other two Axes is clear - indicating that it is quite inappropriate to include agri-environmental investment (Axis 2) in an economically based RoI calculation. Therefore, more specific observations regarding environmental impact are presented below.

3.6.2 Observations regarding environmental impact ^[24]

The principal challenge with evaluation of Axis 2 Measures, with the partial exception of LFASS, is the extreme challenge of separating out the policy effect from the SRDP from a host of other possible causal influences. Although the SRDP 2007‑2013 asserts the need for baseline of key variables, the absence of effective monitoring procedures has made it almost impossible to separate out a policy effect or impact. Studies were commissioned early in the Programme period (FERA 2009) ^[25] and a report was undertaken to evaluate the biodiversity Measures in 2015 ^[26] using a more modest set of methods, but still the working group for agri-environment for the SRDP 2014‑2020 felt it necessary to state that: "the design of the next Programme must have management information and monitoring requirements incorporated from the beginning including improved spatial recording".

The FERA recommended approach was based on Klein and Sutherland's 'gold standard' BACI approach: Before: After; Control; Impact. ^[27] This requires a statistically representative group of beneficiary farmers being compared with respect to a control group. So for example, the change in population of corn buntings of participant land in the corn bunting Measure would be compared with a statistically valid control sample of similar holdings. This approach is data intensive and expensive but likely to yield very effective assessment of policy impact.

The Scottish Government has developed an approach to the appraisal of HNV farming and forestry over the last Programme period ^[28] . Building on earlier work undertaken by McCracken et al. ^[29] it designated a large proportion of Scottish rural land as High Nature Value with a semi-natural vegetation criterion being critical in such a large area being designated. We argue that the predominance of this variable and the relative neglect of landscape variables may mean that areas with strong type 2 HNV farming will be understated and that some areas which are included have relatively low nature values. Indeed associated maps show very little Type 2 HNV land. Over the last Programme period the area of HNV land classified under an agreed Scottish Government scheme remained virtually unchanged ^[30] but it is likely that there were qualitative changes (up and down) in HNV land in different places. Brunbjerg et al. (2016) ^[31] illustrate how the highly regarded Danish HNV procedures link targeted Measures to enhance HNV qualities.

Qualitatively it can be surmised that a scheme that was conceived with good stakeholder engagement, and good alignment with European priorities is likely to have delivered positive outcomes and impacts among the Axis 2 Measures. Accurate attribution of positive change to policy remains extremely difficult in the absence of robust monitoring procedures.

The direct effects of the Programme have been indicated earlier. About three quarters of the land area targeted for LFASS support received assistance. The fact that it is not particularly discriminating in supporting high quality environment weakened its overall environmental effect. The fact that it is not degressive weakened it social impact.

The biodiversity Measures were assessed in the ESTE study of 2015 ^[32] . It evaluated the impact of just over 20 agri-environmental Measures using a relatively naïve approach which ignored counterfactuals and had limited baseline data. Notwithstanding the relative simplicity of the evaluative methods, it would seem likely that there were significant environmental improvements on around one third of Measures, a reasonable likelihood of improvement on another third and much less success in another third. Overall the levels of uptake were highly variable, with some Measures effectively oversubscribed and others undersubscribed.

Animal welfare Measures reached far fewer farmers than intended but there is a high likelihood of positive results.

Forestry Measures involved less hectarage but more farmers than the target, which suggests a rather small size of afforested parcels.

Indirect effects (also termed horizontal effects) are the wider effects arising from the policy intervention. They can be positive or negative.

Here we flag the most likely secondary/horizontal effects of the Axis 2 Measures:

• LFASS: may well increase GHG emissions; significant income enhancement;
• agri-environment Measures: decreased GHG emissions (mostly); soil protection; largely income neutral but least so for more extensive farmers within a farming type who may make small gains; water quality normally will benefit;
• animal welfare: reduced GHG emissions per unit output;
• forestry Measures: GHG emissions reduction; business diversification; landscape quality gain; and
• HNV and biodiversity Measures generally provide green infrastructure for tourism sector.

There are a number of intervening and confounding factors at work that make establishing a policy effect rather difficult. First, there have been significant market fluctuations in input and output prices, with this Programme period including a period when global wheat prices doubled and all cereals prices were very high and, at other times, energy prices pushed up fertiliser costs. The general reduction in stock in upland areas was a response to both market pressure and social pressures in remote places. High food prices and high-level science-policy discussion about the food energy water nexus being under strain reinforced an already existing food security argument being used by farming unions and others and may have reduced uptake of Measures which would have removed better quality land from food production. Extreme meteorological events especially major rainfall and floods can have a profound effect on water quality. In spite of its severe effects in some places and sectors the banking crisis, and the general economic crisis had a modest impact on the land use sector, but the longer term effect may be a reduction of amenity buyers who may have a greater propensity to engage with Axis 2 Measures.

3.7 Success and Failure Factors, Good Practice

The major contextual factors impacting on the achievement of SRDP aims can be considered as:

• the financial crisis from 2007 to 2011 shaped public sector responses to development and created an emphasis on employment creation and economic recovery. However, incomes in the farm sector were relatively high in this period, although they have become increasingly volatile as farm commodity prices are now more connected to global markets; ^[33]
• the food product crisis (of rising food prices that affected the global food economy) and rhetoric surrounding the threats to the food-water-energy nexus and the possibility of a perfect storm ^[34] has reinforced farmers' identity as food producers and created a reluctance to divert land to less intensive uses. In some ways this resurgence of productivism is reflected in this Programme period by a levelling off of what had been very significant decline in fertiliser usage that had occurred up to around 2007; ^[35]
• changes in renewable energy support. FiT and the Renewable Heat Incentive were launched to increase uptake of renewables in this Programme period, which took away the need for RDP support for renewables in the land-based sector;
• for some of the agri-environmental fields (especially biodiversity) it is probable that climate change is influencing numbers of some species and wider land use changes and management practices are influencing others. Migratory birds may be especially affected through migration routes or through changes in wintering or summering locations. For example, some evidence suggests that the decline in upland waders is a function of predation which arises perhaps from forestry or perhaps from less intensive predator control on sporting land; and
• many people would argue that there are more extreme climatic events than even those seen in the relatively recent past. Major rainfall events have the capacity to cause serious soil erosion, major fertiliser flushes into watercourses and increased algal blooms. Warmer weather exacerbates problems of algal blooms.

The key positive factors that strengthened SRDP interventions include:

• a desire to bring together a number of schemes from different agencies such as SNH and the Forestry Commission as well as SGRPID and SEPA to create a more holistic view of the environmental challenges and their remediation;
• a strong influence of environmental NGOs who felt able to assert their preferences in the expectation that Measures would be developed in the Programme;
• the existence of some RD-relevant policy areas being addressed by other policy means (e.g. renewable energy (FiTs), GHG emissions (FFABC- Scottish Government funded) and peatland restoration (Scottish Government funded);
• a political desire to leave the established scheme LFASS intact, which included manifesto commitments at Scottish elections;
• a highly heterogeneous farming community undergoing rapid structural change in some regions (Cook et al. 2016); ^[36] and
• behind the complicated set up of the RP was the desire to promote the integrated approach in rural development, to facilitate joined-up action and synergies between different types of support and activities. Although this was a difficult process for many applicants, a number of stakeholders felt that it did have a positive learning effect and made applicants more aware of important linkages. It was also felt that to achieving lasting know-how and to securing transformational change, this learning process would rely on a more intensive process of awareness raising and know-how transfer to applicants, which could not simply be accomplished by asking a lot of questions in an application form. Other stakeholders believed that an approach such as integrated development should only be pursued at strategic level and that it was too much to ask to transfer this type of knowledge at beneficiary level ^[37] .

The major negative factors weakening the impact of SRDP interventions include:

• a patchy network of advisors and agents creating scope for stronger engagement in agent-dense areas than in agent-light areas. If the SRDP had made available better and more consistent advisory services and 'gatekeepers' to support applicants through the application process might have caused less frustration. This type of support should have been there from the outset, particularly if it was the aim to improve awareness and know-how in integrated rural development including economic and environmental actions. This would have particularly benefited the many small and micro businesses which predominate in rural areas many of whom, despite their highly relevant needs felt unable to participate in the SRDP due to the high transaction costs;
• a rather complex approach to developing proposals in a number of Schemes, which militated against small farmer (and rural business) engagement with the Programme;
• the approach to regionalisation could have been strengthened, in selecting priorities. RPACs tended to focus on securing the maximum level of support for their region rather than the optimum to meet the differentiated needs;
• the appraisal process could have been stronger, delivered by dedicated appraisal staff with a more holistic and informed level of expertise and a shared understanding across agencies and organisations involved;
• many beneficiaries of a variety of schemes typically suggested simplification of the application process, regardless of satisfaction with their overall experience;
• the PMC could have been more involved in managing the balance between optimising spend and EAFRD draw down and adjusting the Programme priorities and logic to better meet changing needs;
• in terms of environmental achievements, a re-emergent food security culture in the wake of the global food price increase in and after 2007 which was actively promoted by farmer unions and may well have influenced farmers' desires to take land out of productive agriculture;
• a significant cohort of 'hard to reach' farm households operating out-with the advisory systems, who are less likely to engage with discretionary Measures;
• extreme meteorological events which causes major nutrient flushes and soil erosion;
• an anti-forestry culture among traditional productivist farmers, rooted in a history of landlord tenant rights around trees and a lack of integrated farm forestry training and education; and
• global agricultural product price volatilities and global market developments.

Key Lessons Learnt can be summarised as follows:

• target setting for the performance of the SRDP was poor, with the result that it is challenging to assess the extent to which the Programme has been effective in achieving its aims and objectives. Despite small improvements made during the Programme period to how data were collected, this was not extended to revising the targets of the Programme effectively;
• if the SRDP sought to support those businesses in most need, then appropriate advisory services and less complex application procedures would have had to be put in place to ensure that the accessibility of the Programme is inclusive of those businesses with less capacity and know how in sourcing funding; and
• there is consensus about the importance and relevance of creating new jobs, yet the safeguarding of jobs is considered as important as creating new jobs, particularly in view of challenging economic times. In many cases, stakeholders confirmed that without SRDP intervention and support, the viability of many small farming businesses would not have been maintained.

3.8 Common Evaluation Questions

3.8.1 Economic growth

The start of the SRDP coincided with the financial crisis in 2008. The farming community was less affected by the immediate impacts and benefited from a number of economic circumstances, such as low interest rates, increasing prices in sheep and cattle until 2012/13 which helped the rural economy to maintain its employment levels. However, rural businesses suffered considerably from uncertainty and a reduced level of confidence regarding new investment - this also created hesitation in taking on new risk in relation to creating new jobs. Bad weather in the winter of 2012, but also varying exchange rates, increasing fuel costs and product price fluctuations (grain prices fell, steel machinery prices rose) contributed towards a decreasing business viability by the end of the Programme period.

However, the perception was that SRDP did not do enough to support agricultural businesses to become more resilient to change and it was thought that non-agricultural rural businesses fared better through SRDP investment in adapting to changing circumstances, experiencing rising incomes and increases in job creation and productivity. It was felt that one reason for these positive developments was that the number of tourists increased (although the experience of this in the North (very positive with lots of innovative initiatives) and the South (negatively - tourists heading North rather than staying Sough) of Scotland differed considerably).

Looking more specifically at GVA, the agricultural GVA in Scotland rose by approximately 35% from 2007 (£887m) to 2014 (£1.19bn) an increase of £307m, Figure 3.4 . In total, between 2007 and 2014 Scottish agriculture generated £7.8bn of GVA.

Although difficult to compare directly , the estimated economic impacts ( Appendix A) of around £2.8bn ^[38] created by SRDP over seven years have had a key role in sustaining the agricultural sector.

This includes GVA created through new as well as safeguarded activity supported by Axes 1 and 2. It should be noted that the majority of the £2.8bn GVA created was associated with sustaining employment (in Axis 2) rather than creating new jobs.

Figure 3.4: Agricultural GVA

Source: Total Income from Farming Estimates for Scotland

According to the statistics, the average GVA per FTE agricultural worker increased over the period by 37% to around £29,000. The economic impact assessment indicates an average GVA per newly created/ safeguarded agricultural job of around £37,000, indicating that the SRDP has contributed to the Scotland's rising agricultural productivity (creating higher value jobs and including supplier impacts).

Primary research confirmed that stakeholders feel that the SRDP had a good impact on creating economic growth which received the highest scores (see Figure 3.6 in Section 3.8.5).

3.8.2 Employment creation

The total agricultural workforce remained fairly stable between 2007 and 2014, barring a sharp fall and recovery between 2007 and 2009, Figure 3.5 . The total workforce was just over 66,000 in 2014, down 1.3% from 2007. The economic impact assessment ( Appendix A) suggests that in the region of 26,000 jobs (in Axes 1 & 2) were created / safeguarded over seven years by the SRDP, suggesting that there might have been a greater decline in jobs in the absence of the Programme.

Figure 3.5: Total Agricultural Workforce

Source: Scottish Agricultural Survey

Measure 212 comprises 37% of the total SRDP 2007-2013 spend. While it cannot be regarded as a significant employment creator, (c. 10,000 jobs, mostly safeguarded) it almost certainly has had an effect in retaining population in hill and upland areas. In that these funds are non-competitive, they have low transaction costs and management costs and comprise a significant transfer payment to the farm sector at low administrative cost.

It is interesting to see in Figure 3.6 , Section 3.8.5, that although beneficiaries and stakeholders felt that the SRDP created positive change leading to economic growth, their scores regarding job creation and safeguarding jobs were significantly lower. In this context, note should be taken however that the largest number of responses (over 800) were gathered at a relatively early stage during the Programme. Looking at individual Spider scores, it appears that the respondents of later surveys were much more positive about their perception of employment creation and safeguarding.

3.8.3 Natural resources and biodiversity

The contribution of the SRDP to protecting and enhancing natural resources cannot be assessed accurately from the available data sources, or from the studies commissioned by the Scottish Government. There is some circumstantial evidence about the impacts of some Measures but the evidence base is simply insufficient to address the challenge. The principal reason for the inability to assess impact is that no real attempts were made to establish a baseline against to which to measure the impact of SRDP Measures. From 2007 to 2014 studies have noted major changes taking place in Scottish agriculture including a destocking of significant areas of hill land (SAC 2007, SRUC, 2011), significant restructuring of farm businesses and the emergence of complex tenurial arrangements on the most dynamic business (Macaulay Institute et al. 2008) and the simplification of farming systems and reduced numbers of ruminant livestock (Cook et al. 2016). The extent to which these changes have been induced by the SRDP, Pillar 1 of the CAP or other social and economic factors cannot be established retrospectively. The absence of any use of counterfactuals, in spite of clear recommendations to Scottish Government in the FERA report 2009 that they should be used to monitor the impact of agri-environmental Measures means that any impact assessment is to a degree speculative. These data deficiencies are not remediable in a low-budget ex-post evaluation. Notwithstanding these observed weaknesses it is still possible to make informed judgements as to the likelihood of success of SRDP Measures and some options) in the last Programme period.

Farmland biodiversity generally has been in long term decline (SNH 2013). It is conventionally measured by the Farmland Bird Index. The trend for upland waders is regionally variable with lower declines in the north west of Scotland, but is described in the SNH report (2013) overall as one of "steep decline." Changing agricultural practices, including drainage, tree planting (potentially SRDP-grant aided tree planting) and increases in predation are identified as likely causes by scientific studies. The trend for seed eating birds is much healthier. There is a longer term increase of a composite index of four seed-eating birds, although since 2007 the increase has levelled off, in spite of Measures in the SRDP to promote habitat management for seed eating birds. There are regional differences with the west actually showing a decline in seed eating birds in the most recent Programme period. Availability of nesting habitat and the use of game cover crops are seen as positive influences on seed eating bird numbers (SNH 2013). For many farmland bird species, the increase in numbers of species using woodland edge habitats is noted (SNH 2015), suggesting that new woodland creation may be implicated in these population gains.

The ESTE Report (2015) details the findings from 20 agri-environmental options from the LMO and RP schemes. Our reservations regarding the overall effectiveness of the monitoring practices have been stated elsewhere. They concluded that: "Overall, 12 out of 20 Options were considered to either fully meet their biodiversity success criteria (6 Options) or to, despite not fully meeting them, having a strong positive impact on the biodiversity of the site (6 Options). 7 Options showed limited positive effects, which will still benefit biodiversity to a considerable extent. Only 2 Options had a negligible effect, no Options led to negative ecological changes." However, there was a large number of schemes with insufficient data (and uptake) to justify monitoring which suggests that the menu of options was too long and that the targeting was not always successful.

As measured by the Scottish Government, HNV farming and forestry has increased slightly over the SRDP period. This is surprising for two reasons. First, the area on which LFASS has been paid has declined rather sharply as a result of land abandonment or more likely, transition to other uses, as evidenced in the Evaluation of LFASS/ development of ANC report 2016 ^[39] . Second, the key bird species found in the uplands that provide a major input into the Farmland Bird Index, have been in serious decline. It seems likely that part of the large extent of semi-natural habitat which included in these figures is likely to be sport-shooting land (possibly with sheep used primarily as tick mops). Gains in the abundance of indicator species in woodland birds have occurred in the last Programme period (SNH 2015) and woodland fringe birds will have benefited from the small size and therefore larger fringe area. HNV forestry gains are more likely to be real gains as a result of new mixed and native woodland planting and more active management of existing forests for environmental purposes.

Increases in some target species of birds are likely to be a result of RDP options under Measure 214. Corncrake expenditure increased rapidly in this Programme period and trends in that species are healthy. Similarly corn bunting numbers which had been in long term decline in eastern Scotland have increased and this increase is almost certainly attributed to the SRDP. Going forward, it remains vital that robust evaluation methods are used to be able to filter out confounding factors and establish policy impact with accuracy.

Water quality was recognised as a key concern in the SRDP 2007‑2013. Fourteen priority catchments were identified, mostly in eastern and south-western Scotland. In the course of the Programme, catchment walking and one-to-one meetings with advisers and farmers were used to select and implement Measures to enhance water quality. The GNB indicator was argued to be flawed in a paper by Vinten et al. (not dated). Using modelling approaches they argued that woodland creation and wetland creation Measures in the SRDP were the most cost effective Measures to mitigate phosphate pollution, and that a catchment by catchment and ultimately field by field approach was needed to ensure cost-effective water quality remediation. Most Measures on the list of options generated a beneficial effect but the degree of benefit varied greatly from catchment to catchment. More effective targeting by both Measure and catchment would improve cost-effectiveness.

3.8.4 Renewable energy

On farm renewable energy production has risen rapidly in the last decade, but the rapid increase has recently been replaced by a much slower uptake of renewables. The increase in land manager engagement with renewables is almost entirely a function of national energy policy changes in 2010, when earlier support for renewables was replaced by a much higher level of support under the Feed-in tariff (FiT) scheme, which provided long term (mostly 20 year) guaranteed income streams for the development of renewable energy projects up to 5 MW. Although Measures were made available in the SRDP 2007‑2013 these are almost inconsequential causal factors in the rapid (but not fully documented) rise in farm based renewables production. Using the FiT funding data base, there were just over 400 operational schemes before 2007, a further 273 started production (with a capacity of nearly 2,200 MW and a further 521 schemes with a capacity of 4,500 MW were approved or under construction. Not all of these were farm based but the majority were. The overwhelming majority of schemes (over 90%) were wind turbines with the residue mostly hydropower or biomass systems.

The emergence of a very supportive funding stream out-with the SRDP meant that almost no-one used the Measures in the SRDP 2007-2013. However, the rapid fall in FiT rates in 2015 and major policy shifts in the renewables sector signalled the end of the rapid expansion of farm level deployment of renewables. A rapid tailing off of engagement of Scottish farmers is an inevitable consequence. Community energy production has been especially important as a revenue raiser for many rural communities, especially remote island communities. Again, it has been funded primarily by FiTs. The already cited OFGEM figures will include any installations below 5 MWs (which is the majority) which are not distinguished in these tables by ownership structure. Community energy production too will decline rapidly as a result of policy change.

3.8.5 Qualitative perceptions of change created

Figure 3.6 below shows the perceptions of stakeholders and beneficiaries regarding the extent to which the SRDP has created positive change on a scale from 1 (no impact) to 5 (significant impact) by the key performance indicators posed by the CMEF. The score is an average from the MTE survey, ex-post stakeholder consultations and beneficiary surveys.

3.8.6 Competitiveness of the agricultural and forestry sector

Axis 1 supported a wide range of initiatives targeted to modernise and restructure the agricultural sector and thereby improve its overall competitiveness.

With Measures 121 and 123 consuming 90% of the Axis budget, the strategic emphasis was firmly set on supporting transformative change, improving competitiveness and bringing about innovation and added value.

Feedback on the extent to which the SRDP was able to contribute to this effect was generally positive throughout the primary research. Much of the response related to being able to maintain employment in the sector, but some of the more targeted support options and schemes, enabling an improved marketing, market access and / or achieving quality improvements were thought of highly. The majority of respondents during the MTE survey felt that the SRDP investments had a lasting impact on the sustainability of their farm holding and half of the respondents felt that 121 contributed well to improving the competitiveness of their holding.

The MTE found Measures within Axis 1 to be effective from a business development perspective. The survey of beneficiaries found that almost 70% of respondents reported positive impacts on their business efficiency and in many cases output, quality and competitiveness have been enhanced. The greatest contribution to employment arose from Measure 123 (Adding value to agriculture and forest product), Measure 121 (Modernisation of holdings) and Measure 111 (Skills). Measure 123 is highly valued by recipients and stakeholders and has generated significant positive impacts.

Local stakeholders and beneficiaries reported some distinctive local competitiveness effects in the agricultural sector e.g. in egg production or the dairy sector, this was in response to local concentrations of activity and the expansion and development of this was thought to be a direct effect of the SRDP. SRDP supported slurry storage investment was felt to have had a major impact on the sustainability of the pig sector, as well as parts of the dairy sector.

Infrastructure investments supported under Axis 1 of the SRDP are thought to have contributed to improved agricultural efficiency and labour use and are reported to have helped businesses maintain incomes. Whilst some of these reported productivity increases and efficiency improvements have improved business competitiveness this may also have resulted in employment reductions.

There is evidence that the SRDP has had an important effect on competitiveness:

• high uptake of modernisation investment support;
• skills development uptake three times greater than target;
• production chain and added value sectors very active within the SRDP, volume of investment exceeding target and 50% more participants than target;
• strong evidence that the availability of skills training support over a long period has led to long term gains; and
• general perception that the forestry and agricultural sectors benefited from increased productivity.

The Ex Ante Working Group stipulated that achieving increased business competitiveness should be a clearer requirement in future.

The SRDP is not thought to have had a substantial effect on the competitiveness of the Scottish forestry sector where the main influences have been increased production and strong markets. The long term nature of forestry investment and production was stressed as a limiting factor on the effectiveness of any relatively short term SRDP interventions. For the forestry sector the SRDP support is therefore seen more to have accompanied rather than driven the competitiveness of the sector.

Forestry experienced a sustained positive period throughout the programme with increased business incomes mainly due to high timber market prices and a production / harvesting increase which brought about real job increases. Throughout the period productivity improvements were achieved, often due to new equipment and forestry business incomes had increased whether assisted or not assisted. According to forestry stakeholders consulted, the SRDP's influence on employment and incomes was thought to have been negligible however. Sectoral skills required were largely already in place prior to 2007.

3.8.7 Restructuring the dairy sector

The Scottish dairy herd has been restructuring rapidly with cow numbers and dairy farm numbers dropping by almost a quarter between 2007 and 2014. Milk production has not declined at a similar rate, with productivity increasing in the early years of the programme. The herd is declining fastest in the East and North (where grain and beef were attractive alternatives) and is concentrating in the South West (where crops are not a viable option and grass growth is favoured).

Feedback during the MAPP workshops in the North and South West highlighted the extent to which change and restructuring in the dairy sector is being driven by the often extreme volatility in market and supply chain factors. World milk commodity price volatility and the actions of milk processors squeezed by retailers reacting to the post 2008 recession, have been the main driving factors. The collapse of the farm gate milk price in the latter part of the programme saw production costs /litre exceeding the price producers received. Taken in conjunction with the limits on the sector's ability to reduce production costs, these factors substantially outweighed the scale and scope of any SRDP intervention to influence restructuring.

SRDP support for modernisation assisted some major investments in upgraded and new dairy facilities, particularly in the earlier part of the Programme when there was something of a peak in incomes. This has allowed some producers and regions to increase production while others have ceased. Investment in slurry storage and waste management in general has been widespread, often SRDP supported. Skill development support through Measure 111 has been applied to the dairy sector through the "monitor farm" approach.

3.8.8 Climate change mitigation

Climate change was directly promoted in the SRDP 2007-2013 through afforestation Measures. As is well known from the work of the IPCC work on Land Use Land Use Change and Forestry there is considerable scope for mitigating climate change through land use. Detailed investigations have been undertaken which have explored the cost effectiveness of mitigation options both globally and in a Scottish and UK context. Apart from afforestation Measures, climate change has not been addressed directly in the SRDP 2007-2013. Instead, a Scottish Government funded Farming For a Better Climate project has been launched. In consequence, the pursuit of emissions reduction in the farm sector falls largely out-with the SRDP, except insofar as new SRDP-assisted afforestation occurs on farm or other rural land. The Scottish Government has by no means neglected the question of farm based emissions of GHGs but has chosen to date not to have this topic as a mainstream component of the SRDP.

Nearly 42,000 hectares of new woodland was planted with the assistance of the SRDP. This was only 60% of the target area. The available data do not detail the type of woodland or its yield class or soil type which factors together will largely determine the potential to sequester carbon. On poorer land lower growth rates on some large native woodland extension forests funded by the SRDP will sequester carbon very slowly, but are likely to deliver high biodiversity benefits. Generally new woodland habitat will benefit biodiversity, but as noted elsewhere, woodland cover might house predators on waders and displace wetland habitat and has been implicated in their further decline.

3.8.9 Water quality and water management

Water management has been modestly improved by the SRDP 2007-2013, primarily in priority catchments, but modestly elsewhere as a result of other Measures not primarily directed at water quality improvements. The key area of improved water management is with respect to manure storage and application and water quality enhancement as a result of diffuse pollution mitigation, especially of phosphate pollution. As with biodiversity Measures accurate impact assessment is rendered impossible because of the presence of confounding factors such as stocking rate changes and other farming system changes arising from market and social factors and other initiatives being pursued which were connected with but not funded by the SRDP.

It is not clear how much of SEPAs expenditure in improving water quality in priority catchments is funded by the SRDP and how much from other sources. The principal legislation driving water quality improvement in Scotland is the Water Environment (Diffuse Pollution) (Scotland) Regulations which are referred to as the Diffuse Pollution General Binding Rules (DP GBRs). The seven DP GBRs focus solely on rural land use activities. Catchment walking revealed hundreds of breaches of GBRs across priority catchments, many of which could be remedied relatively easily by land managers. Others demanded more intensive interventions some of which were fundable under the SRDP. Attribution of improvement to a policy-supported Measure is extremely difficult to establish, but the model of catchment walking coupled with advisory input is robust as long as catchment walkers are working with robust intuitive or empirical models of which interventions will deliver the most cost-effective improvements. Vinten et al's work for the Scottish Government suggests that the suite of interventions funded may not always have been optimal.

Water is widely used for irrigation in eastern Scotland on field vegetables on about 7300 ha of land. Water abstraction is controlled under the CARs of the Water Environment and Water Services (Scotland) (WEWS) Act 2003. Water use efficiency Measures were not part of the SRDP 2007-2013.

Water quantity becomes an issue in periods of drought when there is insufficient water or when there is too much water and flooding occurs. Run off is a function of many factors including vegetation cover. In general, more woodland cover in a catchment will reduce flood peaks, while in contrast, intensive agriculture is implicated in higher flood peaks and faster run-off with implications for soil loss and phosphate pollution. Afforestation Measures will assist in pollution abatement and these have been a significant part of Axis 2 of the SRDP. Unfortunately, there has been no targeting of catchments most at risk, so any remediation and risk reduction is effectively an unforeseen benefit of afforestation. Further action research along the lines of the Pontbren project in Wales might be merited to better parameterise the impacts of tree planting on flooding in a Scottish context.

3.8.10 Quality of life and diversification of the rural economy

The development opportunities and weaknesses of rural Scotland in terms of business diversification and quality of life improvements were predominantly identified for SRDP investments regarding Axis 1 and Axis 3 only.

On-farm and rural business diversification opportunities were seen to represent a particular strength of rural Scotland regarding both tourism but timber processing, at the same time the diversification into renewables and wood processing were considered worth improving.

The SRDP included a dedicated Measure to support the diversification of non‑agricultural businesses (Measure 311).

Generally speaking at the time of the ex-post evaluation, the performance of Axis 1 schemes and interventions was felt to have been fairly successful in introducing innovative approaches and in restructuring/modernising some elements of the forestry sector (both closely associated and often linked to business diversification), thereby helping to increase capacities and productivity and improve the quality of life in rural communities.

The MTE survey of beneficiaries found that under the suite of Axis 1 Measures, respondents reported positive impacts on business diversification in 39% of cases. The Axis 1, and particularly Measures 121 and 123 investments were associated with the adoption of new techniques in 56% of cases, increased family employment in 28% of cases, increased sustainability in 35% of case and increased income in 47% - all suggesting that these developments are linked to the diversification of on-farm income sources by releasing family labour, and complemented renewable energy developments. Interestingly, higher positive values were found for beneficiaries supported under the FPMC scheme.

The MTE survey findings indicated that beneficiaries thought the investments from Measures 311, 312 and 313 were likely to contribute positively to the diversification of the rural economy. The other Axis 3 Measures were thought to have had very little if any effect in this regard at the MTE.

At the end of the programme uptake, outputs and results from Measures 311, 312 and 313 as per the CMEF indicators remained very low (as noted above for Measures 311 and 312) in relation to the previous evaluation question. Their contribution to rural diversification therefore appears likely to be limited but this pattern of uptake reflects the effects of the overall situation facing the sector and the necessary responses in favour of consolidating, sustaining and enhancing existing business. Although the number of beneficiaries participating fell short of target the intensity of support per applicant was greater, applicant businesses had strong capacity and diversified enterprises are likely to be more substantial and sustainable building on strong existing business performance. Once again the lack of LEADER reporting against these Measures is likely to have resulted in underreporting.

Over the course of the SRDP, there has been a shift in the balance in Axis 3 towards the Quality of Life objectives. This appears to have been largely driven by the high level of participation and the associated demand and uptake of Measure 323. The greater accessibility of support under these Measures appears to have contributed to the numerous smaller scale projects supported.

The MTE reported the potential Quality of Life contribution of Measures 312 and 321 as being moderate, the other Axis 3 Measures implemented were thought to contribute little if anything. At the end of the programme the reporting data suggest that Measures 321 and 323 are likely to have made the greatest contribution here by delivering a high level of participation and population, in the case of Measure 323 very substantially above the targets set, including in the forestry sector. It should be noted that LEADER also contributed in this regard although its contribution (including Measure 321) has not been fully quantified.

Most surveyed beneficiaries reported an improved quality of life due to the investments made. This included very high ratings for improving the attractiveness of the rural area and improved well-being. In addition, beneficiaries were very confident that the investments would lead to a reversal of economic and social decline and therefore would fight the depopulation of their rural areas.

The predominant view which emerged from the MAPP workshops was that the integrated approach to Axis 3 worked, for example there was evidence cited of improvements funded under Measure 323 contributing to tourism projects and activities. Improved rural services and recreational activities and facilities benefitted visitors and residents alike. Retaining and securing tourist visitor markets and the tourism offer helped rural areas remain competitive in a time of economic downturn. More secure and sustainable businesses and employment contributed to communities' economic wellbeing and sustainability. Taken together these factors contributed to maintaining and enhancing the attractiveness of the area for both resident and visitor and thus the quality of life.

3.8.11 Innovative approaches

Feedback from surveys throughout the Programme period as well as primary research at the ex-post evaluation stage found fairly strong agreement that the SRDP has supported and contributed towards the introduction of innovative approaches.

For example, one of the Programme's largest Measures (Measure 121) for modernisation was thought of having stimulated innovative restructuring and a considerable amount of up-grading existing on-farm technology. The positive contribution of the SRDP in helping to up-grade technology was also noted by stakeholders with regard to forestry businesses. In addition, Measure 121 has also provided complementary investment for renewable energy such as woodchip storage sheds, kilns, which is innovative.

Further innovative investment in the form of new technology was promoted through LMO (a Scheme considered easy to access) supporting small investments including livestock electronic identification kits, precision farming/mapping equipment and such like. Importantly, these type of investments represent a first step into a whole new area of livestock and nutrient management.

Measures 123 and124 have also supported innovative investments in primary processing technology and food technology.

A number of the Axis 3 Measures may be expected to have resulted in the development and /or implementation of innovation, particularly those Measures concerned with business development and diversification, Measures 311, 312 and 313. The levels of reported activity and achievement here are low and the extent of innovation is therefore likely to be lower than may have been expected. Qualitative feedback from beneficiaries suggests that much of the supported activity concerned making improvements in what was done and how, in effect doing more with less. This suggests that a significant degree of innovation may have been employed. Furthermore the data do not provide insights into levels of innovation, activities supported through LEADER which may be expected to have a degree of innovation are not reported.

3.8.12 Access to broadband internet

The SRDP devoted just over €1m for rural broadband projects through the Broadband Challenge Fund. The Fund gave LEADER LAGs the opportunity to bid competitively for funds to enhance broadband infrastructure and to help rural businesses and households in their areas. Five LAGs were successful in their bids. Supported activities were small scale and location specific focusing on innovative pilot solutions.

3.8.13 Scottish National Rural Network

The Evaluation of SNRN and the SRDP Communication Plan indicated that the general level of awareness of services provided by the SNRN amongst stakeholders was low. Some respondents had little or no experience or awareness of the SNRN or its services, and some questioned the relevance to rural Scotland as a whole (this was also confirmed by primary research undertaken by the ex-post evaluation).

Overall, one in three respondents had attended an SNRN event, or an event organised by SNRN. Participation was highest among community and third sector respondents (over 40%), followed by other rural businesses and LEADER, with land-based businesses having the lowest participation rates.

Networking and information exchange were the principal reasons identified for attending events, particularly amongst businesses. Information on funding ranked lowest overall although somewhat higher amongst communities and third sector respondents.

The MTE findings indicated that SNRN's Programme of delivering 20 regional events in under one year was ambitious and resulted in a formulaic approach which was not universally well received. Involvement tended to be concentrated within the territory rather than involving wider experience. This limited the exchange of good practice and establishment of wider links. Local involvement at the LAG level was reported but thought to be limited with each LAG representing a significant local network with wider links to other territories and sectors. However, the extent to which the LAG networks linked up with the SNRN was generally regarded only as average.

Therefore, it was suggested in the MTE that the Scottish Government and SNRN should consider a Programme of events which more directly involve regional stakeholders in both their design and delivery to strengthen the relevance of the approach. Events should be based on a wider geographical area in order to strengthen inter-regional networking and exchange.

The importance of the SNRN as a source of information on the SRDP and LEADER was split relatively evenly between those who see it as having some importance (56%) and those who see little or none. There was little differentiation by user group.

The SNRN's importance as a source of information on good practice in rural development was similarly rated, especially among other rural businesses, communities/third sector and private individuals. Land-based businesses ranked this markedly lower, but only 16% saw good practice dissemination as of no importance.

The MTE findings suggested that in its start-up phase the SNRN had some success in broadening the base of rural networking in Scotland, but that the main challenge facing it was in engaging the wider community. The beneficiary survey raised questions over the way in which rural stakeholders were being informed and about its overall communication strategy.

It was also suggested that the Scottish Government and SNRN should review the approach to communication with and engaging rural stakeholders with a view to improving wider engagement and both the website and the regional coordination service.

Finally, it was suggested by beneficiaries that the Scottish Government and SNRN should review the service provision to identify potential improvements to strengthen relevance and uptake by rural stakeholders. As well as this, the SNRN should take forward the provision in exchange of more good practice examples is a development priority for Scottish Government and SNRN managers.

Overall, the ex-post evaluation has concluded that the extent to which the SNRN contributed to SRDP objectives was limited.

3.8.14 Technical Assistance

The use of technical assistance in the 2007-2013 period was initially set at £2.3m and used primarily for the creation and maintenance of the SNRN and to fund monitoring and evaluation activities within the SRDP.

A significant proportion of the budget was used to build and maintain the SNRN website which provided information and news articles relating to SRDP and rural Scotland with the intentions of promoting the SRDP to a wide audience. With a strong community and LEADER focus, LAGs engaged with it in the absence of a formal LEADER network. One of the aims of the development of the SNRN website was to share knowledge and experience amongst various interest groups at country, national and transnational level to help strengthen the rural voice in policy design and maximise the use of the rural development resources available. The technical assistance budget was used to fund two large national events in the early years of the Programme to introduce the new SRDP and inform a broad range of stakeholders of its purposes. The SNRN also co-ordinated training and networking events across Scotland which, following recommendations in the MTE, included project visits to share good practices and experiences.

A smaller proportion of the budget was used to fund monitoring and evaluation activities such as the First Stage Review in 2009, the mid-term review in 2010 and two ongoing evaluation contracts that ran throughout the life of the programme. The First Stage review was commissioned as a result of the difficulties encountered in the delivery of what was an ambitious and complex SRDP. Lessons were learned from this that helped in the implementation of the programme going forward.

The first ongoing contract focused on availability of data and identifying any gaps. This resulted in evaluators working closely with Scheme managers helping them understand the importance of monitoring and ensure that data capture points were identified. The aim of this work was to improve the collection of data and to provide management information allowing the performance of the SRDP to be measured and adjusted where necessary. In reality this proved difficult and the lack of management information was a concern throughout the Programme. Further work on Result indicators, monitoring of LEADER and the environmental Measures in Axis 2 was carried out in the second ongoing contract. In the end implementation was beyond the scale and scope of practical and financial resources and therefore the focus turned to learning lessons and improving the new 2014-2020 programme. Towards the end of 2012 a modification to the SRDP was approved to allow resources from the SRDP 2007‑2013 to be used to pay for part of the Ex Ante evaluation of the 2014‑2020 Programme.

A subsequent modification in 2012 saw the budget increase to €5.2m when the decision was made that the development of the new IT system required for the 2014‑2020 Programme was to be include as part technical assistance. The intention was to start building the IT system early to allow time for development as a bespoke system and making it more user-friendly.

3.8.15 Efficiency of resource allocations

Chapter 2 presented the output performance indicators and showed that the achievement against output targets varied widely.

Mid-term and ongoing evaluations over the years have pointed out relevant shortfalls in target setting, however, final agreed targets compared to reported achievements still presents some significant shortcomings in anticipation of the efficiency of the agreed resource allocations. The extent to which the efficiencies of inputs were intentionally changed to achieve less intensive interventions for the benefit of reaching much higher numbers of participants (such as in Measure 111 for training) is unknown.

The following graphs of unit costs demonstrate some of the significant differences between intended and actually achieved efficiencies. It is demonstrated that there are substantial variations across the various Measures, in most cases indicating that the quality of the original target setting must have been limited. The target setting and therefore unit costs were most consistent in the case of LFASS which applied payments on the basis of hectares.

The following graphs and other information presented, represent estimates of unit costs for two thirds (66%) of total SRDP spend.

Figure 3.7 Anticipated and Achieved Unit Costs of Measure 111 budget

SRDP public sector spend of Measure 111 only represented 0.3% of the total Programme spend achieved, but in terms of unit costs shows some of the most significant differences between anticipated and achieved efficiencies. The above graph shows that originally it was expected that on average each of 10,000 participants would receive training to the value of €5.476 - this was revised down to expecting 2,820 participants receiving training to the value of €1,280. Instead the SRDP delivered training to 24,789 (almost 10-times more) participants to the value of €151 each on average (almost 10-times less). The unit cost differences are similarly extensive for the costs per training day and for costs for each successful trainee.

The achieved efficiencies of allocating resources by output are much closer to those expected in Measure 212 that presented 37% of SRDP total spend. This Measure allocated LFASS payments to eligible farm and forestry holdings whereby a per hectare allocation was applied (the graph below shows an average spend per hectare between €185 and €143). The original target of supporting 13,000 holdings was overachieved with 13,251 beneficiaries, which reduced the average unit cost per holding from €47,957 to €39,261.

Figure 3.8 Anticipated and Achieved Unit Costs of Measure 212 budget

In the case of Measure 214 (agri-environmental payments) the unit costs targets i.e. average payment per holding supported dropped significantly between original target (€82,476), finally agreed target (€40,890) and what was actually achieved (€22,038). Originally it was thought that one hectare under successful management should cost on average €3,711, however the actually achieved value was €81.

Although these reductions in unit costs suggest that outputs were achieved much more efficiently, the fact that the Measure budget was cut by 53% yet the output targets were exceeded by nearly 200% queries the quality of the target setting.

Figure 3.9 Anticipated and Achieved Unit Costs of Measure 214 budget

The unit costs of Measure 121 (Restructuring and modernisation of the agricultural and forestry sector) increased from an originally anticipated support per holding of €9,811 to an achieved value of €26,142 on average (primary research findings indicate that larger investments in RP were favoured particularly in the early part of the Programme). Figure 3.10 below shows that a large increase occurred in the average costs per new product introduced which rose from €19,622 to €80,338.

Figure 3.10 Anticipated and Achieved Unit Costs of Measure 121 budget

Finally, Figure 3.11 presents the unit costs of Measure 313 that supported tourism activities. Over the course of the SRDP, its original budget was cut by 55%, yet the target for its output (number of beneficiaries) was increased by 574% (the combined effects of these are shown by the drastic fall in unit costs between original and final target). The shortfall in the achievement of the number of beneficiaries target resulted in actual unit cost exceeded the revised (final) target by 70%.

Figure 3.11 Anticipated and Achieved Unit Costs of Measure 313 budget

Based on substantially varying unit costs it is difficult to assess if the SRDP has been delivered efficiently.