Developing Performance Indicators for Rural Scotland: A Scoping Study

2.0 Background

2.1 Why develop rural specific indicators?

With rural policy in Scotland mainstreamed across the five strategic objectives set out previously, there is a need to ensure that the particular issues and needs of rural areas - particularly those in the more remote rural regions - are captured in the Scottish Government's National Performance Framework.

Whilst this report does not anticipate or propose that any new 'rural' indicators should be incorporated into the performance framework, it is beneficial to consider whether a supporting or shadow set of rural indicators should be established that could provide finer grain focus upon key issues facing rural Scotland, providing help to policy makers and adding value to the decision-making process.

2.2 Defining rural Scotland

Critical to the development of a set of meaningful indicators for rural Scotland is the need to understand what is meant by 'rural Scotland' and for data supporting indicators to be available at the appropriate granularity to allow for targeted responses to be made.

For the purposes of this report we have primarily followed the Urban-Rural Classification as developed by the Scottish Executive in 2000 and subsequently updated every two years ⁸ . This methodology provides a simple classification which distinguishes between urban, rural and remote areas on a six and eight category basis, according to settlement size and accessibility based on drive time. The 6-fold and 8-fold classifications are illustrated in Maps 2.1 and 2.2.

Map 2.1: 6-fold Urban Rural Classification

Map 2.2: 8-fold Urban Rural Classification

Source: Scottish Executive (2006)

2.2.1 OECD definition

An alternative methodology for defining urban-rural Scotland is provided by the OECD and is useful to understand for the purposes of this report as much of the comparative data from case study countries is taken from the OECD and the benefits of using their methodology allows comparability across countries.

In brief, the OECD methodology provides a three fold classification based upon population density and defines predominantly rural regions, intermediate regions and predominantly urban regions according to the share of the population that live in areas of less than 150 inhabitants (more than 50%, 50% to 15%, and less than 15%, respectively). This is illustrated in Map 2.3 below:

Map 2.3: OECD urban-rural definition applied to Scotland

Source: OECD (2008)

2.2.2 The Randall definition

A further analysis of the spatial geography of rural Scotland is provided by the Randall definition, based upon allocating a definition of urban/rural to local authority areas based upon population density, as illustrated in Map 2.4 below:

Map 2.4: Rural Local Authorities in Scotland

This definition is set out in addition to those above in recognition that some data are only available down to Local Authority level. Although this provides challenges in terms of data interpretation, and Local Authorities classified as 'urban' may include rural areas, it is possible to use such data at a strategic level where fine grain analysis may be achieved through local level investigations.

2.3 Identifying good performance indicators

The key task of this study is to consider the scope for developing a set of rural performance indicators for the Scottish Government. In order to do this effectively it is necessary to understand the critical components of a good performance indicator. The necessary features of an effective indicator can be distilled into four principal requirements that must be satisfied if it is to be effective. An indicator must be:

• Strategic - the indicator must be linked to the strategic objectives of the programme/policy/intervention.
• Quantifiable - it must be possible to quantify the indicator and definitive targets must be able to be set against which progress or change can be assessed.
• Continuous - it must be possible to collect data relating to the indicator from sources that are updated at regular intervals or continuously.
• Comparable - it must be possible to compare the indicators with other recognised norms or standards at the local, sub-regional and national level.

2.3.1 What makes a good indicator?

Research compiled by the Department of Land Economy at the University of Cambridge ⁹ identified six essential features of a good indicator:

• A good indicator should measure conditions in a locality at a point immediately prior to the commencement of a particular policy, programme or project intervention. Ideally, local conditions should be measured relative to some standard or norm that may represent a wider local, sub-regional, regional or national average. This comparison helps to identify whether a given change in the value of an indicator is due to the impact of programme in question.
• The same conditions should be capable of being revisited at the end of (and fairly continuously during) the implementation period of the policy, programme or project intervention, again relative to a standard comparator. This converts the static baseline indicator into a dynamic measure of change. It serves to reveal the degree of local change irrespective of cause.
• Indicators that are designed to measure outcomes should relate to a broader set of local changes than are captured by indicators that measure specific outputs. A good indicator must be capable of being influenced by programme or project interventions but are also subject to a wide variety of other policy and non-policy influences.
• Indicators should be measured consistently across an entire policy, programme or project intervention.
• Indicators should as far as possible be quantitative in nature even when it is qualitative change that is being assessed.
• A good indicator should be capable of measuring change or progress.

The last feature is particularly relevant. It is not enough for an indicator to simply establish local conditions at a particular point in time. In order to be meaningful, an indicator should also be capable of being used to set definite targets against which change or progress can be assessed. There are three types of targets that can be adopted:

1 Directional change - to increase/decrease the incidence of the condition measured by the indicator.

2 Absolute change - to increase/decrease the incidence of the condition measured by the indicator by a specified interval or ratio.

3 Gap change - to increase/decrease the incidence of the condition measured by the indicator by a specified interval or ratio to better reflect the norm or standard in a comparator area or in a larger area of which the local area is part.

We are also able to think of indicator suitability in terms of their fit with the ' AIMS' criteria - are they Action oriented, Important, Measurable and Smart:

• Action oriented - does the indicator provoke people to debate, change the way they think and behave? Is it obvious to all in which direction policy should be going? Can achievable targets be set?
• Important - does the indicator describe an important asset of the issue clearly and unambiguously? Is the indicator relevant to the local area and community? Does it reflect national and global concerns?
• Measurable - can you identify where to get information for this indicator? Can data for the indicator be collected?
• Smart - would most people be able to understand what the indicator is about, and find it interesting?

2.3.2 The choice of spatial building block

In order to be effective, indicators must pass two principal tests:

• Are they appropriate to the area concerned?
• Are they measurable at an appropriate spatial level?
• Indicators must relate to the strategic objectives against which they are seeking to assess impact and progress. The assessment of measurability breaks down into a further three tests that must be passed:
• Can the data set be produced to fit the geographical boundary of the area concerned?
• Can the information and data sets be easily updated?
• Can the data sets be updated at time intervals useful for the purposes of the project?

The first test is often difficult to meet and this has a negative impact on meeting the second and third of the above tests. Many secondary statistical sources provide information at a national and regional level, and also for Local Authority or 'travel to work areas'. Fewer produce results for smaller areas because of the limited sample size or, in the case of administrative data, to avoid disclosure. Some small area data has to be constructed by combining samples from several years.

2.4 Conclusion

In this chapter we have discussed the need for rural specific indicators. We have explored the different methodologies in classifying urban and rural areas in Scotland and while the Scottish Government 6-fold classification is most appropriate, other classifications can be useful in some circumstances too. We have set out what makes a good performance indicator and concluded that one of the most important features is ability to be able to track the change in the indicator over time.