International Development - national indicator development: research report

5. Future Development of the Indicator

5.1. Potential changes in the indicator in future

5.1.1. The literature on how policy coherence and the activities of non-governmental actors lead to sustainable development is broad but often imprecise. A number of countries have established monitoring systems that are intended to track progress in policy coherence but on closer investigation contain vague concepts rather than measurable indicators of change. Large scale evaluations have so far failed to establish an impact that can be clear attributed to policy coherence action. Moreover, these evaluations are largely focused on policy activity, while the indicators for the NPF is intended to measure the contribution of development support to other nations from Scotland as a whole.

5.1.2. This suggests that greater attention to the Beyond Aid agenda and related policy coherence activity will enhance our understanding of the critical factors that deliver results. Further research and evaluations should help bridge the 'logic chain gap' and may offer a new source of potential indicator compoenents to be included this NPF index.

5.1.3. This implies that Scotland should keep the NPF measure under review and seek to update and amend the individual indicators periodically as necessary. There would appear to be a number of areas where this may be worthwhile in future:

• Include more policy areas if and when these are devolved to Scotland.
• Seek to include wider measures of partnership working and the contributions of Scottish expertise that is not captured in simple input measures. These remain key advantages in the Scottish approach that many stakeholders pointed to as having a key influence in their relationships with ODA countries. While they have proved intractable to a low-cost data collation process, new data sources may become available in future that could address this gap.
• A more specific gap that may be filled by Scottish Government itself is the reporting on partnership activity associated with the International Framework Strategy. At present no data is collected on the spend on partnerships at this level but this may also change in future.

5.1.4. Other data sources should persist into the future as most are based on official statistics. The calculation of the value of the low carbon economy is a specific large scale survey initially undertaken by BEIS but now adopted by National Statistics. This is to undertaken biennially. However, as with all survey evidence there are no guarantees that these will persist for the next decade.

5.1.5. Other sources such as data on the Grand Challenges Research Fund (GCRF) and Newton research funds are expected to include greater detail on the nature of research partnerships with ODA countries and networks. These are in development but may provide source data that could add to the indicators on research collaborations between research organisations in Scotland and developing countries.

5.2. Stability and Sensitivity testing

5.2.1. The previous section highlights that due to a limited number of observations, we have been able to undertake only limited analysis of the stability and sensitivity testing. As more data becomes available it should be possible to explore how the value of the index responds to changes in individual indicators. As there is no weighting involved, any variation in the overall value of the index will be driven by the scale of change (year-on-year) in the individual indicators. The use of a geometric average does have the effect of dampening any significant variations in individual indicators and is the primary reason for its adoption.