Monitoring the outcomes of planning: a research study

8. Key findings and conclusions

8.1 The study brief focused on monitoring the outcomes of planning. This report places outcome monitoring in the context of a broader performance management framework for the Scottish planning system ( Section 2).

8.2 The background to the study was a recommendation by the independent review panel that there should be a move towards monitoring planning outcomes and away from a narrow focus on procedure. This recommendation has been pursued and tested through the review process, and the consultations revealed broad support for the panel's conclusions. The Planning (Scotland) Bill proposes measures to strengthen performance management, including greater scrutiny of planning authorities. Feedback from our consultations and the workshops suggests that many would prefer a more holistic approach, monitoring the performance of the Scottish planning system in totality ( Section 3).

8.3 The changes proposed by the independent review panel and endorsed by Ministers need to be seen in the context of the National Performance Framework and the 16 National Outcomes. They are part of the Scottish approach to public service reform, which recognises that traditional, "cause and effect" performance management models have been challenged by a clearer understanding of the complex environments in which public services operate ( Section 4).

8.4 A number of experts have concluded that applying an outcomes-based performance management model to the planning system is inherently difficult. Monitoring the core technical and regulatory planning functions is relatively straightforward, but, increasingly, policymakers expect planning to deliver "better places" or to contribute to wider policy goals such as health and wellbeing, learning, productivity, community cohesion and climate change resilience. Measuring the impact of planning on this much wider agenda is "a wicked problem" ( Section 5).

8.5 The Policy Memorandum accompanying the Planning (Scotland) Bill 2017 confirms that the core purpose of planning continues to be "to guide how land should be used to meet the needs of society", but Scottish Planning Policy stresses the Government's wider ambitions to create the kinds of places where people want to live, work, relax and invest and to contribute to economic, social and environmental wellbeing more generally ( Section 5).

8.6 The report introduces a performance management logic chain ( Figure 5-2) which would (i) capture planning inputs, activities and outputs, (ii) enable the outcomes of planning to be monitored, and (iii) capture through post hoc evaluation the direct impacts (better places) and the indirect impacts on the wider policy agenda. The current Planning Performance Framework focuses primarily on monitoring activities and outputs, with very limited coverage of outcomes ( Section 5).

8.7 The report presents a draft performance management framework for the Scottish planning system ( Figure 6-2). The framework summarises performance management activities and measures at each stage, and distinguishes between ongoing monitoring activity and post hoc evaluation. We show how this model could expand the scope of performance management activity ( Section 6).

8.8 There is strong support for moving towards a regime of outcome monitoring and impact evaluation, but there are no ready-made models to adopt. There is a consensus that progress should be made pragmatically and incrementally, not "letting the perfect be the enemy of the good". A staged approach is suggested: first, an expansion of the scale and scope of performance monitoring to include a wider range of outcomes; next, an initial tranche of impact evaluation studies, focusing on direct (placemaking) impacts; finally, more wide-ranging evaluation studies addressing the indirect impacts of planning ( Section 7).

8.9 To be useful, monitoring should be a universal system, capturing outputs and outcomes from all planning applications with the exception of minor works. By contrast, subjects for in-depth evaluation studies will need to be selected based on agreed criteria ( Section 7).

8.10 Figure 7-3 examines monitoring in more detail, and stresses the need for informed interpretation of the results, taking account of local market conditions and other variables as well as the policies of individual planning authorities; a number of quantitative measures are identified. The report discusses the case for creating a national planning database, which could enable reliable and comprehensive on-demand and periodic reporting ( Section 7).

8.11 A model for impact evaluation (direct and indirect) is set out in Figure 7-4. Evaluation studies will need to be conducted by independent experts, using guidance created for the purpose. Projects or groups of projects will be selected on the basis of scale, strategic significance or sensitivity, and baseline studies will need to be conducted at the pre-application stage to enable robust before-and-after analysis. The report records strong support for using the Place Standard as a tool for assessing place quality, augmented by national and local policies, best practice case studies, awards nominations and other sources. These sources will also be valuable as the focus extends to include evaluation of the indirect impacts of planning ( Section 7).

8.12 The benefits of developing and adopting a comprehensive performance management framework will only be fully realised in the context of a culture of learning, innovation and continuous improvement. The report includes suggestions for publishing and disseminating lessons from monitoring and evaluation, and feeding those lessons back into the local and national policy loop. The national planning performance coordinator, the High Level Group on Planning Performance, HOPS, RTPI Scotland and others will have a key role to play in achieving culture change ( Section 7).

8.13 The report concludes with a brief commentary on resources. There is strong support for a move towards a more holistic performance management system, but planning authorities are under pressure. There will be short-term costs associated with developing new systems, including the planning database, preparing guidance and training staff. Once the new systems are embedded, the monitoring process should be more-or-less resource neutral because all the data required is already being collected but not yet stored on a database. Evaluation studies will be conducted by external experts, and there will need to be a discussion about how they will be funded. Project managing such studies will be a significant responsibility for planning authorities ( Section 7).

yellow book
Nick Wright Planning

May 2018