Monitoring the outcomes of planning: a research study

5. OBPM and the planning system

5.1 In this section we examine how the principles of outcomes-based performance management ( OBPM) might be applied to planning. This has long been seen as a challenging task. In a report commissioned by the RTPI on the challenges of Measuring the Outcomes of Spatial Planning in England, the Centre for Urban Policy Studies (2008) noted the methodological difficulties of measuring "the effectiveness and outcomes of planning…due to the complexity involved in spatial planning activities and the limitation of any single method as a means of effectively measuring the outcome and impact of these activities".

5.2 Wong (2006) described developing an outcomes-based approach to planning as "a wicked problem". Planning outcomes and impacts are resistant to "traditional linear analytical approaches". Planning operates within "an evolving set of interlocking issues and constraints" and it is "always embedded in a dynamic social context, which makes each problem unique".

5.3 The difficulty stems from the long-term evolution of policy thinking about planning in Scotland, the UK and other countries, and the expanding scope of policy aspirations. Traditionally, planning has been seen as an essentially technical and regulatory activity (managing land use and development), and this continues to be the core function of the planning system. In recent decades, however, there has been a much greater emphasis on the purpose of planning, and in particular the concept of placemaking. Scottish Government policy statements on architecture, the work of Architecture + Design Scotland, CABE and others have explored and attempted to codify the attributes of "better places". The Place Standard was developed as a tool for measuring aspects of place quality, albeit subjectively. Research around these themes has increasingly focused on the potential of "better places" to contribute in turn to a much wider set of goals, including the National Outcomes (see Figure 5-1). ^[2]

Figure 5-1: Planning – core functions, placemaking and wider benefits

5.4 There is a broad and growing consensus that good design and better places can make a positive contribution to better health and learning outcomes, to social cohesion, business productivity and climate change resilience, among others. But, for the reasons outlined in Section 4, attributing and quantifying such benefits is extremely difficult.

5.5 The Scottish's Government's Guide to the Planning System in Scotland (2009) focuses firmly on the core functions of planning:

"The planning system is used to make decisions about future development, and the use of land in our towns, cities and countryside. It decides where development should happen, where it should not and how development affects its surroundings.

"The planning system exists to regulate the use of land and buildings by granting or refusing planning permission. Decisions about planning applications are based on the development plan for your area, which is prepared by your local council or national park authority.

"The planning system balances competing demands to make sure that land is used and developed in the public's long-term interest."

5.6 The Policy Memorandum that accompanies the Planning (Scotland) Bill 2017 reaffirms this message:

"Development, such as new homes and facilities and places for people to work and to spend their leisure time, has always been needed and places continuously change. The purpose of planning is to guide how land should be used to meet the needs of society. Scotland has a plan-led system, whereby policy and proposals for how its areas will develop in future are set out in the statutory development plan. Planning authorities are required to make their decisions on planning applications in accordance with the development plan unless other material considerations indicate otherwise".

5.7 So, while planning in Scotland is expected to contribute to a broad range of policy goals, its core purpose continues to be "guid[ing] how land should be used to meet the needs of society". It is a spatial discipline that seeks to achieve the optimal distribution of development in cities, towns and the countryside. These spatial goals are reflected in the National Performance Framework which includes two National Outcomes which derive directly from planning activities:

• NO 10: We live in well-designed, sustainable places where we are able to access the amenities and services we need
• NO 12: We value and enjoy our built and natural environment and protect and enhance it for future generations

5.8 The language of these National Outcomes confirms that planning is concerned not only with the allocation and distribution of development, but with the quality of the places that are produced as a result. A range of policies and resources offer guidance and good practice exemplars to help all the actors in the planning system achieve placemaking goals as well as the delivery of development. These resources include:

• Creating Places, the Scottish Government's policy statement on architecture and place (Scottish Government, 2013)
• the advisory services provided by Architecture + Design Scotland and local authority design panels
• local development plan policies
• design guides, and
• the Scottish Government's Place Standard.

5.9 The potential strategic impact of planning is articulated in Scottish Planning Policy (Scottish Government 2014) which sets out the following vision:

"We live in a Scotland with a growing, low-carbon economy with progressively narrowing disparities in well-being and opportunity. It is growth that can be achieved whilst reducing emissions and which respects the quality of environment, place and life which makes our country so special. It is growth which increases solidarity – reducing inequalities between our regions. We live in sustainable, well-designed places and homes which meet our needs. We enjoy excellent transport and digital connections, internally and with the rest of the world."

5.10 Scottish Planning Policy introduces four key planning outcomes. The accompanying descriptions provide a useful commentary on the ways in which planning outcomes might contribute to placemaking and other policy goals:

Outcome 1: A successful, sustainable place – supporting sustainable economic growth and regeneration, and the creation of well-designed, sustainable places.

By locating the right development in the right place, planning can provide opportunities for people to make sustainable choices and improve their quality of life. Well-planned places promote well-being, a sense of identity and pride, and greater opportunities for social interaction… promoting strong, resilient and inclusive communities. Delivering high-quality buildings, infrastructure and spaces in the right locations helps provide choice over where to live and style of home, choice as to how to access amenities and services and choice to live more active, engaged, independent and healthy lifestyles… By allocating sites and creating places that are attractive to growing economic sectors, and enabling the delivery of necessary infrastructure, planning can help provide the confidence required to secure private sector investment, thus supporting innovation, creating employment and benefiting related businesses.

Outcome 2: A low carbon place – reducing our carbon emissions and adapting to climate change.

By…encourag[ing] mitigation and adaptation measures, planning can support the transformational change required to meet emission reduction targets and influence climate change. Planning can also influence people's choices to reduce the environmental impacts of consumption and production, particularly through energy efficiency and the reduction of waste.

Outcome 3: A natural, resilient place – helping to protect and enhance our natural and cultural assets, and facilitating their sustainable use.

By protecting and making efficient use of Scotland's existing resources and environmental assets, planning can help us to live within our environmental limits and to pass on healthy ecosystems to future generations. Planning can help to manage and improve the condition of our assets, supporting communities in realising their aspirations for their environment and facilitating their access to and enjoyment of it. By enhancing our surroundings, planning can help make Scotland a uniquely attractive place to work, visit and invest and therefore support the generation of jobs, income and wider economic benefits.

Outcome 4: A more connected place – supporting better transport and digital connectivity.

By aligning development more closely with transport and digital infrastructure, planning can improve sustainability and connectivity. Improved connections facilitate accessibility within and between places – within Scotland and beyond – and support economic growth and an inclusive society.

5.11 These goals reflect the Scottish Government's commitment to the United Nations' Sustainable Development Goals ( SDG), in particular Goal 11: Make cities and human settlements inclusive, safe, resilient and sustainable.

5.12 In their report on The Value of Planning, also commissioned by the RTPI, Adams & Watkins (2014) state that: "Planning is a much broader activity than the narrow regulatory role to which it is relegated by many economists and some politicians. Planning helps to create the kinds of places where people want to live, work, relax and invest – often termed 'shaping places'. Planning is about improving places by helping them to function better economically as well as socially and environmentally." In a conclusion consistent with the findings of the independent review, Adams & Watkins conclude that planning is "about outcomes, not just processes", and they argue that "planning is not always done by people called 'planners'".

5.13 Based on the dictum attributed to the US management guru, W Edwards Deming that "every system is perfectly designed to achieve the results it is getting", it could be argued that place quality (measured by these and other standards) is the ultimate test of the efficacy of the planning system. But, while place quality remains a key goal, the lessons of the Scottish approach to public service reform (Section 4) should remind us of the complexity of the placemaking process and the inherent difficulties of attribution.

5.14 Places are shaped by multiple actors. Local authority planners and politicians create the policy frameworks and seek to enforce them, but every development project of even modest scale will also involve private, public or third sector developers, investors and their professional advisers, architects, statutory bodies, community representatives and local residents.

5.15 Some major projects may be considered to be an exercise in placemaking in their own right: more often, and especially in urban areas, places are shaped cumulatively by an aggregation of individual developments. Development management can be seen as the process of choreographing the efforts of different projects and agents to achieve the best possible place outcomes, balancing the interests of all the players.

5.16 Development occurs in a market context. During our consultations it was argued that planning authorities are better placed to deliver both a greater volume of development and better place quality in areas where consumer/ occupier demand is strong and where developers are active. In these locations a range of development types are viable and attractive to developers and this gives planning authorities greater leverage to "demand better". By contrast, in places where demand is weak and viability is marginal, it may be more difficult for planners to exert a positive influence. It was suggested to us that, in some areas, there is a tendency to think that "any development is better than nothing". The result may be a little-recognised form of inequality, with prosperous communities more likely to attract better quality design and placemaking, while the least well off have to settle for lowest common denominator development. In these circumstances, it was argued that public sector and third sector developers (such as housing associations) need to act as "design champions", commissioning quality projects that will help to close the "place quality gap".

5.17 The Centre for Urban Policy Studies looked beyond placemaking outcomes to argue that "the planning system is now more than ever concerned with promoting the role of planning as a coordinator, integrator and mediator of the spatial dimensions of wider policy streams… This broader role represents an explicit extension of the scope of planning beyond its traditional focus on mediation, management and monitoring land use and physical change within localities." Spatial planning is concerned with policy concepts such as "liveability", "sustainable development" and "sustainable communities", but such concepts may be "too holistic and vague" to be of operational value. They may need to be replaced with "an alternative formulation" of more specific desired outcomes.

5.18 The brief for this study is predicated on planning as an agent for change across policy domains. Adams, O'Sullivan et al (2016) argued that planning has often failed to gain traction, or at least to demonstrate its effectiveness, in this wider policy role:

"…we now have planning systems which struggle to deliver widely-shared economic, social and environmental goals. There is an urgent need to take stock of the planning systems we have now, what they can deliver, and to debate alternative futures for planning that might produce much better results."

5.19 In language which prefigures the findings of the independent review, the authors argue that planners need to focus more on achieving better economic, social and environmental outcomes, and talk "less about planning procedures and processes". The "value of planning" needs to be better understood so that positive outcomes can be achieved and maximised. If measuring the impact of planning on place outcomes is difficult, identifying the effects on wider policy objectives will inevitably be even more challenging. Such effects may be hard to measure in any event, even more so because "delivery is heavily reliant upon the actions of a plurality of actors and agencies across different operationally independent policy sectors".

5.20 The idea that planning can reach beyond its core functions of managing land use and development, to help to create better places and to exert a positive influence on the wider policy agenda has become a conventional wisdom. It justifies concerns that the current performance management regime is insufficiently concerned with the direct outcomes of the development management process, let alone its wider (direct and indirect) impact on the Government's National Outcomes. But this study has confirmed that measuring the outcomes and, especially, the impacts of planning is inherently difficult. Theoretical models of the type set out in reports commissioned by the RTPI or by Arup (2011) for the Welsh Government may prove to be too complex and time-consuming in practice.

5.21 The clear message from the practitioners we spoke to in the course of this study has been that, while we must and should do better in terms of monitoring planning outcomes and evaluating impacts, policymakers must be pragmatic: they "should not let the perfect be the enemy of the good". During the course of the study we developed and tested the logic chain shown in Figure 5-2, which is a no-frills model consistent with the Treasury ROAMEF framework (see Section 6). The terminology used here is discussed in more detail in Annex 1.

5.22 We had a positive response to this model, which we have adopted as the organising framework for the latter stages of this report. It is important to stress that it is a work in progress: the ideas and suggestions contained in the following pages are not presented as policy recommendations, but are intended to provide a basis for future policy development.

5.23 The model is explored in more detail in the following section, but some points are worth highlighting here:

• there is a linear connection between inputs, activities and outputs, and – to a lesser degree - outcomes
• planning authorities have a high level of control over inputs and activities, and a significant degree of control over outputs
• planners and policymakers already monitor activities and outputs through the Planning Performance Framework (see below) and they have the capability to monitor planning outcomes, using data which are already available
• planning authorities have a strong but not decisive influence on planning outcomes, a significant influence on the direct (placemaking) impacts of planning, and a degree of influence on the indirect (wider policy) impacts
• impacts (direct and indirect) can be assessed through post hoc evaluation, drawing on monitoring data as well as qualitative assessments.

5.24 There is already a Planning Performance Framework ( PPF) for Scotland. The framework, which was developed by Heads of Planning Scotland ( HOPS) in conjunction with the Scottish Government, was launched in 2011, and the most recent refreshed guidance notes were issued in April 2018. The PPF involves "a mix of qualitative and quantitative measures to provide a toolkit of indicators". All planning authorities and SDPAs submit annual reports, which are complemented by a peer review process.

5.25 As shown in Figure 5-2, the scope of the PPF (the area between the dotted lines) is largely confined to the operation of the development management process. It monitors the planning process through activities and outputs, with only very limited coverage of outcomes, although the reporting proforma inaccurately describes measures of activity and outputs as "key outcomes". Figure 5-3 overleaf analyses the PPF National Headline Indicators, distinguishing between activities/outputs and outcomes.

5.26 The PPF captures important management information, especially in relation to the determination of planning applications and the speed of decision-making but it is an overwhelmingly process-orientated tool. It is essentially inward-looking and narrowly focused on plan-making and the efficiency of the development management system. It is of more value to those who engage directly with the planning system (planners, policymakers, developers and others) than to citizens and communities. The latter are more likely to be concerned with planning outcomes – development on the ground – and the impact of that development on place quality. The conclusion is that Scotland needs a more broad-based, outcomes-based performance management framework for the planning system. This would require two key steps:

• an extension of the monitoring regime to include planning outcomes – for example, the volume of development on the ground and progress towards local and national targets, and
• a regular programme of post hoc evaluations to assess the direct impact of planning on place quality and the indirect contribution of planning to the achievement of National Outcomes.

Figure 5-3: PPF National Headline Indicators

5.27 A number of our consultees challenged the implicit assumption of the PPF that a decision to grant planning permission should be seen as a positive output of the planning process, and that implementation of that consent is, by extension, a positive outcome. This concern echoes a theme of recent research on barriers to community engagement in planning (yellow book, 2017) which found that citizens who get involved in the planning process believe that "planning appears to be driven more by delivering development than by placemaking". It follows that a future performance management system for Scotland needs to recognise that the refusal of an inappropriate and/or poor quality development is beneficial. Indeed, as the Guide to the Planning System in Scotland (Scottish Government, 2009) points out, stopping the wrong development in the wrong place is precisely what the planning system should be doing. In this context, no development may be a positive outcome.