Town Centre Regeneration: How Does it Work & What can be Achieved?

5 CHALLENGES IN ASSESSING THE SUCCESS OF TCRF

Introduction

5.1 An initial aim of this research was to identify wider lessons from the case study projects which could then inform future approaches to town centre regeneration. This chapter explores some of the challenges identified by the research team in assessing TCRF interventions as a group, and the case studies in particular.

5.2 The information in this chapter is based on the consultations with the case study partners and the literature review into approaches to town centre performance assessment.

5.3 The chapter considers four sets of issues including:

• The nature of the Town Centre Regeneration Fund;
• Limitations in planning and specifying outcomes and outputs;
• Weaknesses in approaches to monitoring town centre performance;
• Lack of robust monitoring and evaluation plans.

The Nature of Town Centre Regeneration Fund

5.4 The aim of the TCRF was to support community and business leaders to regenerate and grow town centres in order to meet the needs of local communities and businesses. The fund was capital only and so could not support non-investment activity. The fund was available from 1 April 2009 to 31 March 2010, with deadlines for applications in June and August 2009. All funding had to be claimed by March 2010. The way in which the TCRF was established with capital only funding, tight deadlines and the criteria used to select projects present challenges in terms of developing effective evaluation plans with the case study projects.

5.5 The first issue relates to the relatively short period of time for the development of applications. Seven out of the nine case study partners noted that the timescale for the preparation of TCRF applications was short. Where local authority departments responsible for town centre regeneration had been directly made aware of the fund they then had around a six week period in which to develop initial applications. Where the applicant was not a local authority, as in the case of the Millport Community Development Company, there was a time lag in being made aware of the Fund. This meant that the Community Development Company only had a two week window in which to prepare the application. (See TCRF Case Studies report).

5.6 This short period for project development and planning had two important implications: it influenced the nature of the projects that were submitted to the fund; and it restricted the time available to develop robust implementation and monitoring and evaluation plans. The latter issue is discussed in more detail later in the chapter, however the first point, in terms of the nature of the projects submitted to the fund is discussed briefly below.

5.7As a result of the short time for the development of applications, many of the projects that were submitted to the fund were project ideas that had been developed some time ago and were 'taken off the shelf' for submission to the fund. These projects were often physical in nature and had been difficult to implement due to the previous lack of funding availability.

5.8This initial potential bias towards physical interventions was compounded by the eligibility criteria used to assess projects. In particular, the availability of relatively substantial funding and crucially, the original requirement to spend the monies before the end of March 2010, meant that physical projects stood more chance of success. Indeed, analysis of the TCRF projects shows that the successful bids were heavily biased towards physical interventions (See Table 5.1)

5.9 Across all of the projects, the most common primary activities were public realm, accessibility and townscape improvements, as shown in Table 5.1. The table shows that this was the primary activity of 27 of the 65 projects, equating to 42%. The next most popular primary activities were housing, leisure & recreation, or community facilities (28%), and business, commercial or retail space (22%). Between them, these three categories account for some 92% of all primary activities. Relatively few projects implemented new transport infrastructure or safety, security & crime reduction measures as their primary activities.

Table 5.1: Summary of Primary TCRF Funded Activities

Activity	No.	%
Public realm, accessibility and townscape	27	42%
Business, commercial or retail space	14	22%
Housing, leisure & recreation, or community facilities	18	28%
Transport Infrastructure	4	6%
Safety, Security & Crime Reduction	1	2%
Other	1	2%
Total	65	100%

5.10 The earlier chapters argued that town centre regeneration has increasingly come to be seen as a wide ranging activity that can include activity targeted at businesses, people, community and civic interventions. The TCRF's overwhelming focus on physical interventions therefore means that care needs to be taken about extrapolating lessons from the case studies, and the TCRF as whole, for town centre regeneration more widely.

Limitations in Planning and Quantifying Outcomes

5.11 Learning from the TCRF projects and case studies is made difficult by a number of weaknesses at the project planning stage in the case studies projects, some around implementation and others around the clarity and logical linkage of the potential outputs and outcomes from TCRF activity. There were a number of aspects to this.

5.12 The first relates to the planning of project implementation. The majority of the case studies experienced slippage of some sort or another, however, this was largely due to factors out with the control of the project. Of more concern, at least in terms of approaches to evaluation, is that the implementation of activities was often poorly specified. An example of this would be the investment in ducting for improved broadband capability in one of the case studies, where the target groups that would make use of that improved capacity were not clearly identified. Added to this, there was no explanation of the scale of the expected improvement against the current position. These are important issues to address if there is an aspiration to carry out effective evaluation of project interventions. These issues are explored further in Chapter 6.

5.13 The second relates to the use of inconsistent terminology and definitions with regards to activities, outputs and outcomes. No specific Scottish Government guidance on this was provided in the TCRF application process. Whilst the TCRF funded projects had provided information on their activities and anticipated outcomes, this information lacked consistency in the way these terms were defined. Figure 5.2 illustrates one of the change paths as expressed in the initial TCRF bids.

Figure 5.2: Change Path as Detailed in Inverclyde TCRF Application

Activities	Economic Outcomes	Social Outcomes	Environmental Outcomes
Upgrade entrance portals/signage and landscaping Provide 4 training places for young apprentices Employ local companies to undertake specialist elements of work Create Wi-Fi access Remove graffiti/fly posting New pavement design Recycle materials wherever possible Use low carbon structure in design	Raise image quality and appeal of town Benefits to businesses Support SMEs, maximise tourist potential Support retail areas around key arrival points Enhance commercial appeal and business start ups Higher footfall on core retail streets Extend economic opportunities to secure local jobs, local facilities and support Attract visitors to retail core Increased demand for goods and services Encourage more active use of centre	Convey perceptions of modern, vibrant culturally inspired town centre Create more appealing safer cleaner centre Reduce perceived and actual crime and antisocial behaviour Provide a strong distinctive centre Build capacity and confidence Make centre the default place for meeting, civic activity, social interaction Improve civic pride	Appealing and distinctive welcoming image Create better connected and functional town centre Improve links to ferry /bus/rail interchanges Reconnect communities to town heritage Better accessibility across modes of transport Reduce car mileage

5.14 In many instances, as above, the TRCF plans expressed outcomes as activities ( e.g. support SMEs, encourage more active use of centre) or outputs as outcomes ( e.g. benefits to businesses). Activities were rarely described in a manner that identified their intended target group. Plans often did not link specific activities and outputs to specific outcomes and did not time order the outcomes into short, interim and longer term. The draft Theories of Change developed by the research team (see chapter 6) illustrates how logic models can be used to better differentiate between activities and outputs or outcomes and to illustrate links between, and order of, these elements.

5.15 The third aspect relates to the nature of the outputs and outcomes that were claimed by successful bids. The research team attempted to undertake a detailed review of all types of these outputs and outcomes but this proved to be impossible to quantify given their very wide range and the different ways in which the indicators for these were described.

5.16 A significant proportion of projects made claims around potential employment outcomes or other economic outcomes. There were also a number of common outcomes and outputs claimed including retail floorspace created, business units created, the retention of retail spend, the attraction of new visitors or improving perceptions of the place, for visitors and for residents. While s ome outputs and outcomes were claimed routinely for many projects, there was often little attempt to quantify them. This is highlighted by the analysis of the quality of employment outcomes that were claimed across the 65 projects in Table 5.3 below.

Table 5.3: TCRF Projects Claiming Job Creation / Safeguarding

Projects Claiming to Increase / Safeguard Jobs
Yes	54
No	11
Total	65
Number of projects quantifying this	33

5.17 A further significant issue in relation to employment outcomes is that no universal standards/guidelines were applied as to how projects were to report potential job creation/safeguards figures. This is reflected in the way in which job creation figures are presented in the TCRF application forms. For instance, some projects are able to tightly define expected job creation ( e.g. the number of jobs created through construction phase; or the number of jobs expected once the new scheme/facility is completed). In other cases, projects are much more vague and some simply state that the project will support jobs within the wider area.

5.18In other cases, even where outputs and outcomes were quantified, they were measured in different ways. For example, some projects claimed outputs of business or retail space improved in terms of sqm; others claimed potential outputs in terms of units improved. Both of these issues would have made the comparative assessment of the potential benefits of projects by the Scottish Government very difficult, and also made it impossible within the study parameters for the research team to assess the potential contribution of TCRF as a whole.

5.19 There was also a lack of consideration of the timing of outputs and outcomes. This is particularly important given the largely physical nature of the projects supported through TCRF. Our review of the rationale and justification of some case study projects revealed that physical investment was often justified in terms of acting as a stimulus for further investment. This was a recurrent theme in the Airdrie, Kirkintilloch, Barrhead and Govan projects. In these cases, however, the potential employment outcomes from later investments was often claimed in whole by the original TCRF intervention.

5.20Overall, the research team's review of the TCRF projects as a whole and the case studies suggests that in some cases, there was a lack of critical thinking at the planning stage. Often, the logical relationship between the initial intervention and the economic and social outcomes and outputs it expected to bring was not considered. The most powerful example of this was the Kirkintilloch case study, where it was claimed that the removal of the Town Hall Annex would create footfall by opening up a new footpath between the town's historic core and retail core. This is in spite of the fact that the Council recognised there was a future need for a significant public realm improvement to bring the path into use and remove an eight foot wall that lay across the proposed footpath.

5.21 On the other hand, in Barrhead, more consideration had been given to the public realm improvements at Cross Arthurlie Street and Main Street, including short and medium term outcomes, such as the inclusion of signage, improvements to walking/cycling links and active business engagement. This has resulted in the local businesses now actively considering the possibility of a BID.

5.22 As part of the Jedburgh case study the purchase of the Porthouse Project was the first stage of a long term strategy. The property has now been successfully acquired and has been made wind and water-tight. In the short-term the ground floor unit is available for let. An options appraisal has been undertaken to advise on the most appropriate future use and an application is currently being progressed to secure lottery funding to undertake a full business case for the project.

5.23 The development of a destination play park as part of the Jedburgh project was also a well considered approach. It was felt that Jedburgh could capitalise better on its location on the A68 and needed further attractions to encourage people to visit the town. The development of a destination play park along with adequate parking and enhanced signage aimed to provide a reason to stop in the town, make it easy to park and provide good direction to other town centre services.

Limitations in the Availability and Use of Routine Data and Town Centre Health Checks

5.24 Effective planning ( e.g. quantifying and detailing timescales for the potential benefits from an intervention or project) is only one element that is necessary to carrying out meaningful evaluation. Any serious evaluation also needs to consider the effect of the intervention against baseline conditions. In the case of the TCRF this would mean assessing the change in the performance of the town centre over time before, during and after a project implementation.

5.25 The research confirmed that the approaches to monitoring the health of town centres vary considerably across the case study towns. In some of the smaller centres such as Millport, there had been no plans in place to monitor the health of the wider town centre. At the other end of the scale, in Barrhead health checks are undertaken every two to three years, and in North Lanarkshire the Council is now aiming to undertake annual health checks across seven town centres. In Elgin, an annual town centre health check will be undertaken which will contribute to the monitoring of the project.

5.26 However, this is not an issue that is confined solely to the TCRF case study towns. The experience of the Scottish Government Town Centre and High Street Learning Network confirms that there is huge variation across town centres, both in terms of the range of issues that are monitored as part of routine health check assessment and the types of measures that are adopted.

5.27 Scottish Planning Policy advises that health checks are an appropriate monitoring tool to measure the strengths and weaknesses of a town centre and to analyse the factors which contribute to its vitality and viability. When used consistently over a period of time, they can demonstrate changes in performance that can inform future decision making. A range of key performance indicators can be used to provide an effective insight into the performance of a centre and so offer a framework for assessing vitality and viability to assist decision makers in identifying new opportunities for improvement. A list of widely-accepted indicators from Scottish Planning Policy ^[4] includes:

• Pedestrian flow (footfall);
• Prime rental values;
• Retailer representation and intentions;
• Commercial yield;
• Vacancy rates;
• Physical structure of the centre;
• Periodic surveys of consumers;
• Crime rates.

5.28 The research team carried out an online survey with Local Authorities across Scotland to explore the availability of town centre regeneration performance data at town, town centre and district centre levels. The survey identified what data is collected, how often it is collected, for what areas and by whom to get a clear picture of activity across Scotland.

5.29 The survey highlighted that monitoring of town centre performance involves a 'patchwork' of local authority departments and community partnership partners:

• Apart from crime and safety, the majority of data is collated by Local Authority ( LA) planning or economic development departments;
• Data is also collated by LA research departments and other unspecified departments/organisations;
• Economic activity, tourism and partnership activity/delivery data is most often collated by LA economic development departments;
• Retail performance, other non-retail uses, accessibility and use and population data is most commonly collated by LA planning departments;
• Some respondents were not sure which organisation or department collates specific data.

5.30There is little consistency across LAs in terms of how often and for where data is collected. Data is most frequently collated on an annual basis, but also often on a two or three yearly cycle depending on resources. In terms of geographical areas data is collated most frequently for defined town/district centre boundaries, but often it is for towns as a whole.

5.31The collation of town centre performance data is not consistent across all towns that make up a Local Authority. Only 28% of LAs are collating performance information for all their town/district centres and 35% collate data only for the town/district centres in which projects are being delivered. Across the broad categories of data specified in the survey some LAs did not collate any information for town/district centres or towns.

5.32 Of the 11 most frequently collated indicators at a town/district level (used by 50% or more LAs) five were retail indicators, four related to economic activity and two were population indicators. These are:

• Number of businesses;
• Retail vacancy levels;
• Land use by type;
• Number of retail businesses;
• Total retail property stock;
• Number of people resident;
• New residential units developed;
• Jobs growth;
• Number of jobs;
• Type of retail businesses, and
• Retail rental levels.

5.33 Just under half (44%) of Local Authorities intended to monitor between 10 and 14 outcome areas in the future, yet 13% intend to monitor four or fewer outcome areas. The most frequently cited indicators that Local Authorities intended to monitor in the future were public realm improvements (88%), followed by retail development (78%).

5.34 In summary there are number of reasons why this inconsistency exists and why monitoring the health of town centres can be difficult:

• There is no one agreed way of carrying out town centre health checks: The monitoring landscape is crowded both as a result of different policy guidance and a wide range of commercial organisations developing alternative approaches to health check assessment;
• Town centres vary in size and role: Therefore the issues that may be important to one town centre may not be applicable to another;
• Town centre boundaries are dynamic and can shift over time: This can present difficulties in developing meaningful comparisons over time; (See Section 4.6)
• Data availability for routine monitoring is often limited: Much of the public data that could be used to track town centre performance ( e.g. employment or population data) is not available and/or not robust for small areas nor available at appropriate time intervals; (See Section 4.2)
• The availability of commercial data usually has cost implications, and is also limited for smaller town centres.

5.35 All of the above means that monitoring of town centre health and performance can often rely on primary research including the use of surveys. This also raises issues around gathering only essential data, particularly in an 'era of austerity' and in generating a consistent approach to town centre health check assessments. Primary research and survey evidence is expensive and has staff resource and skills implications.

Conclusions: Weaknesses in Evaluation Approaches

5.36 The analysis set out above provides a large part of the explanation as to why approaches to monitoring and evaluation of Town Centre Regeneration and the TCRF projects in particular were weak:

• The timescale for the TCRF application process and the eligibility criteria for the fund both mitigated against sufficient time being available for careful planning and for the development of robust monitoring and evaluation frameworks;
• Project activities in some cases were poorly described and often did not specify elements such as the location within, or the target group with whom, the activity was taking place;
• While there were some good individual examples of planning project delivery, the TCRF projects demonstrated some weaknesses in planning for evaluation, particularly around specifying and quantifying outcomes, the relationships between activities and outcomes and the timescale over which outputs and outcomes were expected to materialise;
• These three factors of lack of consistency in approach to monitoring the health of town centres, the lack of specificity around activities, loosely defined outputs and outcomes and weak result chains are all key factors that undermine the potential for effective evaluation of projects;
• At the same time the different scales of town centres, the varying roles they play, the impact of wider social and economic forces and the relationship they have with their economic context makes the development of a single "one size fits all" approach problematic;
• It is important to note that these issues are not an issue solely with the TCRF projects. The wider literature review suggests this is an issue across town centre regeneration as a whole.

5.37 Finally it is also important to note some other wider issues that contribute to difficulties in effective monitoring and evaluation:

• Resources for evaluation are often not a high priority in town centre regeneration. The TCRF focussed - in many respects quite rightly - on the delivery of projects that had the potential to make a change to the performance of town centres at a challenging time. (See Sections 2.12 -2.16) However, without increased resources to support project planning, approaches to monitoring performance and evaluating process and impact it will be difficult to draw robust judgements of the effectiveness of town centre regeneration;
• The consultations and the review of the TCRF projects highlighted skills issues amongst regeneration professionals in respect of project appraisal, development, monitoring and evaluation. These skills are increasingly specialist and require time and experience to develop. The consultations suggested that Town Centre Regeneration practitioners came from a range of backgrounds and drew on generalist skills from a number of areas including planning, retail and economic development;
• Arguably there is gap in terms of demand for these skills. Commissioners and funders of town centre regeneration activity have not applied the exacting standards in terms of approaches to evaluation that might be expected in other policy areas such as health or education.