An Evaluation of the Commonwealth Games 2014 Legacy for Scotland Report 1: Questions, Methods and Baseline

1 Introduction and Background

The Games

1.1 Glasgow will host the 2014 Commonwealth Games (the Glasgow 2014 Games) from 23 July to 3 August 2014. Approximately 6,500 athletes and officials from 71 nations and territories will take part in 17 sports (5 of which include para sport medal events). Scotland, and the City of Glasgow, are expecting many thousands of visitors and spectators. The Glasgow 2014 Games are a major event for Scotland. There is great excitement about the summer of 2014, and a shared commitment to deliver a very successful sporting event.

1.2 This will be the biggest multi-sports event that Scotland has ever hosted. A partnership between the Scottish Government (SG), Glasgow City Council (GCC), Commonwealth Games Scotland (CGS) and the Glasgow 2014 Organising Committee (the OC) continues to underpin the planning and delivery of these Games. The expected cost of delivering the Glasgow 2014 Games is £524m. The SG will provide up to £344m and GCC around £80m; the OC will meet the balance through income from sponsorship, and the sale of ticketing, merchandising and broadcasting rights. Together the Games partners are working to ensure the Glasgow 2014 Games are delivered on time and on budget.

The Legacy

1.3 Crucially, the Glasgow 2014 Games also provide opportunities above and beyond the hosting of a major sporting event. From the early stages of bidding there has been an emphasis on the positive and lasting benefits that could be achieved for both Scotland as a whole, for Glasgow, and the East End specifically. These benefits are often collectively described as the 'legacy'.

1.4 Creating a lasting and positive legacy from the Glasgow 2014 Games is a top priority for the Scottish Government. Legacy activity will help deliver the national government's wider aspirations for Scotland over the next decade. These are set out in the National Performance Framework¹ . The Scotland-wide games legacy plan 'A Games Legacy for Scotland' was launched in September 2009 and sets out the legacy ambitions of the Scottish Government and its wide range of partners. More information on the SG's legacy ambitions is available at: http://legacy2014.co.uk. With a ten year timeframe to 2019, it is set around four themes:

Flourishing - using the Games to contribute to the growth of the Scottish economy.

Active - using the Games to help Scots be more physically active.

Connected - using the Games to strengthen connections at home and internationally through culture and learning.

Sustainable - using the Games to demonstrate environmental responsibility and help communities live more sustainably.

1.5 The City of Glasgow has identified 6 legacy themes which broadly fit with the four national themes above. These are; Prosperous, Active, Inclusive, Accessible, Green and International. More detailed information on Glasgow's legacy plans is available at: http://www.glasgow.gov.uk/en/AboutGlasgow/AGamesLegacyForGlasgow.

1.6 GCC (with the support of SG and partner agencies) is leading on a programme of major capital projects for the venues, the Athletes Village and the transport infrastructure in Glasgow² . Some projects have already been completed and a number of others are underway. Projects already complete include Scotstoun Stadium, Kelvingrove Lawn Bowls Centre, the Commonwealth Arena and Sir Chris Hoy Velodrome. Other projects underway include the Athletes Village, Tollcross International Swimming Centre, Cathkin Braes Mountain Biking Circuit, Glasgow National Hockey Centre, and Scotstoun Leisure Centre. The Hydro Arena is being developed privately as an entirely new venue adjacent to the SECC complex (see map 4 in Annex 6). Venues outwith Glasgow have also been refurbished - in Edinburgh the Royal Commonwealth Pool has now re-opened after refurbishment and in Angus the Barry Buddon Centre will play host to the shooting competition. Many of these were existing venues, or were planned for development in advance of the Commonwealth Games bid. Arguably though, the securing of the Games may have helped ensure their completion to specification and at pace. The activity associated with venue preparation has been used to lever legacy in different ways, in particular business and employment-related outcomes.

1.7 In Glasgow the physical landscape is changing with the building and refurbishment work to support the Games. But there are also plans to affect social outcomes for the city, using the Glasgow 2014 Games as an impetus for raising aspirations, driving achievement and contributing to a positive future for Glasgow. Notably, there has been substantial new investment for the Games in Glasgow's east end, in some of the most deprived communities in Scotland. The Games investment has become integral to plans for a wider, long-term regeneration initiative being led by the Clyde Gateway Urban Regeneration Company (URC). Significant improvements have been made to the transport infrastructure in the surrounding area, including the M74 improvements and the new Clyde Gateway route. The aforementioned Athletes Village will be transformed after the event into around 700 new homes - including around 300 homes for social rent and a new 120 bed care home. Partners in the area are working to ensure that they capitalise on the pace and focus that the Glasgow 2014 Games provide, which sets it apart from other regeneration initiatives to date.

1.8 The Games Partners and a host of other national and local partners are also involved in delivering a wide range of legacy programmes and projects across the 4 national, and 6 Glasgow, themes. At national level, this includes the national government, national agencies, the OC and a number of other partners.³ At Glasgow level, legacy is being led by GCC with support from the Council's arms length external organisations and Glasgow Community Planning Partners. It is hoped that the collective effort and mobilisation of resources across all partners will deliver a games legacy for Glasgow and Scotland. The SG and GCC now have over 2 years of legacy experience behind them. Both have carried out a review of their legacy plans to identify lessons and issues for the years ahead. More details of legacy strategies and programmes by theme are set out in the Annexes. Also provided is information on sources and a glossary containing information on the key organisations mentioned in the report (See Annex 5).

Evaluating Legacy

1.9 The SG, GCC and partners are also committed to assessing carefully whether we are making progress on legacy, and specifically whether progress towards the intermediate and long-term outcomes is achieved. The Games Legacy Evaluation Working Group (GLEWG) was established in January 2012 to design and deliver an evaluation of legacy. The Group is convened by the SG and comprises Glasgow City Council, key national partners and academics. Members generally have a lead responsibility for analysis in their organisations. The membership is set out at the beginning of this report. GLEWG will report to the Games Legacy Executive Board (GLEB). GLEWG has met on three occasions since January 2012 to develop and agree the approach to evaluation that is set out in this report.

This Report

1.10 This report sets out the broad approach to the Glasgow 2014 legacy evaluation agreed by GLEWG, the research questions it will address, and the range of methods that will be deployed between now and 2019. It also sets out the priorities for the next 18 months and a forward timetable for publication of reports.

1.11 The report also has a series of Annexes on the 4 national themes (flourishing, active, connected and sustainable). In each annex we set out:

• A background to each theme;
• The key strategies and interventions at the time of writing;
• The proposed indicator set;
• Early data from the baseline year of 2008 to present (where available).

1.12 It is important to note at this stage, that the indicators presented here are, in theory, amenable to change due to the Glasgow 2014 Games at some juncture between 2008 and 2019, but they only form a small part of the picture. As our work progresses primary research and evaluation will provide the 'bottom-up' evidence that will help assess the direct contribution of the Games. More of these data and analysis will be available for the next report in 2014. More information on methods is provided in Chapter 2.

Previous Research

1.13 We have looked carefully at previous legacy research from major sporting events to help inform and shape our own approach to evaluating the Glasgow 2014 legacy.⁴ This Findings paper is available on the SG website. We reviewed independent and peer-reviewed empirical evidence of past events by searching the relevant databases⁵ using the following search terms: legacy impact, evaluation, Olympics, Commonwealth Games, major event, sporting event. This has unearthed research on a number of Olympic Games, Commonwealth Games and other major events - though events are unevenly represented in the literature. For example, Sydney 2000 and Athens 2004 Summer Olympics are relatively well researched, whereas the Melbourne 2006 and Delhi 2010 Commonwealth Games (as well as many Winter Olympics) appear under-represented.

1.14 Overall, the body of robust evidence on legacy from major sporting events is limited. It varies greatly in quality and rigour. Most studies use a cross-sectional design (often with no repeat or follow-up), very few use a comparison group or area, and some studies have small sample sizes. Further, there is a paucity of long-term assessments of whether legacy has been achieved. The legacy areas covered in the available research also varied greatly. More attention was given to economic and infrastructural impacts whereas social outcomes were only rarely assessed. To some extent the current state of the evidence base probably reflects the challenges of delivering research in what is a complex area with a broad range of proposed aspirations and outcomes, and often with no clear and explicit legacy planning.

1.15 Nonetheless, there are some important messages in the evidence base currently available. What we can ascertain from research to date is that hosting major events are not in themselves a solution to the economic and social problems of cities or regions, and there is much debate about the long term impacts, both social and economic. The evidence on whether there are population health and socioeconomic benefits as a result of hosting such events is also inconclusive.

1.16 For example, previous research has not evidenced a physical activity legacy from hosting. However, this is not to say that it has not, or cannot, happen. Research does suggest that the best way to harness this may be through one of two mechanisms: a 'demonstration effect' that might motivate current or lapsed participants to restart or deepen their participation, and generating a 'festival effect' through a wider celebration that might motivate those currently 'pre-contemplative' or sedentary. Both effects could potentially be generated in Scotland through the hosting of the Glasgow 2014 Games, and by the national and Glasgow cultural programming currently planned.

1.17 Several studies show increases in employment and other economic benefits - though they tend to be short-term and hard to sustain. However, given the economic downturn, short-term employment gains may help partially mitigate the effects of recession. The 2002 Manchester Commonwealth Games' focus on employability and volunteering appears to have led to increases in skills, self esteem and aspirations, as well as a desire to volunteer in the future. A substantial body of research also points to the problems of 'white elephants' and highlights the need for long term plans for the infrastructure. In the past these have arisen due to a number of factors including lack of explicit legacy planning, poor co-ordination between partners and (in some cases) the sheer scale.

1.18 There is some evidence in the literature of strengthening networks and partnerships between different organisations and agencies as a result of hosting a major sporting event. Events have varied greatly in their explicit legacy plans, and the scope and nature of partnership working, with many focusing only on economic and infrastructural benefits. Some events claimed to have social legacy objectives but these were not always operationalised in the articulation of priorities and resource distribution. The London 2012 and Glasgow 2014 Games appear to have more comprehensive and explicit legacy programmes than have been produced for previous Olympics and Commonwealth Games.

1.19 There will be an important addition to the evidence base when the London 2012 meta-evaluation reports later this year, and in 2013. To the best of our knowledge, the London 2012 meta-evaluation is the most ambitious major event evaluation conducted to date in both scale and ambition. This work is very broad in scope, covering all legacy themes, direct and indirect effects, and unintended as well as intended effects. These effects are being measured at host borough, London and national levels. At its heart, this requires the generation of counterfactuals (policy and outcome counterfactuals), and completion of high quality evaluations across large numbers of legacy programmes for later synthesis.

1.20 So, in summary, research to date does suggest that hosting major events can create a legacy - but it won't happen by chance. Legacies are well planned and delivered, and are linked to, and embedded in, existing strategies and policies on place, regeneration, communities and cities. The lessons from this research have informed the SG's legacy review aforementioned in paragraph 1.8.

Developing the Glasgow 2014 Legacy Evaluation

1.21 An evaluation of Glasgow 2014 legacy is potentially a very ambitious and wide-ranging piece of work. It is methodologically challenging because the legacy outcomes are complex. Legacy activity is likely to impact on many people in many different ways, some intended, some unintended. Impacts are likely to be different at different spatial levels - some nationally, some city-wide and some more local to the new infrastructure and investment.

1.22 Nonetheless, there appears to be a clear opportunity to conduct some good quality research and evaluation on the Glasgow 2014 Games and its legacy for a number of reasons. Firstly, there has been a clear and explicit aim to create a legacy from early on among all partners in Scotland. Secondly, legacy plans are often embedded in existing strategies and policies. The literature suggests these are the conditions under which legacy may happen. Thirdly, the international evidence base is currently mixed in quality. This evaluation could add substantially to the knowledge base on delivering major sporting events and planning for legacy, and potentially improve policy making around legacy.

1.23 There were some early decisions to be made on the overall study design. The GLEWG discussed in depth whether a 'counterfactual' or 'control' is possible in such an evaluation of legacy (as has been attempted in the London 2012 meta-evaluation). By 'counterfactual' we mean what would have happened in the absence of the Games investment. We have decided not to follow this approach. As noted earlier, the way in which the work on legacy in Scotland has been designed and delivered is to embed legacy programmes in regeneration, culture and health strategies. This is very much in accord with the evidence on creating legacy, but makes the generation of a 'counterfactual' very challenging.

1.24 This, in turn, makes attributing legacy outcomes to specific elements of Games-related investments and programmes challenging. Therefore, the Glasgow 2014 legacy evaluation has been designed to assess the contribution that different investment and programmes are likely to have made using evidence available from regular and bespoke data collection, research and evaluations. A 'contribution analysis' approach is used in public policy analysis, particularly when it is not feasible or reasonable to design an 'experiment' with a counterfactual or control.⁶ As individual pieces of research and evaluation are specified we expect to set out the logic and expected 'pathways of change' for programmes.

1.25 This decision on the counterfactual also has implications for any economic assessment. The consequence is that it will not be possible to provide an overall measure of the economic impact of the Games legacy. As an alternative, it is proposed that the economic assessment pulls together on a number of sources and evaluation approaches to present a narrative of the economic contribution of the Glasgow 2014 Games over the legacy period on the East End, Glasgow and Scotland as a whole.

1.26 To help develop our thinking further, an 'outcomes map' for legacy has been produced and is presented at the end of this chapter. This is based on discussion with partners and stakeholders. This has helped us frame our key research questions. It sets out the final and intermediate legacy outcomes partners hope to achieve across national themes. More detail on current strategies, interventions and programmes are contained within the themed Annexes. As we draw nearer to the event (and indeed post-event), new programmes will come on stream, and new priorities may emerge. A number of assumptions are made that underpin this model including the delivery of a successful event, and a long-term commitment to partnership working on legacy. Finally, a number of external factors were identified. The most pertinent is that since the bid was won there has been an economic downturn which has changed the context within which the Glasgow 2014 Games is being organised.

1.27 Finally, GLEWG had to consider what an appropriate timescale might be for a legacy evaluation. We decided on 2008 as a baseline year because that was the first full year following the successful bid, and prior to the launch of the Scottish Government and Glasgow City Council legacy plans. Previous research on timing of impacts does suggest that changes in some outcomes (for example international reputation) can be demonstrated in the pre-Games period as a result of winning the bid, not just at the time of the Games, or in their aftermath (when we might expect to see, for example, change in regeneration-related outcomes).⁷ GLEWG also decided that the evaluation should run until at least 2019, replicating the timescale for the legacy plans, and noting that some of the outcomes we are trying to achieve are long term.