An Evaluation of Legacy from the Glasgow 2014 Commonwealth Games: Post Games Report

2. Method

2.1 The evidence on legacy from major sporting events varies in quality and rigour. However, it tells us that hosting major events can potentially create legacy, but that it does not happen by chance. Successful legacies are well-planned, well-delivered and embedded in existing strategies, policies and programmes.

2.2 This key message from the evidence on how to successfully deliver legacy has implications for the design of the evaluation. Early decisions made on the evaluation design were set out in our first report. GLEWG concluded that a 'counterfactual' or 'control' was not feasible. By 'counterfactual' we mean what would have happened in the absence of the Games. Many legacy programmes are embedded in existing programmes. This makes the generation of a 'counterfactual' position challenging.

2.3 This, in turn, makes it difficult to attribute legacy outcomes to specific Games-related programmes. Rather, this evaluation has been designed (where relevant) to assess the contribution that programmes are likely to have made to outcomes. 'Contribution analysis'^[8] can be used when it is not feasible to design an 'experiment', and where it is clear that multiple interventions might affect outcomes.

2.4 This decision on the counterfactual has implications for the whole evaluation, in particular for the economic assessment. As previous reports state, the economic assessment does not produce a single figure of economic impact of the event and its legacy as this was not the intention of the study design. Rather, the economic assessment provides a series of data that describe both the short term economic impact of preparing for and hosting the Games, and the emerging evidence on the contribution of the Games to longer term economic outcomes.

2.5 Finally, the evaluation was developed within constrained resources. A clear principle of proportionality underpins the design. We have made best use of existing data where possible and have added questions to existing surveys to help address the key questions. We have only supported new work where there is a clear gap that could not be filled in any other way. This applies to supporting the new community longitudinal study of Glasgow’s East End – GoWell East, and to commissioning a visitor impact study.

2.6 Our evaluation design comprises a number of key components:

• Review of evidence of legacy from major sporting events
• Tracking of over 50 outcome indicators from 2008-2019
• Monitoring and evaluation of national and city legacy programmes
• A new community longitudinal study of the host community - GoWell East
• An economic assessment
• Secondary analysis of key surveys at Scotland and Glasgow levels
• Online and digital media analysis
• Primary research on partnership and collaboration

Evidence Review

2.7 A review of the evidence of legacy of major sporting events was published in conjunction with the Pre Games Report, and updated to accompany this Post Games Report. The review is a critical part of the programme of evaluation and research. It provides an evidence base to judge the extent to which legacy is possible; evidence on how best to achieve legacy, and evidence on factors which make longer term effects more likely. The evidence review influenced the design of legacy programmes at a national and city level, influenced the design of the overall evaluation and continues to influence efforts to embed long term legacy from the Games.

Tracking Outcome Indicators

2.8 A set of over 50 outcome indicators are being tracked from a 2008 baseline to 2019 across all legacy outcome areas. These provide an important canvas or backdrop for the evaluation that helps us understand the wider context of economic and social trends in Scotland and Glasgow over this ten year period.

2.9 The selection of these indicators was based on four criteria: quality, regularity, disaggregation and relevance. The types of data used as indicators include administrative statistical data, survey statistics and international indices which provide a ranking of Scotland or Glasgow in comparison with other countries and cities.

2.10 Data are at Scottish, Glasgow, and East End level, where relevant data are available. By 'East End' we are referring to the area in Glasgow where there has been substantial investment due to the Games. A study area was defined which accords with the GCC East End Local Development Strategy. This is the 'GoWell East' study area that we refer to in this report.

2.11 The data appear on the 'Assessing Legacy 2014' website (www.scotland.gov.scot/AssessingLegacy2014) which is regularly updated. The most recent update was in July 2015 and data from this update is used throughout the report as the latest evidence on the context for legacy at national and city levels.

2.12 The Games were awarded to Glasgow in November 2007. 2008 was chosen by GLEWG as an appropriate baseline year for the evaluation as it is predates the establishment of national and city legacy plans. If data from this baseline year of 2008 are not available for any reason, an explanation is provided on the Assessing Legacy website.

2.13 It is important to note that the indicators vary in nature, scope and, critically, the extent to which change is directly attributable to the Games. Some will clearly be affected by wider forces than the Games (for example, GVA in the set of economic indicators) and so are used for contextual understanding; some more directly capture Games impact (for example, those on civic pride); some directly capture legacy programme impact (for example, number of qualified sports coaches). Any Games, and Games legacy programme, related changes to indicators would be expected at different points in time and/or at different spatial levels. The major summative reports (including this one) use indicator data in conjunction with evidence from programme evaluations and primary research evidence to make an assessment of the likely contribution of the Games and associated programmes to legacy outcomes.

Monitoring and Evaluation of Legacy Programmes

2.14 Fifty-nine national legacy programmes and over 80 Glasgow city legacy programmes were developed before, during and after the Games. Some were completely new, some were existing programmes that have been scaled up or expanded.

2.15 Evaluating every one of the national and city legacy programmes would be disproportionate and unfeasible. However, collectively with partners in GLEWG we have championed evaluations of the larger programmes, particularly where evidence was required to help directly address the six key research questions. In addition, monitoring information is regularly provided across almost all programmes and overviews are published on the Assessing Legacy website referred to earlier.

2.16 External evaluations of a number of the larger national and city legacy programmes have been commissioned by delivery partners– some of which have reported and whose findings are drawn on in this report.

2.17 This report also draws on other primary research conducted by members of GLEWG and partner research organisations such as the Glasgow Centre for Population Health which helps address the research questions. This includes, for example, primary research with Games time volunteers.

Longitudinal Community Study in Glasgow's East End (GoWell East)

2.18 A key evaluation research question asks what effect the Games (and broader regeneration activity) has had on the lives of the community in Glasgow's East End where much of the new investment is directed. This is not something we can readily answer from existing data.

2.19 GoWell is a Glasgow-based research programme that provides evidence on the impact of regeneration. The study generates evidence on how neighbourhood regeneration and housing investment affect individual health and wellbeing; the degree to which places are transformed through regeneration and housing investment; and the effect of regeneration policy on area-based health and inequalities. We have supported a new study area in the GoWell programme in the East End of Glasgow, where much of the Games related investment occurred. We refer to this study throughout the report as 'GoWell East'.

2.20 GoWell East is a longitudinal community study with several components: three survey sweeps of adult householders; a qualitative component; and an ecological analysis of health and deprivation indicators. As far as we know, this is the first longitudinal community study in a Games 'host population', that is the community most impacted by the Games investment.

2.21 GoWell East will allow us to identify changes among a cohort of the adult population living in the East End. These include: changes in physical activity and wellbeing; perceptions of neighbourhood and housing quality and change; experiences and outcomes of employment, learning and volunteering; engagement with legacy programmes; and support for, and engagement with, the Games themselves.

2.22 We have discussed the difficulties of generating a 'counterfactual' in this evaluation. A benefit of building on the GoWell programme is the scope for comparability with the other GoWell Study regeneration areas in Glasgow. This will help us address in the long term the question of any potential 'additionality' of the Games-related regeneration.

2.23 The study area covers six communities in Glasgow's East End - Calton, Camlachie, Gallowgate, Parkhead (in part), Dalmarnock, and Bridgeton, see Figure 2.1. The study area comprises 27 datazones, 21 of which are in the 15% most deprived in Scotland. The area includes the Emirates Arena, the Velodrome and the Athletes Village, with the International Hockey Centre and Tollcross International Swimming Centre on the fringes.

Figure 2.1. GoWell East Study Area

2.24 A total of 1,015 interviews were achieved in Sweep 1. The headline findings from this sweep were published in May 2013 and reported in the evaluation Pre Games Report. Sweep 2 commenced in October 2014, after the Games and when the Games venues had reopened to the public. A total of 430 interviews were achieved in Sweep 2. The headline findings from this sweep were published in June 2015, and are reported in chapter seven below. Sweep 3 is currently scheduled for 2016.

2.25 A Prospective Economic Assessment - an assessment of the likely economic impact of the Games on the East End - was published in August 2014. A Prospective Physical Activity Assessment – a report on understanding the potential for the Games, and wider regeneration activities, to influence physical activity behaviour in the East End was published in January 2015. Finally, a Community Comparison Report, also published January 2015, based on data collected during the Sweep 1 2012 baseline household survey, presents a more detailed look at each of the six communities that comprise the study area. Findings from all these elements of the GoWell East study are included in this report where relevant. All published reports from GoWell East are available here: - http://www.gowellonline.com/goeast

2.26 The GoWell East study also includes a School Study - a questionnaire survey study of physical activity among a cohort of secondary school pupils at six schools in Glasgow, two situated in the East End. Two sweeps of this study are complete, with reports from sweeps 1 and 2 already published.

Economic Assessment

2.27 The SG and GCC established a Games Legacy Economic Evaluation^[9] (GLEE) group to deliver the economic assessment. The remit of this group was to produce a high quality, robust evaluation of the economic legacies from the Glasgow 2014 Commonwealth Games.

2.28 The economic assessment has a number of components. Firstly, the SG Input Output (IO) Framework has been used to estimate GVA and employment impacts from the Games-related capital investment programme and the activities of Glasgow 2014 to deliver the Games.

2.29 Secondly, to examine the economic impact of visitors, a primary study was commissioned. Fieldwork took place during the Games at Games venues (in and outside Glasgow) and at Games-related cultural festival locations (Festival 2014 locations in Glasgow). An achieved sample of over 2,200 was reached, with some early findings already published. The XX Commonwealth Games Visitor Impact Study: Interim Report was published in November 2014 and provided final estimates of the volume of Games and Festival 2014 visitors, an initial estimate of average spend per visit and an initial estimate of total visitor spend.

2.30 Further reports on the characteristics and experience of Games time visitors^[10] and the impact of Games visitors are published alongside this report. The latter takes account of standard considerations made for leakage, deadweight, displacement and substitution effects to make an assessment of whether there was any additionality of visitor impact.

2.31 Finally, we draw on evaluations of legacy programmes, survey data and the GoWell East study to assess emerging initial evidence on whether any longer term economic legacies are likely to be achieved. This includes the potential longer-term legacies for business, the labour market and the tourism and events industry.

Secondary Analysis of Key Surveys

2.32 We have, and will, undertake further analysis of the underlying trends and patterns for some outcome indicators, where they help address the research questions specifically. Some of this analysis appeared in the Pre Games report, and some further analysis appears in this report.

2.33 In the long term, we have negotiated the inclusion of questions on engagement with the Games in the Scottish sample of Understanding Society^[11] (US) in 2015. This will eventually allow analysis of the relationship between Games engagement and taking part in sport in the long term. We have also proposed new questions on physical activity in the birth cohort study Growing Up in Scotland (GUS) to assess whether there is any evidence of a 'Commonwealth Games cohort' of children in Scotland whose physical activity is influenced through engagement with the Games.

An analysis of online media and official digital channels

2.34 An analysis of online media and official digital channels was undertaken. This analysed online media and official digital channels including social media mentions of the Glasgow 2014 Commonwealth Games.

2.35 This provided some evidence for one of our research questions "Has there been a change in civic pride and/or international reputation?". Specifically, this work helped assess the contribution the Games may have made to the reputation of Scotland and Glasgow, both in the UK and internationally. The time period over which data were analysed was between 14 June 2014 until the 6 August 2014.

Partnership research

2.36 Finally, we examine the longer-term impact of the Games experience on partnership working and wider organisational legacy across sectors in Scotland. Evidence to answer this research question is drawn from a number of different sources – lessons learned research conducted by Scottish Government; organisational learning research commissioned by Glasgow City Council and Transfer of Knowledge reports from key partners.