The Potential of Existing Cross-Sectional and Longitudinal Surveys to Support the Commonwealth Games 2014 Legacy Evaluation
The review focusses on data sources not already in use in the evaluation (eg in the baseline report or on the Assessing Legacy website) that could shed insight into the extent to which Scotland’s games legacy ambitions are met over time, especially in the areas of sports participation; volunteering; cultural engagement and civic pride.
2 INTRODUCTION AND METHODOLOGY
Introduction
2.1 Ipsos MORI was commissioned by the Scottish Government to review potential of existing cross-sectional and longitudinal surveys to inform the Commonwealth Games 2014 legacy evaluation.
2.2 The main objectives of the project were to:
- establish which UK and Scottish surveys can address the research questions set out in the evaluation of the Commonwealth Games 2014 Legacy
- describe and present the data within these surveys which is currently available that will help address research questions on Games legacy
- identify additional future survey questions that would help address the research questions, specify on which survey they would be best placed, and explore the feasibility of getting them included in future survey waves
2.3 This report presents findings related to the first of these objectives.
Methodology
2.4 Existing surveys were identified through several sources. First, literature that highlighted available longitudinal research in Scotland and the UK was reviewed. This literature included:
- 'UK Longitudinal Research and Analysis Network: Inventory of UK Longitudinal Surveys'
- 'Tracking people: A guide to longitudinal social sources'
- 'Use of longitudinal research in the evaluation of the Scottish Government's National Outcomes.'
2.5 Next, data available on the UK data service site was assessed, general internet searches for relevant longitudinal and cross-sectional surveys were conducted and the expertise and knowledge of Ipsos MORI colleagues in our Edinburgh and London offices was sought.
2.6 Data sources identified through this process were then assessed on the following criteria, taken from the Scottish Government's baseline evaluation document Report 1: Questions, Methods and Baseline[21]:
- relevance (e.g. does the data cover the topics upon which legacy is focused?)
- quality (e.g. is the sampling method robust?)
- regularity (e.g. does the data cover the time period between 2008 and 2019?)
- disaggregation (does the survey cover Scotland/Glasgow/east end of Glasgow?)
2.7 For each data source that met these criteria key details were recorded, including the survey sponsor, target audience, sampling strategy, geographical coverage and topic coverage. These details are set out in Chapters 3 and 4 of this document which cover longitudinal and cross-sectional surveys, respectively. Surveys that did not meet the criteria listed were discounted from further consideration (see table 1).
2.8 An assessment was then made of which of the 'in scope' surveys could best address the legacy evaluation questions. As part of this process, consideration was given to the extent to which the surveys could provide information on key equalities group. Tables 1 and 2 below summarises the outcome of this analysis, ranking the surveys in order of applicability to the evaluation questions.
Table 1: Surveys excluded from the data review
| Survey |
Reason for exclusion |
|---|---|
| The Scottish Environmental Attitudes and Behaviours Survey (SEABS) |
SEABS was last conducted 2008 and there are no plans to repeat it. This is because environment-related questions will now be included in the Scottish Household Survey (SHS) instead. |
| Scottish Longitudinal Study |
The information collected in the Scottish Longitudinal Study is made up of administrative data from the Census. Therefore, it contains no relevant questions. |
| The Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS) |
SALSUS currently contains no questions relevant to the Commonwealth Games 2014 legacy evaluation. It previously contained physical activity questions but these have not been included since 2006. |
| Labour Force Survey |
The Labour Force Survey does not contain questions relevant to the Commonwealth Games 2014 legacy evaluation. |
| National Travel Survey |
The only relevant questions this survey contain relate to type of travel to work, which are included in the Scottish Household Survey and are already included in the Scottish Government's baseline evaluation document. |
| Sports Club Survey |
This survey has very low Scottish coverage (less than 5 Scottish sports clubs were included in the last sweep). |
| The West of Scotland Twenty-07 Study |
This was a cohort survey that is now complete. |
| The West of Scotland 11 to 16 Study |
This was a cohort survey that is now complete. |
| Families and Children Survey |
This survey is no longer in existence. |
| Edinburgh Study of Youth Transitions and Crime |
This survey is only conducted in Edinburgh and the future of the funding for the research is unclear. |
| Scottish Household Survey (SHS) |
The SHS has already been evaluated in the document: An Evaluation of the Commonwealth Games 2014 Legacy for Scotland Report 1: Questions, Methods and Baseline. |
| Go Well |
Go Well has already been evaluated in the document: An Evaluation of the Commonwealth Games 2014 Legacy for Scotland Report 1: Questions, Methods and Baseline. |
| Scottish Health Survey (SHeS) |
The SHeS has already been evaluated in the document: An Evaluation of the Commonwealth Games 2014 Legacy for Scotland Report 1: Questions, Methods and Baseline |
| Scottish Crime and Justice Survey (SCJS) |
The SCJS has already been evaluated in the document: An Evaluation of the Commonwealth Games 2014 Legacy for Scotland Report 1: Questions, Methods and Baseline. |
Table 2: Longitudinal survey proformas
| Longitudinal Surveys |
Survey applicability to the Commonwealth Games 2014 legacy evaluation |
|---|---|
| Understanding Society |
High applicability. This survey will be an extremely useful source of information as it contains many relevant questions and is representative at a Scotland level. The sample size is large enough to conduct subgroup analysis among equalities groups. |
| Life Opportunities Survey (part of the Integrated Household Survey) |
Medium applicability. The questions in this survey focus primarily on issues related to disability, and could be used to explore changes among the disabled. The sample size is large enough to conduct subgroup analysis among other equalities groups as well, although pregnancy/maternity is not recorded. |
| Millennium Cohort Study |
Medium applicability. This survey provides a source of longitudinal data on teenagers' attitudes and behaviours. It contains some relevant questions and good contextual information. The sample size is large enough to allow subgroup analysis among some equalities groups, although only parents' religion and ethnicity is recorded, not that of the cohort members. Sexual orientation is not covered. |
| Growing up in Scotland |
Medium applicability, especially in the longer term. Once Birth Cohort 2 is more progressed (in say 5-6 years time) it would be very interesting to compare whether the two cohorts have different patterns of activities. The sample size is large enough to conduct subgroup analysis among equalities groups, although only parents' religion and ethnicity is recorded, not that of the cohort members. Sexual orientation is not covered. |
| 1970 British Cohort Survey |
Limited applicability. The survey covers a single age cohort only. Further, it contains few relevant questions and those were only asked in the 2012 sweep, not in 2008. The sample size is a little small for equalities subgroup analysis and ethnicity, religion and sexual orientation are not covered. |
| 1958 National Child Development Survey |
Limited applicability. The survey covers a single age cohort only. While some relevant questions are included, the Scottish sample size is relatively small, precluding robust national- and subgroup-level analysis. Other longitudinal surveys, listed above, contain similar questions but have larger sample sizes. |
Table 3: Cross-sectional survey proformas
| Cross-sectional surveys |
Survey applicability to the Commonwealth Games 2014 legacy evaluation |
|---|---|
| Glasgow Household Survey (GHS) |
High applicability. This is a quota survey with the data comparing favourably with Scottish Household Survey data. With the exception of the GoWell survey, the GHS is the best available source of data at the Glasgow- and East End-level on key themes of interest. Opportunities to add new questions are also good. The sample size is large enough to allow subgroup analysis among most equalities groups, although pregnancy/maternity is not covered. |
| Glasgow Health and Wellbeing Survey |
Medium applicability. This survey has a good sample size and contains a number of relevant questions. Information is available at a Community Health and Care Partnership level, although the base sizes concerned are small. The sample size is large enough to allow subgroup analysis among some equalities groups but religion, sexuality and pregnancy/maternity are not covered. |
| Glasgow Schools Survey |
Medium applicability. This is a good source of physical activity information for young people in Glasgow. The sample sizes for Glasgow and individual Community Health Partnerships are good. The city-wide sample size is also large enough to allow for subgroup analysis among equalities groups, although religion and pregnancy/maternity are not covered. |
| Health Behaviour in School-Aged Children |
Medium applicability. The Glasgow sample size is small but the survey provides Scotland-wide coverage of physical activity among young people. It would possibly provide a useful comparison with Glasgow Schools' Survey. There is the possibility of adding additional questions in discussion with HBSC Scotland. The sample size is large enough to allow subgroup analysis by age and sex (no other equality characteristics are covered). |
| Ipsos MORI School's Omnibus |
Medium applicability. The survey has good question coverage and there are opportunities to include new questions. The Glasgow sample is small but could be boosted if required. The sample size is large enough to conduct subgroup analysis by age and sex but sexual orientation, pregnancy/maternity, and disability are not covered. Religion was covered in 2008 but not in 2012. |
| Scottish Recreation Survey |
Limited applicability. The survey is very limited in terms of relevant questions but it would be possible to add questions. The sample size is large enough to conduct subgroup analysis by age and sex (no other equality characteristics are covered). |
| Glasgow CPP Neighbourhood Management Survey |
Limited applicability. The sampling methodology for this survey is problematic: the Glasgow Housing Association Tenant Satisfaction Survey sample is taken as a basis and boosted to be representative of all households (4,500 tenants and 2,434 non-tenants). Even with weighting, it seems that this would introduce bias. However, the sample is large enough to allow analysis at the East End-level. It would also allow for analysis by some key equalities groups (including those living in social housing), although sexual orientation, disability and pregnancy/maternity are not covered. |
| European Values Survey |
Low applicability. The sample size in Scotland is too small for analysis and there are few relevant questions. |
| World Values Survey |
Low applicability. The sample size in Scotland is too small for analysis and there are very few relevant questions. |
Contact
Email: Niamh O'Connor
There is a problem
Thanks for your feedback