Scottish household survey 2017: consultation responses analysis

2. USE OF THE SHS

The opening questions in the consultation asked about use of the SHS. Three questions asked about the topics used in the SHS, what people use the SHS survey for, and whether there are any alternative sources of data. The findings are reported in this chapter.

Topic use

Question one asked 'What are the main social survey topics you use in the SHS?' 89 out of 99 respondents to the consultation (90 per cent) answered this question ^[3] .

Each topic from the household and random adult section of the survey is used to some degree by all of the sectors. The topics are used most by local government (48 per cent and 45 per cent for the household and random adult sections of the survey respectively) followed by the third sector (28 per cent and 31 per cent respectively). Compared to their share of consultation responses this indicates a heavier use by local government.

Figure 2-1 shows use of different topics in the household survey by sector. It should be noted that Figure 2-1 (and Figure 2.2) is based on 89 respondents rather than all 99 respondents in order to reflect the usage of the SHS topics more accurately.

Amongst the most used topics were household composition and characteristics (66 per cent of those who responded to this question) and health/disability (61 per cent), followed by household income (54 per cent), employment status (47 per cent), and transport - cars, fuel spend, and bicycles (45 per cent). The least used topics (comparatively) were recycling (25 per cent) and the SHCS social survey - energy efficiency and renewables (30 per cent).

Seven respondents (eight per cent) stated they used other topics as well. However, none of them listed any additional topics to those covered above, except a few which noted topics covered in the random adult part of the survey.

Breaking down by sector, distinctive trends appear. Local government showed a higher and more balanced usage of the SHS household topics compared to other sectors. All topics were used by 58 to 85 per cent of the local government respondents to this question, with the pattern of use largely following that in Figure 2-1. The most used topic was household composition and characteristics, and health/disability (both 85 per cent), followed by household income (eight per cent) and employment status (77 per cent).

Figure 2-1: Household survey - use of different topics in the household survey by sector

The pattern of use of topics within the third sector was more varied. The most used topic was household composition and characteristics (64 per cent of all third-sector respondents to this question), followed by health/disability (48 per cent), household income (45 per cent), and employment status (42 per cent). The topics least used by the third sector were recycling (12 per cent), savings and household finances (18 per cent), SHCS (energy efficiency and renewables) and SHCS (heating, repairs & adaptations) (both 21 per cent).

Figure 2-2 shows use of different topics in the random adult survey by sector.

The most used topics were key adult characteristics (61 per cent) and health/disability and caring responsibilities (55 per cent), followed by neighbourhoods and community safety (53 per cent), environment - access to the outdoors, green space, and land use (51 per cent), and volunteering (47 per cent). Among the least used topics (comparatively) were environment (climate change) (28 per cent) and education (35 per cent).

Breaking down by sector, the pattern is similar to that for the household part of the survey. The third sector showed a greater variability in the usage of topics, while use by the local government was higher and a lot less varied.

The topics most used by local government were neighbourhoods (81 per cent of local government respondents that answered this question) followed by culture, sport, and health (all used by 77 per cent).

Figure 2-2: Random adult survey - use of different topics in the random adult survey

The topics used most by the third sector were adult characteristics (61 per cent) followed by environment (outdoors, green space, land use), health, and neighbourhoods (all used by 45 per cent of third sector respondents to this question). The least used topics by the third sector were environment (climate change) and transport (private/public transport, congestion) (both 24 per cent) and transport (travel diary) (27 per cent).

Six respondents (seven per cent) stated they used other topics as well, however, none of these were truly additional topics but rather subsets of topics listed above.

What organisations use the SHS for

Question two asked 'What do you use the SHS for?' In particular how analysis of the data has been used to inform, monitor and evaluate policy and practice decisions, and asked for examples of where it has influenced decision making.

Eighty-nine out of 99 (90 per cent) responded to this open question. Excluding the eight letter responses received, 89 out of 92 (97 per cent) responded to this question.

Respondents from all sectors reported a wide use of SHS data for a variety of purposes. These have been grouped into the following categories and are outlined in more detail below:

• Develop and inform policy and strategies (around one in three respondents)
• Monitor and benchmark performance of strategies, policies and programmes or service delivery (around one in three respondents)
• Planning services and targeting spending, including identifying need (around one in five respondents)
• Equalities analysis
• Research

In order to bring this section to life, a number of diverse examples from each sector have been chosen in order to illustrate how SHS data is used, including where there is information on how the survey has been used for decision making. These are shown in Box 2-1 at the end of this chapter.

Inform policies and strategies

Around one in three respondents specifically mentioned using SHS data to inform policies, development work and strategies. In line with the pattern of responses to the consultation, local authorities and the third sector were the most frequent users of SHS data to create and inform new and existing policies and strategies, alongside other public sector users and the Scottish Government.

Several respondents noted that the SHS was used to inform policy but did not mention any specific uses or areas of use, for example Edinburgh City Council which noted that the SHS is widely used across its departments and Orkney Council which stated they '…view SHS data as critical when developing policies'. Those respondents who did cite areas of use to inform policies and strategies mentioned the following specific areas:

• Sport participation / physical activity (Scottish Government, local authorities, other public sector, and third sector)
• Volunteering (Scottish Government, local authorities, other public sector, and third sector)
• Access to the outdoors and greenspace (Scottish Government, local authorities, other public sector, and third sector)
• Transport - active travel (Transport Scotland, various local authority and third sector strategies)
• Transport - National Concessionary Travel Scheme (Transport Scotland)
• Housing - housing quality, including disrepair, overcrowding, adaptations required (Scottish Government, various local authority policies and strategies)
• Housing - social housing (Scottish Government)
• Fuel poverty and energy efficiency (Scottish Government, other public sector, and local authorities involved in designing and delivering strategies, policies and programmes)
• Child poverty (third sector and local authorities)
• Cultural participation (Scottish Government, other public sector and third sector policies and programmes)
• Building safer communities /neighbourhood improvements/ community engagement (Scottish Government, local authorities)
• Equalities including racial equality and LGBT equality issues (all types of respondents)

A number of responses recognised not only the importance of the SHS in enabling the examination of individual topics to inform policy, but the value that arises from the cross-cutting nature of the survey and time series data. COSLA noted that:

'The SHS enables an understanding of how different policies interplay and affect the lived experience of real people: its overwhelming benefit is in taking a citizen-centred approach to the experience of life in Scotland which allows essential links to be made between policy areas by recognising that policies do not impact on people in isolation.'

Several respondents commented on future use and the need for the SHS to able to gather new data at national and local level on the following: Community Empowerment (Scotland) Act; public service reform (prevention, people, partnership and place); democratic renewal; social care integration.

Monitoring and benchmarking performance of strategies, policies and programme delivery and own or others' organisational performance

Around one in three respondents mentioned that they used SHS data for monitoring and benchmarking. This ranged from formal outcomes frameworks such as the National Performance Framework ( NPF), to organisational performance indicators, to more informal use for monitoring and evaluating the performance of individual strategies, policies or programme delivery.

Formal Outcome and Performance Monitoring Frameworks and Requirements

Asides from the NPF, there are a large number of formal monitoring frameworks, at both national and local level, that the SHS provides a significant amount of data for. These frameworks draw from data across the household and random adult surveys. Table 2-1 below shows the main frameworks, the percentage of indicators that come from SHS data, and the level of data (national, local authority, equalities breakdowns) that are covered in the frameworks themselves or in regular reporting add-ons.

Table 2-1: Formal Outcome and Performance Monitoring Frameworks and Requirements

	Percentage of Indicators from the SHS	Level of Data - National, local authority, equalities	Topics covered
National Performance Framework ( NPF)	Nearly one in five (10 of 55 indicators)	National and equalities	Traffic congestion; smoking; public service quality; public service responsiveness; internet use; neighbourhood rating; cultural engagement; Scotland's outdoors; green space; public or active transport
Local Government Benchmarking Framework ( LGBF)	8 indicators	LA data	Satisfaction with seven diverse local government services: Schools; libraries; parks; museums; leisure facilities; refuse collection; street cleaning. (Social care to be covered by Scottish Health Care and Experience Survey going forward).
Child Poverty Measurement Framework	Over a quarter (10 of the 35 indicators)	National. But demand for local level data from some LAs	Managing financially well; having a bank account; satisfaction with local schools; influence decisions; condition of house; drug misuse; neighbourhoods as a good place to live; satisfaction with public transport; green space; internet use
Housing and Regeneration Outcome Indicators	Half (15 out of 30 full indicators plus feeds into another indicator)	National	Indicators cover topics within 'A well - functioning housing system'; 'High quality, sustainable homes'; 'Homes that meet people's needs'; 'Sustainable communities'.
Active Scotland Outcomes Framework	Over half (10 out of 19 indicators)	National and local authority data Equalities analysis	Participation in sport; access to outdoors & green space; active volunteering; satisfaction with leisure services. Several indicators are Commonwealth Games Legacy indicators which the Scottish Government is committed to tracking until at least 2018.
Single Outcomes Agreements	Varies by LA	LA and sometimes equalities	Various
Partnership Improvement Plans ( PIPs)	Varies by LA	LA	Various

A number of local government representatives noted the use of SHS indicators in the LGBF, including the statutory nature of the satisfaction with services indicators, which the Accounts Commission requires local authorities to complete and report on as part of their duty of delivering Best Value and Public Performance Reporting ( PPR). It was pointed out by COSLA, the LGBF Board, Glasgow City Council and the Accounts Commission that the SHS is the only source which allows consistent comparisons across all 32 local authorities without having to consider any differences in methodology, the time of year of the survey, etc. Therefore, the view was there is no viable alternative data source for these indicators. Glasgow City Council further noted that:

'The SHS, through its role in the LGBF is contributing in a very substantial way to the process of service review and reform in local government and we need to be able to build on the progress made to date.'

It was clear from consultation responses that other frameworks, including the Child Poverty Measurement Framework and the Active Scotland Outcomes Framework, report annually and are used by third sector as well as a range of public sector respondents. Furthermore, it was also noted that the SHS had contributed to other monitoring frameworks such as the Low Carbon Scotland Behaviours Framework.

Strategy, policy and organisational performance monitoring

Monitoring and evaluating strategies and policies

A number of respondents from across different sectors provided general or specific examples of how SHS data has been used to monitor and/or evaluate the impact of other policies and/or strategies, or of important developments in policy areas that required a response from government. These included:

• Private rented sector ( PRS) - monitoring changes in tenure and the growth of the PRS, including of PRS households with children (Scottish Government)
• Digital participation dashboard - an SG dashboard is being produced in order to make more detailed data from headline internet access to confidence in usage more easily available (Scottish Government)
• State of Scotland's Greenspace reports - informing and monitoring impact of policy and practice changes at national and regional levels (third sector)
• Building safer communities - Neighbourhood rating is one of the Building Safer Community Programme's outcome measures for promoting community assets. The Community Safety Unit uses a variety of measures from the neighbourhood and communities topic to monitor the impact of their policies (Scottish Government and local authorities)
• Local/regional transport strategies including active travel (Scottish Government and local authorities)
• Volunteering - to understand the diversity and demographics of volunteering, and whether volunteering strategies are working (third sector and Scottish Government)
• Land use (Scottish Government Land Use Strategy - Community Inclusion in Land Use Decision Making)
• Recycling (Scottish Government and one local authority)

In this context, the Scottish Volunteering Forum noted that local authorities were '…dramatically reducing business intelligence and data gathering exercises - so we tend to rely more on Scottish Government data.'

Organisational Performance Monitoring

It was noted that Creative Scotland used SHS data for a number of their corporate performance indicators and that Historic Scotland were about to follow suit. In a similar vein, a number of public sector and third sector respondents (e.g. Volunteer Centre Edinburgh) noted that they use local authority data to monitor the performance of and/or work with local authorities. Beyond LGBF satisfaction with local services requirements, a number of local authorities themselves noted that they used different SHS indicators to benchmark themselves against other local authorities and national level data.

Planning services and targeting spending

Around one in five respondents noted that they use SHS data to plan services and/or target spending. This often included identifying need in order to target the effective spending of large budgets. Specific examples included:

• Transport Scotland - Travel Diary data used as the main input into regional and local transport models in order to target and justify major infrastructure spend.
• SportScotland Facilities Planning Model - used by Sport Scotland, local authorities and Leisure Trusts in their planning and spending decisions for sports facilities across Scotland. English sport participation data could be used as a substitute but not desirable.
• Housing needs and demand assessments - several local authorities stated that they used SHS data as one of the main evidence sources, another local authority that they used it as a checking tool.
• Adaptations to housing to support independent living - Scottish Government and local authorities encouraged to use SHS data to inform housing needs.
• Energy efficiency and fuel poverty interventions - targeting and allocating spending by need.
• Internet access - identifying housing tenures with the lowest levels of access and targeting funds to improve access.
• Cultural bodies including museums and galleries, and library and information services - local government has a statutory duty to ensure that there are adequate facilities for recreational sporting, cultural and social activities for residents of their areas, and to ensure that there is adequate provision of library services. Bodies used SHS data to target spend, as well as formulate policy and look at the impact of service provision.

Equalities analysis

All sector respondents, but particularly the third sector, highlighted the importance of and the use of SHS data for equalities analysis. This included helping to take forward policies that advance equality, and/or monitoring if and how certain groups (e.g. ethnic minorities, lesbian, gay, bisexual and trans ( LGBT) people) suffer discrimination. Specific areas that used SHS data for equalities analysis spanned culture and heritage, transport, safer communities, sport and physical activity, greenspace and the outdoors, public health, child poverty, housing and energy efficiency, local decision making and volunteering.

The Scottish Government's Communities Analysis Division (equalities analysis unit) noted that the SHS formed 'an important part of the equality evidence base and helps us meet our corporate and legal responsibilities on equality'. This included Equality Impact Assessments ( EQIAs), which are needed for government wide policy development, and understanding how equality groups are progressing within the NPF indicators.

The Equality and Human Rights Commission ( EHRC) also noted the importance of equalities data for their statutory review of equality and human rights, produced every three years and encompassing different aspects of life in Britain and Scotland. At least nine indicators were drawn from SHS data, including housing/accommodation, neighbourhood issues and decision making, and harassment and discrimination.

Research

Nearly all sectors mentioned use of the SHS for research purposes. Again, this spanned a range of different areas and covered research commissioned directly by The Scottish Government, Transport Scotland and other public sector organisations, including:

• Income/poverty/social exclusion
• Transport and climate change, including models for LAs
• Transport - land use and transport planning
• Health and health inequalities, including physical activity in adolescents and children. NHS Health Scotland/ScotPHO cited a wide use of SHS data for different research outputs including data on income inequality, smoking, community safety, financial management/financial inclusion, greenspace, neighbourhood safety and satisfaction, volunteering, etc.
• Rural issues including travel behaviour of individuals across rural and urban areas, differences in wellbeing, access to, and use of, greenspace
• Culture

Other uses of the SHS

A wide range of other uses of the SHS were cited by respondents, not only from the Scottish Government, but particularly from other public sector and third sector respondents. This included:

• Ministerial briefing/First Minister's Questions
• Member of Scottish Parliament briefing
• Communications and marketing work
• Delivery agencies responding to Scottish Government queries
• Campaigning and/or holding government to account
• Data to inform and support third sector funding bids
• As a key source of input data for other major analytical outputs that are used by all the different sectors that responded to this consultation. This included:
  • NRS household projections
  • ScotPHO Online Profiles Tool ( OPT) - public health community profiles (at local authority level or above)
  • To calibrate Home Analytics Scotland

Finally, several respondents noted that they were interested in making greater use of SHS data and asked for more equalities breakdowns particularly for ethnicity, and to combine SHS data with other large scale surveys through the SSCQ in order to deliver improved local authority level and sub local authority level data.

Box 2-1: Uses of the SHS

Scottish Government - internet use - targeting scarce resources at those most in need

Analysis of internet use by tenure allowed the Office of the Chief Economic Adviser and the Digital Participation team to identify a substantial gap in internet access by tenure, enabling the Scottish Government to focus scarce resources on those at greatest disadvantage.

By breaking down the internet access figures a considerable gap between access in the social rented sector and that in the owner occupied and private rented sectors was identified. It was previously assumed that given the poor physical quality of some private rented stock, access to the internet would be at its lowest amongst this sector. In fact, the SHS showed internet access is highest amongst private rented tenants.

Understanding where people were most likely to be offline and being able to target activity on the social rented sector in particular was key to developing innovative and focussed approaches to the provision of affordable broadband, training and support to develop people's skills and confidence to go online. These interventions have so far contributed to a 19 per cent increase in the number of social housing tenants on line. The digital policy team said that 'being able to track and demonstrate improvement at national and local level is a vital part of understanding which interventions are effective.'

Transport Scotland - Travel Diary - informing transport policy and undertaking due diligence on £billion investment decisions

The SHS is the only source of journey specific travel data for the various transport and land-use models developed for Scotland at both a national and a regional level. These models are needed to undertake economic appraisals, and make decisions on, major transport investments. The nature of calibration of the models requires geographical segmentation across the whole of Scotland which makes the sample size a very important component. Models using Travel Diary data have been used to examine and undertake due diligence on a variety of major transport infrastructure schemes, including the M74 Completion, Airdrie to Bathgate Railway, new Queensferry Crossing, Borders Rail, and Aberdeen Western Peripheral Route ( AWPR). Transport Scotland's future work programme includes the A9 Perth to Inverness Dualling, A96 Aberdeen to Inverness Dualling and a number of other major rail improvement projects. SHS based models are currently in the process of informing decisions around the necessary infrastructure that will be associated with Strategic and Local Development Plans around the country as well as various City Deals.

Uses of the SHS - National Records of Scotland household estimates and projections - £ million housing and infrastructure planning and targeting, used by local authorities and the Scottish Government

The National Records of Scotland ( NRS) uses information from SHS questions on household composition, age and gender in order to produce their household projections. A number of consultation responses, including COSLA, noted that these estimates and projections are vital in understanding the housing needs of the Scottish population and building the right kind of homes where they are needed (e.g. single person or family group). The household projections are used at the core of many local authorities' housing plans, and form the basis of the Housing Need and Demand Assessment system developed by the Centre for Housing Market Analysis in the Scottish Government. Household estimates are also incorporated into other statistics used elsewhere in the Scottish Government, including economic statistics. Most recently the household estimates and projections, and wider SHCS physical survey data, are also being used to inform the Scottish Energy Efficiency Programmes for Scotland ( SEEP).

Uses of the SHS - Sport Participation and Active Scotland Framework - Sport Scotland, Scottish Government, Local Authorities and various third sector sport organisations

The SHS is a key resource in monitoring the impact of a range of Active Scotland work streams and measuring Scotland's progress in increasing population activity levels through the Active Scotland Outcomes Framework and associated NPF indicators ('increase physical activity' and 'increase people's use of Scotland's outdoors'). Each of the Active Scotland outcomes has a number of indicators associated with it, with over half of the indicators in the Framework (10 out of 19) coming from the SHS. In order to monitor delivery it was important that the data sources chosen provided national and local level data that was comparable across local authorities, and was of sufficient scale and detail to enable analysis by protected equality groups.

As well as sports participation, the Framework includes SHS data on perceptions of leisure services, greenspace accessibility and active volunteering. Given limited financial and legislative levers, the Active Scotland Framework is seen by the Scottish Government as an important asset for reaching consensus on proposed activities, and for holding government and the organisations that it funds to account. Some local authorities themselves, and associated Sport Trusts, also use SHS data to monitor their own areas' sports participation and population physical activity levels, as well as to develop services and target them to areas of need. The figures are also widely used across Scottish Governing Bodies of sport.

SHS sport participation data is also a key part of SportScotland's Facility Planning Model which allows for different scenarios to be run, thus helping to consider what the effect of changes to sports facilities might be. The model is used to inform decisions SportScotland make as a capital funder of sports facilities, principally through their Sports Facilities Fund. Local authorities and leisure trusts also use the model to inform their strategic planning and spending on sports facilities.

Alternative evidence sources

Question three asked respondents whether there are any alternative sources of evidence available for the topics and/or questions that they use in the SHS. The question was split into two parts. Question 3(i) was a quantitative question where respondents were asked to respond 'yes' or 'no' to the main question. Question 3(ii) then asked respondents to list the alternative sources of evidence for each topic Eighty nine out of 99 respondents (90 per cent) directly answered this question.

Over half of respondents noted that there was no alternative data sources to the SHS, whilst nearly half of respondents reported that there were alternative data sources. However, most of these respondents noted that none of the alternatives fully met their needs in the same way as the SHS.

Respondents were then asked to list the alternative sources of evidence. For this part of the question, respondents who had answered 'Yes' or 'No' to the previous question both provided comments and these are included below.

The eight responses received by letter (not proforma) did not directly answer question 3(i) and 3(ii). However, some of these respondents noted their views on the lack of alternative local authority level data for many topics in the SHS. These views are incorporated below.

'No' alternative sources of evidence

Those who answered 'no' to question 3(i) either informed us that the SHS is the sole source for this data, left the second part of this question blank, or highlighted areas where some of their evidence needs are met by sources other than the SHS.

Topics specifically identified by respondents as only being available in the SHS are listed below ^[4] :

• Volunteering;
• Housing aspirations;
• Discrimination;
• Harassment;
• Community belonging;
• Fuel poverty;
• Household energy efficiency measures and renewables;
• Cultural attendance and participation;
• Internet access;
• Public transport use and passenger perceptions;
• Walking;
• Travel diary; and
• Sexual orientation ^[5] .

Several third sector organisations were particularly concerned about the volunteering data collected through the SHS. For example, the SCVO noted that 'the Scottish Household Survey is the only source of data of this kind on volunteering in Scotland...'

'Yes' alternative sources of evidence

Those who answered 'yes' to the lead in question 3(i), provided more detailed information on alternative sources. Overall, many respondents noted that the alternatives that they listed did not have as large a sample as the SHS. For example, a number of UK wide sources were cited, however, respondents stated that it was hard to get sub-Scotland level data from them. The longevity of the time series data for many topics (e.g. smoking) was also highlighted as a strong point of the SHS.

Another theme that came across in this section, and perhaps more so in the 'looking ahead' section, was the amount of analysis that looked at sub-Scotland level geographies and sub-populations such as equalities. The SHS sample size enabled this kind of analysis to be carried out and comparisons to be made between different areas, whereas respondents found it difficult to identify or use other sources of evidence for this type of work.

Main alternative sources

Of the alternatives suggested, just over a quarter of respondents to question 3(ii) cited Scotland's Census 2011 and around two in ten cited the Scottish Health Survey ( SHeS). A few respondents cited the Labour Force Survey, and the Scottish Crime and Justice survey. The majority of responses highlighting these alternatives were from local government or central Government.

Several respondents did highlight concerns about the accuracy and relevance of census data given that it only happens every ten years, with its greatest level of usefulness being in the immediate years following the Census. Likewise, a number of respondents pointed out the smaller sample size of SHeS meant local authority and health board level data was not available on an annual basis and that currently smoking data from the SHS is the data behind national indicator on smoking.

Locally collected data

Locally collected data such as citizen panels, user surveys or local house condition surveys were also identified as alternatives. This was particularly the case for local government responses where around four in ten responses from this sector mentioned local data collection sources. However, the response from Scottish Borders Council noted that 'there are no consistent alternatives to sources of data provided by the SHS, which provide a solid, valid comparison across Scotland. It would be difficult to replicate this type of comparison if undertaken locally.'

A similar point was also made by a number of other local government respondents, most frequently around satisfaction with services data for the Local Government Benchmarking Framework, but also by COSLA in relation to the Child Poverty Strategy and other statutory areas such as housing and equalities, as well as non-statutory areas such as volunteering.

A number of responses indicated that they obtained data from their own data collections, however, one Scottish Natural Heritage noted that 'it's important to note that the other sources of data used … are complementary rather than alternative sources of data… [Following a review] Our chosen model was to use the SHS for the delivery of headline data … and to commission … Scotland's People and Nature Survey ( SPANS) to provide the additional data needed to interpret and understand the headline trends reported [in] the SHS.'

Alternative sources that use SHS Data

Some respondents identified alternative sources that are actually fully or partially based on data from the SHS. For example, ScotPHO profiles, Scottish Neighbourhood Statistics ( SNS), and household projections (from NRS).