Scottish 4G Infill Programme Evaluation

9 Conclusions

9.1 Overview

9.1.1 The primary aim of this research was to evaluate the S4GI Programme against its investment objectives which were established as follows:

1) Reduce the Digital Divide by improving available levels of connectivity for people and communities in areas of deprivation, and for people and businesses in rural areas. This includes:

• o Improve resilience in areas with poorer quality or less reliable wired internet connections.
• o Strengthen communities, including in rural areas, by offering enhanced opportunities to work, trade, and deliver public services.

2) Improve wellbeing of people across Scotland by improving available levels of mobile connectivity.

3) Improve business performance in Scotland by improving available levels of connectivity and improved internet speeds.

4) Enable use of technology, or facilitate changes in behaviour, which support a reduction in Greenhouse Gas emissions in Scotland.

9.1.2 This chapter provides a summary of the findings from the research in the context of the above S4GI investment objectives, with each objective considered in turn.

Reduce the Digital Divide

9.1.3 This objective is focused on reducing the digital divide between i) rural areas and other locations and between ii) deprived areas and other locations by improving connectivity in the former in each case.

9.1.4 As shown in Chapter 2, all of the mast sites are in rural areas and fall into the 10% most deprived datazones in Scotland in terms of access. The programme has therefore effectively targeted these locations.

9.1.5 The results of the survey suggest that S4GI has led to improvements in connectivity for both people and businesses in the areas where it has been implemented. These include:

• improvements in mobile connectivity and reliability inside the home / business premises for those without home / business broadband. While based on a small sample, the data suggests that those without broadband saw improved mobile connectivity and were more satisfied with mobile signal reliability inside their premises as a result of S4GI.
• improvements in mobile connectivity outside the home / business premises with 81% (n=95) of residents with access to a smartphone stating they had 4G connectivity following S4GI compared to just 4% (n=5) prior to the masts being installed and 83% (n=29) of businesses stating that they were aware of improvements in mobile connectivity in their local area compared to 17% (n= 6) who were unaware
• improvements in the reliability of mobile connections outside the home, with average satisfaction with reliability increasing from 2.4 to 7.6 amongst residents and from 1.9 to 8 amongst businesses following the delivery of S4GI

9.1.6 In addition to improvements in mobile connectivity, the data suggests that around 20% of both business and residents made a change to their broadband connection (either upgrading or purchasing a new contract having not previously had one) because of S4GI. In addition to enhanced mobile connectivity, the delivery of S4GI has therefore allowed a proportion of those affected to enhance their home / business broadband connectivity so that speeds and reliability are more in line with more populated locations, resulting in additional benefits.

9.1.7 While on the whole connectivity has improved, it is also important to note that some people / businesses continue to experience difficulties with connectivity, with issues associated with topography and / or weather highlighted in both the open text survey responses and the focus groups / interviews. In addition, participants raised concerns around the need to use the MNO linked to the mast site and the more limited consumer choice and higher costs compared to more populated locations as a result.

Key Point: Overall, the research suggests that S4GI has helped to reduce the digital divide and deliver this objective. However, while on the whole connectivity has improved, some people / businesses continue to experience difficulties in part due to topography and / or weather and it is evident that S4GI had at the time the research was carried out not been able to deliver / facilitate the same level of consumer choice in terms of the number of operators as that of more populated locations with the result that there remains a level of inequality. It is noted though that since the research was carried out, customer choice has improved. At the date of publication, more than half of the 55 masts now offer services from at least 3 mobile operators, and this figure is set to rise even further.

Improve Resilience

9.1.8 This sub-objective focuses on improving resilience in areas with poorer quality or less reliable wired internet connections.

9.1.9 Amongst the businesses surveyed, half (50%, n=18) identified ‘resilience when there is a broadband outage’ as a positive impact of the S4GI Programme, higher than any other aspect. This corroborates some of the findings identified in the S4GI Use Cases^[65] and wider literature review which identified the ability of 4G to act as a back-up when broadband is unavailable as a key benefit for businesses.

9.1.10 The resilience benefits of 4G were also identified by residents and businesses in the open text survey responses and the focus groups / interviews:

• “It's been excellent, … good to have back up if broadband is down.” (Resident)
• “We have had quite regular power cuts with storms and snow, in last few years… 4G has kept us connected even when broadband has gone.” (Resident)
• “Very useful during power cuts when we can't access broadband.” (Resident)
• “It gives me confidence that I am not wholly reliant on the broadband I have which is subject to stormy weather” (Business)

9.1.11 In the focus groups / interviews, attendees also referenced the safety / time saving benefits associated with the enhanced resilience provided by 4G, noting that people can still access services and safety updates when there are e.g. power cuts, landslides or road closures and broadband is unavailable.

9.1.12 While potentially slightly beyond the intended scope of this objective, the research also suggests that the delivery of S4GI has led to communities organising online, bringing improvements in terms of community resilience (broadly understood as the cohesion and strength of a community). This was picked up in both the surveys and the focus groups, with 42% (n=15) and 36% (n=13) of residents respectively stating that the availability of 4G had had positive impacts on ‘finding out about what's going on in the local area’ and ‘feelings of connectedness to the local community’. Similarly, attendees at the focus groups / interviews noted that WhatsApp groups and online message boards had provided opportunities for community organisation / interaction. These aspects are discussed further in the section on Wellbeing below.

Key Point: The research suggests that S4GI has resulted in improved resilience for those who have benefited from the enhanced connectivity provided and therefore the programme has delivered this objective.

Strengthen Communities

9.1.13 This sub-objective focuses on strengthening communities by offering enhanced opportunities to work, trade, and deliver public services.

9.1.14 The results of the survey and focus groups suggest that the delivery of S4GI has provided enhanced opportunities in terms of work for both residents and businesses. While based on a small sample, 25% of residents surveyed (n=9) said they were able to work from home more frequently. There was also some evidence that S4GI had enabled some respondents to set up a new business, take on an additional job, or move to higher paid employment although sample sizes were small and therefore it’s difficult to draw firm conclusions on these aspects.

9.1.15 Comments on 4G enabling people to work from home and / or set up their own business were also made within the open text survey responses and the focus groups:

• “It has enabled me to work from home which is positively life changing.”
• “Literally changed our lives as we work for ambulance service from home (paramedic and technician). We could not leave the house before 4G on duty because we had to have the house phone with us. Now we can be reliably reachable from anywhere on our croft.”
• “It allows me to work from home; It allows me to do my job which I wouldn't have if not working from home.”
• “I wouldn’t be able to send files without 4G connection, or do what I do without 4G, so it has given me confidence to pursue my own business.”

9.1.16 A range of benefits with respect to work were also identified in the business survey. Positive impacts on business activities included:

• being able to access online servers (identified by 42% of businesses surveys as a positive impact of 4G)
• the ability of staff to access online resources when outside of business premises (identified by 42% of businesses surveys as a positive impact)
• improved communication with staff and the ability to undertake video conferencing / Zoom / Teams calls (identified by 36% of businesses surveys as a positive impact)

9.1.17 In addition to employment, the research also suggests that the availability of 4G has led to an increase in people taking up both formal and informal online learning opportunities with:

• 69% of residents surveyed (n=25) indicating that the delivery of 4G had enabled them to access learning opportunities which they previously could not access
• 33% (n=42) indicating that it had led to an increase in the frequency with which they undertaken informal online learning
• 21% (n=27) indicating that it had led to an increase in the frequency with which they undertake formal online learning

9.1.18 In terms of providing enhanced opportunities to trade, while sample sizes were relatively small, there was some evidence from the business survey that the delivery of 4G had had positive impacts in this respect with:

• 30% (n=11) of businesses stating that the availability of 4G had had a positive impact on ‘selling products / services’
• 28% (n=10) stating that it had had a positive impact on business promotion, including via social media
• 25% (n=9) stating that it had had a positive impact on the product or services provided

9.1.19 Somewhat related, findings from the resident survey suggest that the availability of 4G has led to an overall increase in the level of online shopping with over 40% (n=52) of residents surveyed indicating that 4G being available had led them to do more online shopping via non-local shops and a quarter (n=31) stating that it had led them to do more online shopping via local shops. The balance between non-local and local shops is an important one given the potential for the availability of online shopping to lead to a decline in local shops / services. In this respect, the higher proportions selecting non-local shops could be interpreted as a potential negative community impact of 4G. However, it is important to recognise that there may be a number of reasons why online shopping in non-local shops as opposed to local shops was favoured, including there simply being a lack of local shops and / or local shops not (yet) offering online services.

9.1.20 In addition to online shopping, there was strong evidence that the availability of 4G has led to an increase in people using online and mobile banking with over half of residents (n=65) stating that the availability of 4G had led to them undertaking household management tasks online more frequently. In the open text responses to the survey, respondents particularly highlighted that the availability of 4G had allowed them to receive security confirmation texts / verification codes without which they were previously unable to access applications which require this security check. Use of online and mobile banking services was also highlighted as a key benefit in the focus groups with this noted as particularly useful where bank branches are a long drive away. However, while identified as beneficial, it was also recognised that the availability of online banking may have contributed to a reduction in the presence of physical banks and therefore the availability of in-person banking in some rural areas.

9.1.21 While the research did not explicitly cover the delivery of public services, 20% (n=25) of the respondents to the resident survey noted that the availability of 4G had led to an increase in them undertaking GP / hospital appointments via a mobile connection. In terms of providing such services, 4G connectivity may therefore contribute to the viability of such services and potentially lead to cost savings for the NHS. In the focus groups / interviews, one attendee noted that using online health appointments means they no longer have to take time off work to attend appointments and that they are typically seen more quickly than they were with face-to-face appointments. While beyond the scope of this research, where this is the case, there could be a range of other benefits, including people attending health appointments more frequently resulting in earlier diagnosis and reduced treatment costs, and cost savings for businesses as a result of reduced sick days.

9.1.22 In terms of strengthening communities, while the sample sizes were small and therefore it is difficult to draw firm conclusions, the results from the resident survey indicate that the future availability of 4G did influence some people’s decision to move to their current property. The benefits for the long-term sustainability of rural areas were also highlighted in the focus groups, with several participants noting that improved connectivity (and particularly the ability to work from home) had contributed to young people moving back to the area and / or been a factor in people’s decision to stay.

Key Point: Overall, the results suggest that the delivery of S4GI has resulted in positive impacts for the communities benefiting from the programme, particularly with respect to opportunities to work and the delivery of public services. While sample sizes are small, there is also some indication that the availability of 4G has influenced people’s decisions regarding moving to / remaining in the area and therefore that 4G has contributed to a strengthening of the communities affected.

Improve Wellbeing

9.1.23 This objective is focused on improving wellbeing as a result of improved mobile connectivity. To help identify wellbeing impacts, two questions were included in the resident survey as follows:

• a question asking respondents to indicate the impact of the availability of 4G on their wellbeing
• a question which replicated the format of the questions on personal wellbeing included in the UK’s APS^[66] whereby respondents are asked one question covering each of the four measures of personal wellbeing (namely life satisfaction, feeling the things done in life are worthwhile, happiness, and anxiety) - this approach to capturing wellbeing impacts is well established in the literature and has been used in previous evaluations of similar schemes although attributing any change identified to the investment being evaluated can be challenging.

9.1.24 In terms of the former, 39% (n=14) of residents surveyed indicated that the delivery of 4G had had a positive impact on their wellbeing compared to 8% (n=3) who said it had a negative impact. It is unclear as to why the latter felt 4G had had a negative impact with potential reasons including the connection not being adequate (i.e. due to topography) or a potential knock-on reduction in-person contact. The latter is discussed further below.

9.1.25 In terms of the UK APS wellbeing question, the results showed that wellbeing was relatively high in the areas surveyed. However, while there were some changes between the ‘before’ and ‘before AND after combined’ surveys, these differences were marginal, and it is difficult to attribute any changes directly to the S4GI Programme.

9.1.26 Beyond the specific questions on wellbeing, a range of positive impacts of 4G which may have an overall positive impact on wellbeing were identified within the survey. For example, a high proportion of respondents reported positive impacts on:

• keeping in touch with family and friends (56% recording positive benefits, n=20)
• feeling of connectedness to the wider world (53% recording positive benefits, n=19)
• finding out about what's going on in the local area (42% recording positive benefits, n = 15)
• feelings of personal safety / personal safety when out and about (39% recording positive benefits, n=14)
• feelings of connectedness to the local community (36% recording positive benefits, n=13)

9.1.27 These findings were also replicated in the focus groups / interviews with respondents noting that they felt more connected to their friends and family living further away and the wider world more generally as a result of being able to make video calls via FaceTime and WhatsApp, share photos, and access news entertainment such as music, podcasts, and TV.

9.1.28 Participants in the focus groups also highlighted that enhanced connectivity had enabled the local community to organise online. It was noted that new WhatsApp groups and online message boards had been set up to share information; offer support; and set up group activities such as yoga classes, badminton groups, shopping excursions or shared shopping orders. As noted above, as well as potentially improving individual wellbeing such activities could also lead to improvements in community resilience (broadly understood as the cohesion and strength of a community).

9.1.29 There was however a recognition amongst some focus group / interview participants that more online activity has the potential to result in a reduction in face-to-face contact / local services in some areas which could have negative impacts.

• “…because it’s easier to do a superficial sharing of photos, and messages than it is to take the time to have a conversation with someone. I think I’m more distant from relatives because of this form of communication which is popular and accessible, and I do think we are losing something fundamentally real.”

Key Point: Overall, while the results of the specific question on personal wellbeing were inconclusive, the wider research suggests that the availability of 4G has, on the whole, had a positive impact on wellbeing with a range of positive impacts identified. However, a number of disbenefits including a decline in face-to-face contact and local services were also identified. The overall balance of benefits / disbenefits in this respect is likely to differ depending on both the community in question and the individual. For example, with respect to the latter, there may be more disbenefits for those who are digitally excluded due to age / level of ability and / or economic situation. It may also be the case that some disbenefits (such as a decline in local services) take longer to materialise.

Improve Business Performance

9.1.30 This objective is focused on improving business performance by improving available levels of connectivity and improving internet speeds.

9.1.31 As noted above, data from the business survey suggests that the majority of businesses saw an improvement in mobile connectivity and reliability as a result of the delivery of 4G and around 20% also made a change to their broadband connection because of the programme. The research therefore confirms that S4GI has generally led to improvements in connectivity and internet speeds for businesses in the areas affected.

9.1.32 While sample sizes were relatively small and therefore it is difficult to draw firm conclusions, a range of positive benefits in terms of business performance were identified within the business survey. The aspects of business performance for which the highest proportion of businesses recorded positive impacts included:

• ‘customer base’ (33% recording positive benefits, n=12)
• ‘employee productivity / efficiency of working patterns’ (28% recording positive benefits, n=10)
• ‘product / services provided’ (25% recording positive benefits, n=9)

9.1.33 A range of benefits of 4G were also identified in the open text responses to the survey and the focus groups / interviews. These included employee productivity / efficiency gains due to enhanced communication and improved levels of safety:

• “It has made communication more flexible. It has made my business more efficient and safer as I can now communicate while working away from the farmhouse.”
• “Farming business, does make it easier to communicate and is a benefit for safety reasons.”
• “I wouldn’t be able to send files without 4G connection, or do what I do without 4G, so it has given me confidence to pursue my own business.”

9.1.34 Specific benefits for tourism focused businesses were also identified in the open text survey responses and focus groups / interviews, including a widened customer base for tourism accommodation businesses; promotional benefits as a result of being able to promote attractions / the area via social media; and enhanced safety for visitors e.g. hillwalkers etc. While beyond the scope of this research, there is potential for such benefits to result in higher visitor numbers, with consequential impacts on visitor spend and business turnover.

Key Point: Overall, the results of the research suggest that the delivery of 4G has led to improved business performance in the communities affected by the scheme with a range of positive impacts identified including a wider customer base, improved employee productivity / working patterns, and improvements to the product / services provided.

Reduce Greenhouse Gas Emissions

9.1.35 This objective is focused on enabling use of technology or facilitating changes in behaviour which support a reduction in Greenhouse Gas emissions.

9.1.36 In terms of broadband and 4G connectivity, the main mechanism identified for a reduction in carbon emissions is a reduction in travel as a result of home working and / or increased use of online services.

9.1.37 As noted above, while based on a small sample, 25% of residents surveyed (n=9) said they were able to work from home more frequently as a result of 4G being available. There was also evidence of increased online shopping, use of online and mobile banking, and online GP / health appointments as well as online attendance at community council and other community meetings. However, while the above may lead to a reduction in travel and therefore reduced greenhouse gas emissions, it is also the case that home working and the delivery of online shopping / services will themselves generate carbon emissions (e.g. through additional domestic energy consumption, more delivery drivers) which will in part offset the savings generated. In addition, the research identified evidence of increased community activities as a result of the ability to organise online e.g. yoga classes and badminton groups which may contribute to additional travel albeit potentially more locally based.

9.1.38 While slightly outside of the scope of this objective, as set out in Chapter 2, the use of sustainable construction methods was a key aspect of the mast design and build programme. One example was prioritising the rock anchoring of masts instead of using a more standard concrete base. Other examples include using locally sourced stone in the walls of mast compounds and using as-dug material in the construction of access tracks – both in preference to importing non-local materials into the area.

Key Point: Overall, the results suggest that the delivery of 4G has enabled people to use online services and work from home more, thereby suggesting a reduction in resident travel at the local level. Some of these savings will however be offset by carbon emissions generated via other activities with more detailed analysis required to determine the overall impact.