Connecting Scotland: phase 1 evaluation

Executive Summary

This report is an interim evaluation of the Connecting Scotland programme which was set up to provide digital devices and support to people on low incomes who are digitally excluded.

In the primary phase of the programme, devices were issued to people shielding during the COVID-19 outbreak, or who were otherwise vulnerable, to alleviate the effects of social isolation brought about, or exacerbated, by the national lockdown.

Further roll-out of the programme has seen devices go out to other digitally excluded groups, including young care leavers, and low income families with children.

This report evaluates the experiences and outcomes of users in phase 1. Subsequent reports will focus on the succeeding phases.

People receiving devices through Connecting Scotland can use them as they wish, though the anticipated benefits for people of consistent online access include enhanced mental wellbeing, improved financial management, greater opportunities to access learning and training, and enhanced societal engagement.

About Connecting Scotland

Connecting Scotland is funded by the Scottish Government and administered by the Scottish Council for Voluntary Organisations (SCVO). Third sector organisations, or local authorities, apply for devices on behalf of individuals on low incomes, who are digitally excluded. Applicants can apply for Apple ipads or Google chromebooks, as well as 'MiFi' (mobile wifi) devices so that their clients can get a smart device, and a means of connection should they need it. Limits on data use initially applied to users in phase 1 but Connecting Scotland now provides users with free unlimited internet access for 2 years; this includes phase 1 users for whom the extended data offer was retrospectively applied. Users own the devices they receive and can keep and use them for as long as they wish.

End users who receive devices are offered support from a 'digital champion'; a nominated person from the applying organisation who can provide digital skills support and help with using devices. End users can also call the Connecting Scotland helpline if they need help with a specific issue.

Connecting Scotland has delivered on its target of getting 60,000 digitally excluded households online in the first 3 phases, distributing devices via more than 1,000 organisations.

Evaluation approach

Throughout the lifecycle of the programme, data is collected about the characteristics of the people receiving support from Connecting Scotland, as well as their thoughts and opinions about their experience of using devices and going online.

People are invited to complete surveys at the beginning of, and around 9 months into, the period of support. A sample of users is also selected to take part in qualitative interviews in which they are asked about their experience of the programme after they have had their device for a few months. These interviews allow for more in-depth understanding of people's use of their device and the internet, as well as any issues or concerns they may have.

All of these sources of evidence, as well as details from the applications submitted by organisations, are anlaysed to help the Scottish Government understand the needs of various types of users, as well as the elements of Connecting Scotland that are working well, and any areas for improvement as the programme extends into further phases, and reaches more people.

Connecting Scotland is still at a relatively early stage of delivery. Because of the phased roll-out, data collection is staggered. This report focuses on phase 1 users – those shielding, or oterwise vulnerable, during the COVID-19 restrictions - and collates evidence from both the welcome survey and the impact survey (including telephone 'top up'), as well as qualitative interview data. Reports for subsequent stages of the programme will be provided as more data becomes available.

Interim findings

Evaluation of the first phase of the programme shows that Connecting Scotland has increased digital skills and confidence and supported those shielding to cope with the severe social restrictions resulting from COVID-19.

The main findings from this evaluation are:

• Over the life of the programme, Phase 1 users reported increased levels of skill and confidence in using their devices and going online.
• Devices have primarily been used for keeping in touch with family and friends and, to a lesser extent, for entertainment purposes, which has helped people cope with lockdown restrictons.
• Use for other reasons, such as online banking, accessing health, or public services online, was less frequently reported, though several users acknowledged that they were still learning to use their devices and would look to learn further applications in the future.
• Phase 1 users reported improvements in their mental health and said that the device had helped them cope with restrictions resulting from COVID-19.
• Around half of users said they had received support from their digital champion; of these the vast majority were satisfied with this support. 14% said they had never met with a digital champion while a further 15% said they had not needed to. Around a fifth of interviewees said they didn't know about digital champions.
• As well as digital champions, people reported receiving help with their devices from family and friends (37%) or learning for themselves (37%).

Programme learning

The research indicates some areas where there may be scope for changes or improvements as the programme continues to be implemented.

• There was an appetite among phase 1 users for face-to-face learning opportunities and/or formal courses. As COVID-19 restrictions ease, there may be scope to consider how structured, in-person engagement might be supported by Connecting Scotland.
• A minority of users indicated that, for varying reasons, devices issued weren't the most appropriate to their needs; providing a broader range of devices could be a consideration in order to meet diverse needs of users.
• People receiving support from digital champions responded positively to their experience, however, there appear to be differences in levels of awareness of, and willingness to engage with, digital champions. All Connecting Scotland users should have equal awareness of, and access to suitable and relevant digital skills support.
• Users' main concerns were around sharing personal data and entering financial information online. These concerns may present a barrier to people taking advantage of some services available through internet connection.

Analysis and Evaluation

The approach to data collection, analysis and evaluation is evolving throughout the programme. Reflecting on these processes for phase 1 has led us to identify a number of ways in which changes and improvements can be made to evaluating the programme. These include:

enhancing the detail of data collected on end users and improving the degree to which we can make comparisons between surveys. We have recently collated our data sets together onto one platform to enable data linkage and are collecting postcodes in phases 2 and 3 which should allow for finer grained analysis

relatedly, we have identified a need for more statistical support for the analysis of survey results which will allow us to disaggregate data by respondent characteristics and also identify response patterns across different questions

the need to use more than one channel to approach participants so that we include people who will not be in a position to respond to a survey delivered digitally. To this end we have implemented a 'top-up' survey by telephone and are considering further methods, such as postal surveys

considering different ways to engage and communicate with users to encourage participation in research and boost response rates.

the need to consider coherent ways of measuring the impact of the programme beyond surveys and interviews. For example, using the 'Social Return on Investment model' to demonstrate the broader financial benefits of the programme