Connecting Scotland - evaluation: qualitative research - implementation and early impact

Chapter 3: Programme impact

To date, information on the impact of the programme on participants and on applicant organisations has largely been anecdotal. While 82% of survey respondents indicating that they undertook some form of evaluation, our sense from conversations we had with interviewees and the data currently available, is that there is little systematic collation of impact data being gathered currently. While some organisations are undertaking formal evaluation work with participants, many are not.

While further, more in-depth, evaluation will be undertaken later in relation to impact, this evaluation was an important opportunity to begin to gather evidence of impact from participating organisations. We did so through the survey and interviews, in addition to analysing a small number of evaluation reports which organisations submitted alongside their survey response to us.

Responses to our survey indicated that the programme has had a significant impact. Figure 7 below shows the range of outcomes that have been delivered:

Figure 7: Outcomes delivered (n=547)

Impact on Participants	%
They have a device to access the internet	90%
Better able to stay in touch with friends and family	87%
They have a WiFi connection	86%
Improved wellbeing	79%
They have gained IT skills	74%
Improved mental health	74%
They are more connected with local services	71%
They can access advice and guidance on important issues	62%
They are more engaged with their local community	59%
Family members benefitted from Connected Scotland devices	56%
Able to access public services such as the benefits system	55%
Able to access health services	52%
They are better able to engage with their children's education	51%
They can search and apply for jobs	47%
They are more employable	27%
Able to save money on bills	24%
Other	12%
Have made money online (e.g. through selling things)	3%
None of the above	1%

N.B. Multiple responses to this question were allowed.

In the next sections we give further details of the impact the programme has had on participants.

Impact on participants

Increased engagement with family and friends

The most frequently cited outcome of the programme was that by providing people with devices and an internet connection, many were able to connect with family and friends for the first time since the pandemic had begun.

87% of respondents to our survey indicated that they used their devices to engage with family and friends, and among interviewees, organisations most frequently described having given their participants the skills to connect with family and friends using apps such as Facetime or Zoom.Organisations reported the enormous impact this had on people's wellbeing and sense of being connected with the outside world again:

"The joy on people's faces was just unbelievable."

"They thought they had won the lottery."

Some organisations also reported the programme having had an impact on inter-generational interaction – training older people and increasing their knowledge and understanding of the internet made them better able to engage with young people in their household.

A few organisations also noted the impact on carers and peer supporters, whom it helped to have a life outside their caring role.

"It is easy to under-estimate the impact of Connecting Scotland. Our tenants are among the most vulnerable and they hadn't seen anyone up until that point during the pandemic. It meant they were instantly able to Facetime with family and friends."

Improved access to services

The programme was instrumental in giving people access to services which, during the pandemic, would have been otherwise inaccessible to them. In addition, it equipped people with the skills to access services online outwith the pandemic, and in addition to practical services such as online shopping or medical appointments, opened up a whole range of new sources of entertainment, information and education to them.

71% of respondents to our survey indicated that having an internet connection and device had enabled them to engage with local services. Representatives of organisations that were interviewed during the research were equally positive about this outcome.

"One deaf-blind lady we work with can now use the iPad as a white-board to communicate with her carer. She is also able to have contact with her sister and her church. It has had a massive impact."

Other examples of the ways in which people accessed new services are given below:

One organisation that worked with teenage mothers had some of their participants under child protection orders. They were able to join Child Protection Team Meetings by Zoom (or similar) which worked far better than over the phone which is what had been done previously. In addition, it enabled them to work simultaneously with two separated parents which they said was highly valuable.

During the pandemic, one organisation reported that they were able to undertake remote "accompanied home visits" by having one person attend in person, and the other join in via Microsoft Teams.

People who were shielding and had visual impairments had not been able to access talking books during the early stages of the pandemic, but Connecting Scotland enabled them to access these through an app on their device.

A number of organisations spoke of the value of people being able to access church services online, giving them access to their church which would not have been otherwise possible at that time due to lockdown restrictions.

For one council service, it meant that local people could access services which they could not have otherwise during the pandemic.

"It has helped people to connect with real-world groups in their communities."

As a result of the success of the programme, many organisations are delivering more services online now that more of their clients have access to devices, and intend to continue to do so once restrictions related to the pandemic have eased. There has been a positive response to this so far, with many of them reporting ever-increasing numbers taking part in these sessions. They also emphasised the importance of having upskilled people so that they have the skills and systems for accessing services when they become less mobile – for example, enabling them to continue to shop for food and to socialise.

Impact on health and wellbeing

Respondents to the survey of participating organisations indicated that the programme had had a positive impact on health and wellbeing, with 70% reporting improved wellbeing among their service users, 74% reporting improved mental health, and 52% reporting that the programme had enabled service users to access health services. Interviewees also emphasised health benefits of the programme. For example, some interviewees reported that the programme had given people a means to order repeat prescriptions, or to have video consultations with their GP.

Some organisations also reported that having a device gave people better access to health information online.

"Our participants have been accessing self-help information online since they received their devices."

Other examples reported included a person with a brain injury using their device to organise themselves – for example by storing emails and linking them to Siri and their calendar; and in another case one organisation reported a young mother keeping her Chromebook hidden from her partner which then enabled her to contact friends to let them know that she was being abused. When she eventually left her partner, she was able to take the Chromebook with her.

Another organisation piloted virtual annual reviews for people with epilepsy which avoided people who are prone to seizures having to travel. It has increased the number of people attending their annual reviews as a result and has had a positive impact on their health (for example, by enabling earlier prescribing of medication).

Organisations also reported people using their devices for a wide range of leisure purposes, including things like accessing information online about local history, and a photographer learning to upload all of his photos to his device. Organisations emphasised the importance of enabling people to take part in leisure activities in improving their mental health and wellbeing, particularly during the periods of lockdown last year, but also as a lasting legacy for the future.

It did, however, also higlight some challenges – for example, one service found that the programme they delivered online required significant amounts of data to be available. Until unlimited data became available through the programme in the second round, they could not deliver the programme effectively online. This re-emphasises the importance of people having access to high speed broadband and unlimited data to ensure that they can access the full range of services available online.

Access to benefits and welfare

62% of survey respondents indicated that the provision of devices had enabled their service users to access advice and guidance on important issues, and 55% specifically reported that it had given their service users access to public services such as the benefits system.

Among those organisations interviewed during the research, some also reported that providing people with devices has given them easier access to benefits, which must now be applied for online, and that it enabled them to update their Managing Universal Credit journals more easily. Some organisations anticipated that this will impact on the ability of people to maximise their income going forward.

"Some of our homeless clients in temporary accommodation were able to keep on top of their benefit and undertake virtual tours of properties before they took them on. That was a huge benefit."

Impact on education

One of the most significant shifts during the pandemic took place in education, with young people suddenly having to undertake all of their school work online. Many families did not have devices or an internet connection and so were disadvantaged compared to those families that did. Giving families access to a device ensured that children had equal access to online learning and educational materials. Prior to Connecting Scotland, organisations reported that many of the families they were working with either had no device, or were trying to do school work on a mobile phone, which was challenging.

In addition, parents were also upskilled as a result of the programme and were better able to supervise their children's wider online activities.

51% of survey respondents reported that issuing devices to families had resulted in them being better able to engage with their children's education. Interviewees were also positive about the impact on education. For example, one school is hoping to offer SQA accredited training for parents in future.

"It helped to contribute to the vision that every young person deserves equity in relation to digital inclusion and it sped up the process of getting resources."

Access to training and employability support

Survey results and feedback from interviewees indicate that the programme has had a substantive impact on services users' ability to access training or employability support. 47% of survey respondents reported that as a result of the programme their service users were better able to search and apply for jobs, and 27% reported that their service users are more employable as a result of Connecting Scotland.

A number of organisations interviewed reported that the programme had encouraged their clients to access online learning. For example, one client completed a hairdressing course online and other organisations reported that clients had been accessing Open University courses. One project provided a client with a Chromebook which they are now going to use to support their college work.

"Chromebooks were useful for completing college applications which require online accounts and are challenging to complete on mobile phones."

"The role of CLD is taking adult learning out to the community so this programme fitted well with it by offering people the opportunity to take part in a learning opportunity."

One organisation also noted that taking part in the programme provided valuable information about what did and did not work online – their clients were more willing to engage in practical face-to-face learning than online.

Some organisations also gave examples of people using their devices for job searching. This would previously have had to be completed on a mobile phone or they would have had to leave their house to use the internet, making the whole process significantly more challenging.

"We have one person who is now back on track with their life and applying for jobs online. The iPad made a huge difference."

Impact on participating organisations

In addition to having an impact on the individuals who have received devices, Connecting Scotland has had an impact on delivery organisations – in relation to their delivery of online services, staff skills and capacity, and on organisations' wider approach to digital inclusions. We describe these impacts further below. It should be noted that there were no particular trends by sector or type of organisations. The issues outlined related to a wide range of participating organisations.

Increased delivery of online services

Many organisations reported having increased the range of services that they deliver online.

Some organisations had conducted their own research into digital connectivity among their clients prior to Connecting Scotland and so had a good sense of where there was demand and what the benefits of having a more connected group of clients would mean for service delivery. They described Connecting Scotland as having helped to make this happen and having sped up the process.

For others, working online was entirely new, and prompted only by the restrictions arising from the pandemic. Connecting Scotland was a lifeline for these organisations (and their service users) – enabling them to deliver services which would otherwise have had to shut down during the various periods of lockdown.

For example, one organisation has introduced an aftercare programme for its programme participants which is fully online. They would never have been able to do so previously as not enough participants were digitally connected. Another organisation now offers health and wellbeing sessions online facilitated by an instructor, with the aim of progressing participants from this to re-engaging with local services and community life.

Many of the organisations that participated in the evaluation described a permanent shift in service delivery model – moving from services delivered face-to-face, to services being delivered through a more blended approach. COVID has shifted how organisations across the board work, with many now realising that much more can be done digitally.

Enhanced capacity

Some organisations reported that providing their clients with devices, and shifting to a more web-based method of service delivery has impacted positively on their staff's capacity. They described time savings for staff due to reduced travel time, particularly staff in rural areas. They reported that this would enable more efficient use of the staff resources they have.

"There will be a huge impact on time and money savings. We don't want to jump back in to regular ways of engagement. We will take a more blended approach from now on."

In addition, some organisations described the benefits of online work if staff are off sick – with other staff able to step in to appointments remotely now that their clients have access to a device.

Many organisations reported greater attendance at activities that they run. Running activities online has not only benefited people who have been limited from attending in person due to restrictions arising from the pandemic, it has enabled people to take part in activities who live further away and for whom attendance would previously have been impossible due to journey times (i.e. there will be a continued benefit even after pandemic restrictions are lifted completely).

For example, one training organisation reported having had 120 people attend a recent graduation ceremony online which is a far higher number than had ever attended previously. Another organisation runs an arts therapy course and when this shifted online, attendance was higher than previously.

One housing association reported that they now have higher participation rates in consultations about affordability of rent, heating etc. They hope that this will lead to fewer issues with outstanding debts.

In the case of one school, they were able to hold virtual parents' nights, avoiding need for parents to travel to the school and to arrange childcare for their children.

"We can now reach more families in a day."

Upskilling of staff

Some organisations have observed that taking part in the Connecting Scotland programme has improved digital skills amongst staff involved in delivery. This is particularly the case for staff who have been trained as digital champions (although it should be noted that some of these staff already had good digital skills), but in addition some other organisations have observed that the programme has encouraged their wider staff group to improve their digital skills.

One organisation, for example, reported that many more staff are competent using iPads now and that previous resistence to their tablet-based MIS system had decreased as a result of participation in Connecting Scotland.

Increased emphasis on digital inclusion

Some organisations reported that participating in the programme has led to an increased emphasis on digital inclusion within their organisation. 81% of survey respondents indicated that it had helped to advance their organisation's digital inclusion provision, with 18% indicating that it it had not helped.

Some organisations have now appointed a member of staff to provide digital support on an ongoing basis because they have seen the benefits through this programme. Other organisations reported a change in mindset at management level.

"It has really shifted thinking – we were reticent before about what can be done."

"It has sped things up – the Council in our area is now starting to factor in WiFi to sheltered housing complexes and to gypsy traveller sites."

We also heard good examples of organisations already on a journey to providing better WiFi for their clients. For example, one housing association reported that the case for communal WiFi has already been made and that their future approach will be to give tablets to all tenants and provide them with WiFi through their main complex and its surrounding streets.

But for some organisations, progress has not been as significant as they would have liked. One organisation noted the challenge of community work being seen as a "lense to make the housing association's work easier, but not as a solution." There was a sense that this still needed to change and that Connecting Scotland had not impacted on this as much as would have been desirable.

"We don't have the scale to change practice yet. We issued 133 devices and have over 4000 tenants!"

"It has tempered ambition about what we can do online – this is a massive step change for people."

While much progress has been made, opportunities remain to continue to develop digital inclusion practice in organisations.