Scotland's recovery from COVID-19: learning

Annex A: Learning from evaluation evidence - thematic analysis of evaluations

This paper is structured around nine cross-cutting themes from a range of sources, with the aim of supporting organisational learning from Scotland's approach to the pandemic. Under each of the headings key findings are presented and policy and practice implications are drawn out, informed by contributions at the workshops.

1. Organisational trust

2. Use of data

3. Digital technology

4. Acting with urgency

5. Collaboration and partnership working

6. Reappraising sectoral contributions

7. Risk and accountability

8. Geography

9. Equality and inclusion

Annex B provides a list of the evaluations reviewed, and primary research and other reports that informed these summaries.

As the work of the group was primarily to inform Covid recovery, within the theme summaries there is a particular focus on findings that can help Scotland progress post-pandemic, and lessons that can be learned from how organisations and communities adapted during COVID-19.

There is a degree of cross over between the different themes, something that was difficult to avoid.

Organisational trust - Evidence from numerous studies illustrates the importance of public trust in effective policy delivery

Key findings

The OECD argues that: Public trust leads to greater compliance with a wide range of policies, such as public health responses, regulations and the tax system.

During the pandemic many voluntary sector organisations were able to quickly mobilise to provide support to vulnerable and marginalised communities. They could do this as they were already 'trusted partners', and due to the strength of pre-existing relationships within communities, social trust and 'on the ground knowledge'.

Building trust takes time. In the words of one participant in a review of the needs of Scotland's migrant and minority ethnic population under COVID-19 ^[8]: "We cannot just ask communities what their thoughts on this or that are when we have an emergency. You cannot expect to build that trust from one day to the next."

The academic literature suggests that trust in government takes time to build but can be quickly diminished. In order to be effective trust-building must take place consistently, regularly and not just at the point of need during crises.

The Social Renewal Advisory Board report ^[2] states that frontline teams were able to build trust more quickly: 'Another lesson from the collective response to COVID-19 has been that teams who have more flexibility and freedom to prevent or solve problems can quickly build trust in communities and deliver impressive results. These frontline teams are often best placed to help individuals, families and communities'.

A review of prison responses to COVID-19 ^[9] found interventions were less effective where there was a lack of pre-existing trust. The study also found a lack of trust in officials can exacerbate existing inequalities in outcomes – this was seen, for example, in lower levels of vaccine take up amongst some minority groups. It was also identified within the review of the Scottish Welfare Fund ^[10] which found that some people may be less likely to apply for a grant where 'they had low trust in the Council – stemming either from past experiences with the Scottish Welfare Fund specifically, or with Council services more widely'.

Several evaluations also highlighted that the reporting requirements associated with funding were loosened during the pandemic serving to create a stronger sense that organisations were being 'trusted'. In the words of one respondent to the Supporting Communities Fund Evaluation: 'I fully appreciate the requirement for effective due-process, evaluation, monitoring and assessment of value for money but there seemed to be a "needs must" minimization of this which felt much more trusting'. ^[21]

Who delivers public health measures is also important. A summary of evidence from Scotland on the COVID-19 mitigation measures aimed at children and young people ^[11] found that: Young people reported higher levels of trust in medical advice when it came from health care professionals (65% of the sample trusted them a lot) compared with medical advice from the Scottish Government and the UK Government (32% and 18% respectively of the sample trusted them a lot).

During the pandemic there was some evidence that more 'person-centred' approaches were effective in building trust with and reaching people who had had little previous contact with services.^[12]

The evaluation of the Asymptomatic COVID-19 testing programme in Scotland ^[13] concluded that involving community leaders and stakeholder organisations in the development and implementation of testing programmes could help build trust, share goals, and bridge cultural and language gaps.

Trust between organisations was also important. An evidence review of the response of the voluntary sector ^[14] concluded that strong relationships and trust between the leaders of community-based organisations and local government officials allowed for the release of funding quickly without the restrictions and limitations of normal grant management processes, enabling a flexible and prompt response to the pandemic.

Similarly the evaluation of the Near Me Video Consulting Service ^[15] demonstrated the importance of trusted relationships in supporting the fast roll out of digital services. In the words of one respondent:

"….To spread something you know works and has been well received in one part of the population, and trust in the people….I think that does make it faster "

Kieran, National stakeholder.

Policy and practice implications

• Building and maintaining trust between different organisations and between public services, voluntary sector organisations and the public they serve will increase Scotland's future resilience. Trust should be seen as central to national resilience and future pandemic preparedness. Workshop participants stressed the role of clear communication about accountability in building and maintaining trust: "build in sustaining trusting and collaborative relationships into accountability. If progress is dependent upon it, make it a key measure of performance."
• It will be important to continue to monitor public levels of trust. Evidence on trust in public organisations is collected in the Scottish Household Survey and is an NPF Indicator. The OECD technical report and Trust Survey 2022 emphasise the need for regular surveys to measure trust in government, as an integral aspect of democratic governance, ideally with running time series to analyse data over time.
• One way in which public trust (in government specifically) could be enhanced is through further promoting transparency via the Scottish Government's work as part of the Open Government Partnership. Openness is one of the five drivers of trust in public institutions identified by the OECD. The other drivers of public trust are responsive and reliable institutions that act in line with the values of fairness and integrity.
• The public sector should continue to invest and learn from person-centred approaches – particularly in the context of working with people who have had limited previous positive contact with services. The Social Renewal Advisory Board report^[2] reports that many councils have been revisiting their values, culture and behaviours and trusting their citizens as they build new relationships around respect and collaboration.
• Workshop participants raised the following as priorities for Organisational Trust: maintaining / reverting back to co-production with local partners and communities during the pandemic; valuing and trusting local knowledge in national forums; giving autonomy to local partners without unnecessary reporting requirements; and increased openness and transparency between cross sector partners. One workshop participant wrote: "Maintain recognition of importance of place in building trust. Local partners have trust in the bank. Continue to build on that trust."
• This is particularly pertinent in the current fiscal context. As a workshop participant remarked: "given resource pressures, we cannot afford to duplicate, building trusting relationship across sectors/organisations - is also 'the work'."
• The Verity House Agreement ^[6] presents a key opportunity to develop organisational trust, and sets out how the Scottish Government and Local Government willwork together, stating: "A positive working relationship should be based on mutual trust and respect, recognising the need for effective and responsible joint leadership as we work on our shared priorities".

For this opportunity to be realised, the agreement will need to be fully implemented and the implications of this for the voluntary sector made clear.

• The Social Renewal Advisory Board report's ^[2] recommendations for new ways of working, recognises that developing collaborative approaches that focus on change will require a high degree of trust and strong relationships, and recommends that "Purposeful cross-sector engagement and exchange opportunities should become a normalised part of career development and induction" and "It is time to trust organisations to do good work without onerous requirements, in a way that delivers for and with communities".
• The Health Foundation report on health inequalities in Scotland ^[16] identified a number of barriers to successful policy delivery and reform that are critical to improving future health. One of these is 'Restoring trust and empowering communities', and it states that 'a lack of trust can exist between institutions involved in delivery – across national government, local government, agencies and the voluntary sector, caused in particular by a lack of empowerment among actors in the system or in engagement between sectors'.

Use of data - During the pandemic new ways of using and sharing data emerged that allowed for more targeted & localised responses

Key findings

The UK National Data Strategy states that by adopting an open and transparent approach to data, the UK government can build greater levels of trust with the general public and deliver more cost-effective, and better targeted and tailored services. ^[17]

During the pandemic there were legislative and cultural changes that allowed for improved sharing of data and intelligence across boundaries, allowing government, voluntary organisations and public services to use and share data to help and protect people. This included a relaxation of GDPR legislation. This meant that different organisations could more freely share data between them, facilitating targeted and person-centred local interventions.

In the words of one interviewee, who took part in a review of the needs of Scotland's migrant and minority ethnic population under COVID-19 ^[8] : '[Before the pandemic] shared documents, things like that would be seen as a total no-no, you know, and then suddenly you just open up: "Okay, let's accept that, it'll be good to work on shared documents." And so once you have this acceptance you can say, "Okay, how can we actually make it GDPR compliant?"

The evaluation of the Near Me Video Consulting Service ^[15] highlighted the way in which staff found new ways of coordinating work and sharing data across organisational boundaries in order to request and access clinical tests. The Shielding Programme evaluation ^[18] also highlighted ways in which data analysis was used to effectively identify and reach people at higher risk of mortality from COVID-19.

Information was shared between different organisations using common platforms. A review of Youth work ^[19] showed how youth workers were able to collaborate and digitally share information (on best practice, lessons learned from new ways of working etc.) with colleagues both locally and nationally.

The Wellbeing Fund evaluation ^[20] highlighted how the Scottish Government, SCVO and funding partners used the same shared, secure cloud-based platform. As one civil servant described it:

"[Everyone] could see a dashboard with the number of applications currently queued, the money going out of the door, the location data… So there was a good sense of everybody working together on the same platform. It was a good example of how you connect the local to the national, and have those organisations working together".

The speed at which issues were resolved was accelerated. For example, the Shielding programme evaluation ^[18] highlighted how data governance issues that would have taken weeks or months to resolve, had been resolved in days.

An evidence review of the response of the third sector ^[14] illustrated how community organisations self-organised using digital tools to match people to volunteering opportunities and shared information on local support offers. The review of the needs of Scotland's migrant and minority ethnic population ^[8] highlighted the development of a shared directory of local and national services providing support for given needs, with co-ordinated contributions by staff from across different local organisations

During the pandemic improvements were seen in the collection and use of particular types of data. For example, Public Health Scotland developed disaggregated data showing vaccination uptake for different ethnic groups. This allowed local organisations to engage with particular communities in order to foster engagement and uptake.

The Covid Recovery Strategy Assurance Report (unpublished, August 2022), highlights the sharing of data and intelligence across organisational barriers as one of several common barriers that are observed both nationally in policy design and local service delivery that impede the effectiveness of delivery. It states: 'As there is an increasing requirement to deliver more of services in collaboration with partners, across Local Authorities, NHS, or organisations from the voluntary or private sector, evidence is clear that the absence of common platform for secure data sharing hinders the legitimate exchange of intelligence and/or information.'

Several evaluations highlighted that there were significant data gaps in our understanding of how interventions were being experienced by and delivered to different groups. This is particularly in terms of protected characteristics, geography and socio-economic status (see Asymptomatic testing evaluation, ^[13] Scottish Welfare Fund Review ^[10], the Supporting Communities Fund evaluation ^[21] and The Social Renewal Advisory Board report ^[2]).

Although there were positive examples of using and sharing data between organisations there was a reluctance to put in place data monitoring to support evaluation recognising the pressures organisations were under (see for example the evaluation of the Business Grant Support Scheme ^[22]).

In retrospect, more could have been done to put in place proportionate monitoring processes to support evaluation. As acknowledged in the asymptomatic testing programme evaluation ^[13] : 'Much of the data was set up for operational and not evaluation purposes'.

Policy and practice implications

• The comparative study of international COVID-19 recovery strategies ^[23] reported that data, data skills and high-performance computing are critical in conferring competitive advantage in recovering from the pandemic.
• Investment in a common Scottish platform for secure data sharing between organisations is required in order to enable the legitimate and timely exchange of intelligence and information necessary to deliver more services in collaboration with partners, across Local Authorities, NHS, the voluntary sector and (where appropriate) business. Standardisation of systems and collective data sharing protocols and approaches could also help eliminate duplication.
• Progress will require strong leadership on Data to agree a joint way forward, embedding citizen rights and agreeing on the creation of a shared approach for Scotland.
• Some workshop participants suggested citizen control of their data, enabling citizens to see what is held about them and empowering them to share: "Develop a whole of public sector approach to data privacy, citizen control of their data, a shared digital eco system and platform of shared digital services (identity, payments etc.)"
• Data collection needs to be rooted in what is important to the citizen, rather than primarily the priorities of the system. The Promise Scotland's collaborative project Doing data differently, is developing a data map to improve transparency about the data Scotland holds on issues which directly and indirectly impact children, young people, and their families. This project emphases joined-up data, that allows people and organisations to see entire journeys and changes over time, and could be a model for others to follow.
• Workshop participants highlighted several key strategic opportunities to improve the use of data: 1. Prioritisation and implementation of the National Information Governance Plan for Health and Social Care; 2. Potential for data sharing – the introduction of legally mandated standards for the safe and effective sharing of information across health and social care, under the powers proposed by the National Care Service.
• Public and voluntary sector organisations should guard against unnecessary risk adversity and actively consider opportunities to share data (in a GDPR compliant and ethical manner) where this can support improved local delivery. Education about the requirements around GDPR could also prevent misinterpretation of GDPR rules and regulations, leading to potentially unnecessary risk adversity.
• The Verity House Agreement ^[6] also presents an opportunity for improvements in the use of data. The agreement not only commits partners to evidence-based policy making (using data to understand issues and to then identify the most appropriate means of tackling them), but also stresses that any required reporting and data collection be proportionate, fit-for-purpose and support transparency, accountability and future decision-making at both a local and national level.
• The Social Renewal Advisory Board report's ^[2] Call to Action 20 states that 'Decision-makers must commit to co-designing the means of gauging progress towards renewal'. One of the essential building blocks of this is the importance of capturing equality data systematically: 'Consistent use of equalities and human rights data to improve capacity to monitor, alongside independent evaluation of how far investment is driving renewal and for whom.'

Digital technology - The challenges of service delivery posed by social distancing measures accelerated the digitilisation of services, but may have widened the digital divide

Key findings

One of the most significant 'silver linings' emerging from the pandemic was the huge leap in the use of digital technology across sectors. The speed and flexibility with which numerous face-to-face services pivoted in order to provide remote support, demonstrates what can be achieved in crisis conditions, often with long term benefits.

An evidence review of the response of the third sector ^[14] discusses the rapid use of digital media across the voluntary sector to replace and/or enhance communications between groups, staff, volunteers, and communities.Often, this switch to virtual working allowed volunteering to continue, enabled beneficiaries who were remote or isolating to be reached, and enabled organisations to engage more volunteers. This was a significant change, especially for voluntary sector organisations in rural and remote areas. The evaluation of the 'Near Me' video consulting service in Scotland during COVID-19 ^[15] is another example, demonstrating a significant increase in the use of video technology in the early stages of the pandemic, especially in hospital and community care services.

Moving services online was often challenging because service providers had to learn to apply digital skills in the workplace very quickly. For many – especially those with little experience of digital service delivery – this was a steep learning curve. There were also some initial issues with infrastructure, connectivity, and access to devices for service users and providers. Implementation issues could be critical to success. The Connecting Residents in Scotland's Care Homes (CRSCH) evaluation ^[24] found that the effective implementation of interventions such as this requires trained staff, and the provision of devices alone is not sufficient to increasing use.

Significant additional Scottish Government funding to support the move towards digitalisation enabled the rapid increase in the use of digital services and technology. Delivered in partnership with the voluntary sector and local government, this funding:

(i) enabled services to purchase digital devices and invest in training so that their workforce could quickly set up and deliver services remotely;

(ii) increased digital access by providing devices to households, enabling them to access services and alleviate social isolation (e.g. through the Connecting Scotland programme, see below). The evaluation of the Supporting Communities Fund ^[21] showed that 40% of the organisations (for which monitoring data was available) focused on improving digital access, and 32% focused on online activities.

Digitalisation enabled the continuation of certain services which would otherwise have stopped resulting in a number of benefits. Evaluation findings demonstrate how the introduction and expansion of digital technology helped service users to: mitigate anxieties due to lack of access to services; increase access to advice, information, services and support; benefits in terms of time, costs and comfort (research on the civil justice system's pandemic response ^[25]); reduced social isolation and loneliness and increased social connectedness and independence. Both during and after the pandemic, digital skills and confidence have improved amongst service users and providers and there is evidence of improved engagement with and coordination between organisations.

However, the move towards digitalisation was not universally welcomed. For example, research on the civil justice system's pandemic response ^[25] explored the effects of remote hearings and other measures introduced or expanded during the COVID-19 pandemic on Scotland's civil justice system. The research found that the adoption and use of remote hearings varied considerably between the different court and tribunal settings researched and there were diverse views on their impact and continued use among different legal practitioners.

Some services were not able to digitalise and were paused, creating backlogs. For others, the quality of digital delivery was less than optimal or diminished by the move to entirely online. The evaluation of Perinatal experiences during the COVID-19 pandemic in Scotland ^[26] showed that for more than half of staff, the virtual delivery of care led to reduced job satisfaction and that opportunities to build rapport had worsened, as had opportunities to assess mental health services.

Some evaluations suggested that digital service provision was not suitable for particular groups. The review of Youth work during the pandemic ^[19] found that while switching to providing online provision helped provide some continuity for young people, for certain cohorts of young people, digital youth work was simply not appropriate. Similarly, the CRSCH evaluation ^[24] found that a digital solution was sometimes not suitable for an individual in a care home.

The pandemic exposed pre-existing inequalities in terms of digital exclusion and marginalization in Scotland. The evaluation of COVID Support in Low Income Households ^[27] found that while only a minority of households were not digitally connected, the effects were significant and included: difficulties in accessing basic information and good quality advice; it being harder to have tailored advice with follow up, and children being unable to access home learning. A lack of digital connectivity primarily affected those lacking proficiency in English, those who reported living in areas of poor connectivity, and those in severe financial hardship (although people entirely digitally excluded were unlikely to be included in the evaluation).

The Connecting Scotland programme provided targeted support to people who were digitally excluded, and was set up to provide digital devices, connectivity and digital skills to people on low incomes who were digitally excluded during the pandemic. The aims of the programme were: to enhance mental wellbeing; improve financial management; provide greater opportunities for learning and training; access to public services; and access to more employment opportunities. The evaluations of Phase 1 ^[28] and Phase 2 ^[29] of the programme, while focused on different groups, both demonstrated the positive impact of increased access to digital services.

In the review of the needs of Scotland's migrant and minority ethnic population during the pandemic ^[8], charities mentioned how SG funding for devices and connectivity packages was key to addressing digital exclusion. However, some groups – including, (but not limited to) some minority ethnic, older and disabled people – remained digitally marginalised or excluded, due to barriers such as affordability or skills. In the words of a participant at a Community Listening Event:

"Digitally excluded people were disadvantaged as so much is being done online. These disadvantages also showed a lot for groups such as people with learning difficulties." – Social Renewal Advisory Board report ^[2]

Similarly, the evaluation of the health impacts of COVID-19 on people who use drugs ^[30] in Scotland showed that for this group, existing inequalities deepened, one of which was a digital divide between participants with regards to healthcare access during the pandemic. Those who did not feel they were coping were those least likely to have access to, or be comfortable with, digital technology.

However, digital services also offered some benefits for some vulnerable groups. For, example research on civil justice system's pandemic response remote hearings ^[25] found that these offered potential benefits for certain groups of vulnerable court users (such as children and young people with additional needs, and parties who had experienced domestic abuse) in terms of allowing easier, more effective participation. It also identified common challenges associated with remote hearings, including: issues arising from technical problems; digital exclusion and literacy (particularly, though not only, among parties); and challenges around communicating, both verbally and non-verbally.

There are several emerging longer term impacts of the rapid move towards the provision of digital services. Pre-pandemic plans for digitalisation were dramatically accelerated, resulting in a more digitally literate workforce and population, better able to respond to future similar events. The review of Youth work during the pandemic ^[19] states that the challenges of the COVID-19 pandemic have left the youth work sector in a much stronger position in terms of responsiveness, able to mobilise quickly and now being more skilled and knowledgeable about digital youth work.

While many services could be delivered digitally during the pandemic, there was public support for a more hybrid model of service delivery. There was some evidence of digital fatigue as Covid restrictions eased. The review of Youth work during the pandemic ^[19] states that the third sector have had to remain adaptable, for example, pivoting back to in person support, in areas where digital fatigue has set in.

Policy and practice implications

• The speed and flexibility with which numerous organisations and services pivoted in order to provide remote support, demonstrates what can be achieved in crisis conditions, often with a legacy of increased adaptability and responsiveness. The Scottish Government and COSLA Health and Social Care Data Strategy ^[31] states: 'The onset of the pandemic demonstrated that the workforce can adapt quickly to digital technology changes and adopt flexible ways of working to deliver services as effectively as possible. We need to build on that momentum and support the ongoing flexibility and cultural change that is required'. The pandemic has also shown us the real benefit that digital inclusion can bring to people and communities (the Social Renewal Advisory Board report ^[2]).
• One of these benefits is the normalisation of flexible and remote working. While there are also downsides to home working, for many people this culture shift has been positive, resulting in less time and expense spent travelling to workplaces (and fewer emissions), and greater flexibility for caring responsibilities. It will be important to mainstream these benefits/ hold on to these progressions and not revert to pre-pandemic ways of working.
• Digitalisation in Scotland has accelerated as a result of the pandemic. This has exacerbated the gap for people and communities who lack digital tools, connectivity and literacy. There is now increased awareness of digital inequity. A legacy of COVID-19 that should also be addressed in future pandemic preparedness is that technology (and organisations having the ability to keep their infrastructure and staff up to date with technology) has a key role to play in future resilience. However in doing so, attention also needs to be paid to how to continue to reach and support those who may not have equitable access, knowledge and abilities.
• In their 2021 report ^[2], the Social Renewal Advisory Board called for the next Scottish Government to set a target to end digital exclusion in the next parliamentary term (and to consider creating a duty on public bodies to enable digital access). It states that parallel opportunities to engage as well as / instead of, digital must be available (during pandemics or major disruptions to services) for people who can't or don't want to use digital, so that they are not left behind.
• To address this, the Auditor General for Scotland and Accounts Commission are carrying out a joint audit on digital exclusion during 2023/24 ^[32], which will look at how well public bodies are tackling digital exclusion, how it affects access to services and the impact this has on people in Scotland, and their human rights. The findings (available in early 2024), will provide recommendations and identify innovative and effective practice in tackling digital exclusion.

Acting with urgency – Public services urgently adapted their practices in response to the COVID-19 pandemic and responded with considerable speed and flexibility

Key findings

The Scottish Government, Local Authorities, health boards and third sector organisations adapted their practices and were able to respond with urgency and pace to the extreme pressures of the pandemic.

The SG responded to the emerging crisis at considerable pace. For example, the Evaluation of the Shielding Programme ^[18] highlighted how 'a comprehensive programme was set up, at speed, despite the logistical challenges involved'.

At key points in the pandemic the Scottish Government and Local Authorities acted quickly to distribute funding. Evaluations of the Wellbeing Fund ^[20] , the Supporting Communities Fund ^[21] and the Small Grants Fund ^[33] highlighted how funding was able to be released quickly through simplifying and streamlining the application process and prioritising assessment.

The Third Sector adapted their practices and responded quickly and flexibly. An evidence review of the response of the third sector ^[14] stated that:'Organisations worked to adapt their practices and sustain delivery of support to some of the poorest and most vulnerable people in society, often far more effectively and quickly than could have been imagined'.

Across sectors, and in particular within Health and Social Care settings, there was a rapid acceleration in the use of digital technologies. This included the expansion of Near Me video consulting ^[15], schemes to digitally connect residents in care homes ^[24] and the expanded Distress Brief Intervention ^[42].

Research on civil justice system's pandemic response remote hearings ^[25] showed that while the Scottish Courts and Tribunal Service's (SCTS) Digital Strategy for 2018-2023 planned for digitalisation, the COVID-19 pandemic dramatically changed the context for the digitisation agenda, both accelerating the introduction of new digital measures, and expansion of existing technology, to enable SCTS to continue delivering court and tribunal services in Scotland while COVID-19 restrictions remained in place.

A number of factors facilitated the range of urgent responses. These included: a reduction in red tape, more flexibility in adjusting service provision, less strict GDPR protocols, more independent decision-making, expanded crisis funding to address urgent needs, less rigorous funding checks, a higher 'risk appetite' and more collaborative working.

There were also certain pre-pandemic conditions which evaluation evidence showed supported a quick response. This included:

• Pre-pandemic reform work. For example the Near Me evaluation ^[15] highlighted the significant ground work and strategic planning that had been done which allowed services to hit the ground running and transform at pace and scale as the pandemic took hold.
• The strength of relationships across sectors. For example the Wellbeing Fund evaluation ^[20] highlighted the importance of historically strong relationships between organisations which could be drawn upon to deliver the scheme.
• Public trust. Third Sector and Public Sector organisations were able to respond more quickly and effectively where there was a pre-existing level of trust.

The rapid response across the public and third sector resulted in some positive outcomes across different settings as highlighted in multiple evaluations. For example the evaluation of the Business Grant Support Scheme ^[22], the evaluation of the Small Grants fund ^[33], the evaluation of the Supporting Communities Fund ^[21], the Near Me Evaluation ^[15] and the Wellbeing Fund Evaluation. ^[20]

However, the speed at which organisations were required to respond to COVID-19 presented some challenges. For example the evaluation of the Supporting Communities fund ^[21] highlighted that working intensively to support communities for sustained periods of time could impact negatively on staff wellbeing.

Implementation was not always smooth, for example the Near me Evaluation ^[15] highlighted some early technical issues. The evaluation of the Small Grants Fund ^[33] highlighted that with retrospect a number of organisations would have liked to have taken a more organised approach to their work and 'jumped in' less quickly. It was also reported within the Wellbeing Fund evaluation ^[20] that some funded projects did not meet the quality standard that would have been applied in non-emergency times'.

There was also a recognition that as a result of the requirement to act at pace, decisions had to be made quickly. Several of the evaluations emphasised the importance of re-visiting and re-appraising decisions as interventions evolved and new evidence emerged (e.g. Shielding Programme Evaluation ^[18])

Evaluations highlighted some longer term changes that could help support quick responses to future crisis's for example the Shielding Evaluation ^[18] noted that 'A time-pressured environment makes it difficult to pause, reflect and critically consider programme assumptions and rationale. Thinking through different scenarios, in advance and in detail, around how at-risk groups could be supported in future pandemic situations, is therefore recommended'. The Small Grants Fund evaluation ^[33] highlighted how several organisations reported having learned from this experience and have subsequently drawn up contingency plans for future similar events so that they can take a more structured approach in future.

A number of organisations highlighted ways in which funding arrangements should be revised to support faster and more flexible responses post pandemic. For example several respondents within the Supporting Communities Fund evaluation ^[21] and the Wellbeing Fund evaluation ^[20] argued that the administration of schemes should be simplified, the level of documentation should be reduced and funders risk appetite should be reviewed and increased.

Finally, as noted in the evaluation of the Connecting Residents in Scotland's Care Homes Programme ^[24], the urgency of the situation resulted in many interventions being developed and implemented far more rapidly than is the norm. There is therefore an acute need to evaluate these programmes to ensure that the lessons and learning generated from its rapid deployment can be shared.

Policy and practice implications

• The Covid Recovery Strategy Assurance Report (unpublished, August 2022), highlighted that throughout engagement with officials in Scottish Government and CPPs, 'there has been repeated positivity for the urgency and empowerment felt during the Covid response, and a desire to continue working in that way to address new and emerging challenges, but that a move back to Business as Usual has been felt, with ways of working tending to revert to 'how things were before'.
• Contingency planning should involve considering the pre-conditions that enable organisations to respond rapidly. A review of prison responses to COVID-19 ^[9] highlighted the need to develop training, emergency plans and build in 'organisational agility' to support staff respond quickly to future emergencies.
• We need to learn from the speed at which some services were delivered during the pandemic. The pandemic response provides examples of how organisations were able to respond at pace. The Social Renewal Advisory Board report^[2] wrote: 'Scotland has shown that real change can happen at scale and pace when we work together with values-based leadership, a passion to deliver positive outcomes, flexible resources and empowered communities and teams. We have all demonstrated a real desire to work differently and not to accept the inevitability of the same old failures. We have shown together that we can renew, reform and reimagine.' However, this involved some trade-offs in relation to risk and accountability.
• Recognising the importance of partnerships and collaboration with other public and third sector organisations is also important so that these relationships are strong in future crises. We also need to recognise what can be achieved quickly when we respond with a common sense of purpose. However, workshop participants also stressed that the pandemic created a clarity of focus and an emergency mindset on a single issue which has now fragmented into numerous competing policy priorities with a lack of consensus on the key priorities. It is not possible to recreate this, but the focus should be on which elements of this type of response could be replicated.

Collaboration and partnership working - The pandemic necessitated and facilitated collaborative partnership working, between and within sectors, enabling an effective crisis response

Key findings

Meaningful collaboration between organisations and across sectors has historically been difficult to achieve. In 2011, the Christie Commission on the Future Delivery of Public Services ^[5] stated that fragmentation and complexity in the design and delivery of public services should be addressed by improving coherence, collaboration and integrated service provision between agencies.

The pandemic led to examples of increased collaboration, between and within sectors.

In some cases, the onset of the pandemic triggered new collaborations, whereas in other cases pre-existing partnerships were developed and strengthened. The Social Renewal Advisory Board report ^[2] noted:

"The immediate response to the COVID-19 crisis has been shaped by remarkable partnerships across communities, volunteers, national and local government, businesses and a wide range of third sector organisations."

Collaborative relationships were reported in evaluations in Scotland including; the evaluation of Telemedicine Early Medical Abortion at Home in Scotland ^[34], and the evaluation of the COVID-19 Shielding Programme ^[18] which stated: 'Stakeholders reported extensive collaboration within and across organisations, across levels of government and across sectors. The way in which people had come together was described as 'magical' or simply 'unusual'.' The evaluation of Near Me video consulting in Scotland ^[15] stated: 'while the pandemic has caused significant disruption to staff and patients, it has also demonstrated an effective capacity for intra- and inter-organisational collaboration'.

A review of Youth work ^[19] illustrated how Local Authority and voluntary sector youth workers worked closely with social work, education, and other agencies, leading to an increased understanding of the value of youth work. An evidence review of the response of the third sector ^[14] shows officials and voluntary organisations worked collaboratively to design new approaches to food provision such as pop-up larders, food pantries and community hubs. A review of prison responses to COVID-19 ^[9] illustrated how Public Health Scotland were effectively involved with prisons in decision-making.

There were numerous benefits of collaborative working between organisations.

In some cases, co-location and joint working strengthened local partnerships at the level of planning and operational delivery. Staff from different sectors worked together, helping to build mutual respect and confidence. Partners were also able to draw on, and pool expertise and resources, enabling services to more effectively and flexibly target needs at the local level, including delivering support for vulnerable people.

In some instances collaborative working was not just beneficial, but central to achieving successful delivery.

The evaluation of the Wellbeing Fund ^[20] found that most of the interviewees involved in the set-up and management of the fund highlighted the importance of strong relationships between national funding partners, SCVO and Scottish Government, as central to the success of the fund in distributing funding quickly to third sector organisations.

Organisations often showed flexibility by expanding or changing their original remit in order to take a more holistic approach.

A review of Youth work ^[19] states: 'During COVID-19, the youth work sector worked in partnership with other agencies and services in a much deeper and more collaborative way. It often went beyond what would traditionally have been seen as youth work... such as tackling food poverty, more intensive mental health and wellbeing, and total family support.'

Increased collaboration was facilitated by a range of factors, including increased use of digital technology, communication and knowledge-sharing between leads and changes to funding arrangements.

Simplified application and reporting requirements, and flexibility in funding schemes allowed for collaborative applications among charities. A review of the needs of Scotland's migrant and minority ethnic population under COVID-19 ^[8] demonstrated how the Govanhill Community Development Trust re-distributed a large grant from the SG, encouraging partnership working by offering more money to collaborative rather than individual applications.

The evidence review of the response of the third sector ^[14] states that collaboration was most successful when local government recognised the voluntary sector as a long-term trusted strategic partner, with a complementary role. Important features of this model are the pooling of resources to sustain locally embedded voluntary organisations and developing place-based collaboration through local networks.

However, greater collaboration was not always evident during the pandemic and evaluations identified a number of barriers to effective collaboration.

An evidence review of the response of the third sector ^[14] shows that while in some areas, voluntary sector organisations quickly formed collaborative relationships, in other areas relationships were fragmentary, diffuse, antagonistic or absent altogether.

Barriers to collaboration during the pandemic included: IT systems which were not fit for purpose, restrictive internal policies, and strict COVID-19 guidelines. The evidence review of the response of the third sector^[14] highlights local area deprivation as a barrier: 'Areas of multiple disadvantage are not only more vulnerable to pandemics but they are also more exposed to the effects of cuts to public funding which, in turn, then reduces the potential for cross-sector collaboration'.

The Connecting Residents in Scotland's Care Homes (CRSCH) evaluation ^[24] found that the fragmented nature of the care home sector meant there were few opportunities for care homes to collaborate and share knowledge. It recommended the creation of a community of practice to support and sustain communication and knowledge exchange around the programme.

In some cases, collaborative partnerships forged during the pandemic have led to lasting changes in attitudes and relationships.

A review of Youth work ^[19] shows that collaborative working during the pandemic has created a shift in the perceptions and reputation of youth work within Local Authorities and partner organisations. Youth work is now seen as an essential and effective service, and recognised as a critical strategic partner within Local Authorities. Despite the ongoing challenges of poverty, youth work groups continue to build strong partnerships with local communities.

However, there remain questions relating the extent to which the collaborative spirit of the pandemic has endured.

An evidence review of the response of the third sector ^[14] suggests that despite the beneficial new collaborations spanning the public and voluntary sectors, voluntary organisations who enjoyed 'a seat at the table' have reported a frequent reluctance among public sector providers to engage in partnerships outside crisis conditions, and charities are sometimes perceived as unwanted competition rather than partners:

"The evidence on whether the spirit of collaboration that emerged during the pandemic would be sustained was inconclusive (and indeed, emerging evidence from live projects seems to indicate that the feelings of being a trusted collaborator of government may well have waned somewhat since the pandemic was at its height)."

Policy and practice implications

• Numerous evaluations show how the pandemic increased collaboration in the design and delivery of public services, sometimes triggering new organisational collaborations, and other times reinforcing pre-existing partnerships. Collaborative working and the forging of new working relationships, have for some, led to reported longer term changes in the status and role of particular partners.
• There are significant benefits to collaboration, and now an opportunity for organisations to learn from and build on the progress made during the pandemic. Linking to Christie ^[5], the Social Renewal Advisory Board report ^[2] states: 'We need to make sure we embed the best partnership and practice that we have seen from people across the full range of public, third and community sectors during the response… We need strong communities and a vibrant third sector, working with national and local government, to deliver the long-term change we are looking to see.'
• However, as a recent report from the Health Foundation on health inequalities highlights, these changes will need to be supported and maintained to avoid an 'implementation gap' between lessons learned and future ways of working. ^[16]
• Several evaluations made recommendations about the importance of investing in collaborative working (e.g. evaluation of the Near Me video consulting service in Scotland ^[15] and evaluation of the COVID-19 Shielding Programme ^[15]). For example, the Supporting Communities evaluation ^[21] argued that further funding will be required to continue to effectively develop the relationships and joint working arrangements they had established during the pandemic.
• A report on the ongoing impacts of COVID-19 in Scotland's rural and island communities ^[35] notes that rural and island grassroots organisations became accustomed to making decisions for themselves during the pandemic, and recommends building on existing/new partnerships and supporting community anchor organisations, stating that these organisations needed more government support - 'This could be more readily achieved by bringing together policies from different sectors to address the needs of local communities (there is currently too much silo-ing).'
• A culture shift may be required that encourages collaboration and co-ordinated approaches that bring organisations together to address specific needs. Flexibility of funding was a key issue raised by workshop participants - changing funding practices and principles to encourage and incentivise collaboration and pooling of resources as a way of delivering outcomes, could help facilitate this. There is also evidence that the co-location of services provides numerous benefits and encourages a shift from a competitive to a more cooperative environment.
• The Verity House Agreement ^[6]presents a key strategic opportunity, setting out a vision for a more collaborative approach to delivering shared priorities for the people of Scotland, including new arrangements around powers and funding for Local Government. This includes a default position of no ring-fencing or direction of funding. Workshop participants were hopeful about the impact of this new deal, but also concerned that it is implemented effectively.
• The Scottish Council for Voluntary Organisations encourage the implementation of their paper a SCVO%20defines%20Fair%20Funding%20as%20a%20long-term,%20flexible,,with%20inflation-based%20uplifts,%20and%20proportionate,%20transparent%20approaches%20">Programme for Government proposal: Delivering Fair Funding by 2026. This makes the case for 'long-term, flexible, sustainable, and accessible approach to funding essential to a sustainable sector which can offer Fair Work, support volunteers, and deliver quality outcomes for the people and communities the sector works with'.
• The evidence review of the response of the third sector ^[14] stresses the need for trust and formal recognition of the important role of the third sector as collaborative partners, not only in situations of profound uncertainty and crisis, but as everyday "key workers" providing vital public services.
• A review of the needs of Scotland's migrant and minority ethnic population ^[8] suggests that local government and voluntary organisations should establish (adequately funded) needs-based partnerships within and between local government and other public and voluntary organisations to holistically address given needs and issues arising among migrant and minority ethnic populations.
• The Scottish Leaders Forum (SLF) may have an important role to play in developing a more pan-public service approach to leadership practice and offer opportunities to raise the profile and value of collaborative leadership for outcomes.

The SLF report on 'Leadership, Collective Responsibility and Delivering the National Outcomes' (produced by the Accountability and Incentives SLF action group) states that "… effective delivery of the NPF outcomes requires cross-organisational collaboration and coordination across organisational boundaries."

Reappraising sectoral contributions - The COVID-19 pandemic resulted in a re-assessment of the relative importance of the contribution of different sectors

Key findings

Within Scotland there is evidence to suggest that COVID-19 led to a re-appraisal of the value accorded to organisations in the public and voluntary sectors ^[36].

COVID-19 led people to re-appraise and accord greater value to the role played by the Voluntary Sector.

Across numerous evaluations, there is recognition of how the pandemic led to areappraisal of the value of the role played by organisations in this sector, who were essential to the COVID-19 pandemic response. During the pandemic there was a shift in power dynamics, with voluntary sector organisations being afforded greater power and responsibility.

A review of Innovation and Creativity in the Third Sector in response to COVID-19 ^[14] found that the pandemic led to a 'greater recognition of the role of third sector organisations and volunteering in local and national emergency responses'.

During the pandemic the voluntary sector took on new and expanded roles. A expert review examining the experiences of migrant and minority ethnic populations under COVID-19 ^[8] highlighted the way in which voluntary organisations provided public health outreach through providing interpreters and translating information to help migrant and minority ethnic communities understand public health guidance.

Similarly, a report examining Youth work's role during and in recovery from COVID-19 found that Local Authority and voluntary sector youth workers worked closely with social work, education, and other agencies and this has led to an increased understanding of the value of youth work. ^[19]

This closer collaboration led to voluntary sector organisations reporting that they felt 'part of the team' and an equal partner (Bynner et al., 2022 as cited in ^[14]).

However, as restrictions eased it appears that these new ways of working were not embedded and despite a strong desire 'to cement' partnership working and retain the enhanced role and voice of the third sector, there is limited evidence that this new relationship has been sustained (Thiery et al., 2021 as cited in ^[14]).

COVID-19 led to stronger recognition and appreciation of the value of local knowledge and understanding and the importance of this in directing resources to those most in need.

In the pandemic's early stages, some local statutory services were suspended. Aware of the unmet need within their local communities people mobilised, based on local knowledge, to support the needs of those requiring help such as older and medically vulnerable people.

At a local level, community-based voluntary organisations had a key role in coordinating local efforts, linking their work with social action by citizens and addressing disadvantage (den Broeder et al., 2022; Fransen et al., 2022 as cited in ^[14]). They became hubs, rapidly repurposing their activities, coordinating volunteers and food supplies, and acted as 'cogs of connection' (den Broeder et al., 2022; Locality, 2020 as cited in ^[14]).

The review of Innovation and Creativity in the Third Sector in response to COVID-19 ^[14], outlined how as the pandemic progressed self-organising groups of local volunteers, often referring to themselves as 'mutual aid groups,' were set up very quickly and in order to respond to hyper-local needs.

As the pandemic moved beyond the initial 'lockdown', mutual aid groups adapted from being primary service providers (supplementary to public sector activities) to a complementary role, addressing needs not covered by public services such as delivering small quantities of food, replacing light bulbs, taking rubbish bins out, and providing a source of local information (Dayson & Damm, 2020; Rendall et al., 2022, as cited in ^[14]).

The evaluation of Scotland's Shielding Programme ^[18] highlighted the importance of local knowledge on the part of Local Authorities in supporting the delivery of the programme within particular locations such as Island communities.

The Social Renewal Advisory Board report ^[2] illustrated how communities responded to the pandemic with courage, hard graft, kindness and togetherness and calls for 'the balance of power to be further shifted, so individuals and communities have more control over decisions that affect their lives'.

COVID-19 led to the Scottish Government placing a stronger policy priority on supporting particular areas of the economy, such as the green economy, as Scotland emerged from COVID-19.

A review of international pandemic recovery strategies ^[23] found that within Scotland and many other comparator countries' new sectors of the economy are being prioritised. This is perhaps most apparent in relation to the energy sector where efforts are being made to create new green jobs and help workers to transition out of high carbon sectors.

Scotland's Covid Recovery Strategy, published in October 2021 placed a renewed emphasis on good, green jobs and fair work as one of the three outcomes central to Covid Recovery.

Policy and practice implications

• Third sector organisations and volunteering were essential to the COVID-19 pandemic response. It will be important to recognise the role and value of the voluntary sector in all existing and future emergency and resilience arrangements.
• COVID-19 evaluations highlighted the importance of these organisations and their role in the community. They were uniquely situated to identify need and respond quickly. As a result, funding and investment in local organisations is likely to be important in supporting recovery and renewal in communities.
• The review of third sector responses to the pandemic ^[14] concludes that in Scotland, there is a clear opportunity to significantly reset the relationship between the sector – particularly those organisations operating at a vital, hyper-local level – and different layers of government. This requires moving well beyond the minimum statutory obligation that exists to involve (some parts of) the third sector in certain circumstances: in community planning (for example).
• The Social Renewal Advisory Board report ^[2] states: 'It is soul destroying to constantly have to restate the needs of the third sector when it has proved so fundamental to sustaining people and places throughout this crisis. It is time to trust organisations to do good work without onerous requirements, in a way that delivers for and with communities.'
• A central message from this report is the need to do more to share power more equally – 'giving more power to people and communities, empowering frontline teams, and building new ways of working, based on what has worked well during the pandemic and developing new arrangements for local governance. Changes needed are not all about funding: they are about leadership, culture, values, a commitment to place-based working, and an enduring commitment to change.'
• The Verity House Agreement ^[6] has the potential to contribute towards the reappraisal of sectoral contributions. However, given the fundamental role of the voluntary sector during the pandemic, workshop participants questioned what the broader implications of the Verity House Agreement were for the voluntary sector.
• The review of international pandemic recovery strategies ^[23] shows that many countries share Scotland's aspiration to create new good, green jobs as part of their recovery from COVID-19. Over coming months and years there is likely to be intense competition to create and attract these jobs. As Scotland transitions towards Net Zero it will need to carefully consider where its unique advantages lie.

Risk and accountability - COVID-19 resulted in a re-assessment of risk and accountability, this resulted in organisations being given greater autonomy to take risks

Key findings

During the pandemic, Local Authorities, community partnerships and voluntary organisations were given greater autonomy to take risks to implement solutions to support individuals and communities. The removal or reduction of 'normal' bureaucratic processes, and the simplification of funding applications and reporting requirements, enabled organisations to adapt and quickly coordinate their services.

A review of the needs of Scotland's migrant and minority ethnic population under COVID-19 ^[8] highlighted: the reduction in 'red tape', greater flexibility in adjusting service provision, less strict GDPR protocols, and more independent decision-making as key reasons why third sector organisations were able to rapidly respond to the crisis.

A review of Youth work ^[8] showed how youth workers were able to respond flexibly and at speed because of less stringent risk assessment procedures. Youth workers moved quickly to establish digital services, being "nimble and adaptable". This was a steep learning curve for many youth workers who had little/no digital youth work experience. They were aware of safeguarding issues with digital youth work, but planning and risk assessment were done on an evolving, iterative basis, which may not have been possible in normal times.

An evidence review of the response of the third sector ^[14] showed how organisations were permitted to make decisions based on local knowledge about where funds could best be spent, smaller organisations/ community-based groups were able to access appropriate levels of funding to meet their needs, due to greater flexibility on funding amounts. There was also less competition for funding and third sector organisations were able to release funding quickly without the restrictive limitations of normal grant management processes.

The evaluation of the Shielding Programme ^[18] highlighted the importance of taking a more person-centred approach that included taking into account the ways in which people assess and manage risk. Shielding people reported that they were used to undertaking risk assessments, negotiating complex situations and being involved in decision-making about their care. They felt that recognition of their experience had been lacking in the shielding programme, especially early on. Over time the evaluation found the programme evolved to place a stronger emphasis on enabling people to make an informed choice to shield as much as was optimal for them.

New skills were gained and retained and there were shifts in practice, as a result of less risk averse policies:A review of Youth work ^[19] shows how post-lockdown youth work is in a much stronger position digitally and now has the systems required to work with young people online.

However, whilst acknowledging that overall the general sense amongst interviewees was that the 'right balance' was struck, the Wellbeing Fund evaluation ^[20] did highlight how as a result of a relatively high risk appetite some projects were funded that did not meet the quality standards that would have been applied in non-emergency times.

The pandemic led some services to re-appraise the relative advantages of some practices. For example, the evaluation of Near Me video consulting ^[15] stated 'there was broad consensus among interviewees that the pandemic has meant a long-term shift regarding the role and risk-benefit balance for video consulting in the 'new normal'.

The Near Me evaluation ^[15] also outlined how 'The crisis saw some relaxing of governance and regulatory requirements. This came as a relief for many, highlighting the importance of creating organisational cultures that are conducive for experimentation and risk-taking. However, the evaluation recognised that whilst regulatory structures are burdensome, quality control serves an important purpose and that in sustaining and building on recent developments, it will be important to find the right balance between safety and risk'.

The Near Me evaluation ^[15] further highlights the importance of continuing to re-assess the risks and benefits of video consulting over time in order to effectively balance the advantages of reduced travel, increased service capacity and more flexible working with risks associated with clinical safety and the preferences of staff and service users for face to face services.

In considering ways in which some aspects of the 'light touch' approach could be progressed to improve future funding initiatives, the evaluation of the Supporting Communities Fund ^[21] acknowledged the importance of balancing agility and speed with the need to collect evidence on outcomes.

The Wellbeing Fund evaluation ^[20] questioned whether the pandemic presented an opportunity to review the level of 'risk appetite' in non-emergency funding situations, to assess the risks and opportunities involved in lowering the overall level of documentation required from third sector funding applications.

It is possible that a greater organisational risk appetite during the pandemic to keep people safe is being replaced by a return to risk aversion and concern about lines of accountability. An evidence review of the response of the third sector ^[14] argued that some cultural barriers between statutory and third sector organisations have returned, with third sector organisations observing that funders quickly returned to a pre-pandemic culture of more transactional and bureaucratic practices.

The Covid Recovery Strategy Assurance Report (unpublished, August 2022), states that while sharing of statistical data to support a range of Community Planning Partnerships (CPPs) reports that include the Local Outcome Improvement Plans (LOIPs), Local Authority plans, CPP Groups and Community Justice is commonplace, a broad cultural and behavioural barrier is risk aversion to sharing data, with individuals defaulting to not sharing data to ensure that they are not in breach of any legislation.

Policy and practice implications

• Risk Management: The COVID-19 pandemic helped facilitate person-centred approaches through allowing services to increase their risk appetite and take proportionate risks. In order to maintain these benefits, good risk management frameworks are required, which identify risk/ opportunity trade-offs and include the risks of maintaining the status quo and the opportunity costs of not pursuing alternative approaches.
• Reporting Requirements: It will be important to learn from the pandemic and consider whether funding principles for third sector provision should be reviewed, including rethinkingguidelines for evidencing success and considering alternatives to quantifying results in light of different local contexts.
• Getting the Balance Right: It will be important to bring people together to reflect on learning from the pandemic and consider whether we have the right risk appetite as part of this. We may need to consider new collaborative relationships based around models of shared accountability for public service delivery.
• Leadership: It is important that the toneis set from the top on risk appetite to ensure it filters to all levels (key programmes and projects). The workshops showed that participants wanted to keep the clear guidance at strategic level and risk assessment at both regional and local level but stated there should be greater awareness and analysis of risk/opportunity in advice and decision-making.
• The Social Renewal Advisory Board report ^[2] makes recommendations for new ways of working based on what worked well in the pandemic. One of these is the recommendation that public sector and voluntary sector partners should commit to long-term systems of risk and reward which focus on long-term outcomes to achieve shared goals, with the aim of delivering a secure and sustainable future for the third sector.
• The 4 Harms Model was highlighted over the course of the workshops as a useful tool for accessing risks and navigating potential trade-offs. Workshop attendees suggested that the model could be adapted and used as a tool to support evidence informed decisions and as a means to support improved policy coherence.

Geography - Evaluations of COVID-19 interventions suggest that the effects and legacy of COVID-19 is likely to vary for rural and urban areas

Key findings

Around a third of the COVID-19 evaluations included some consideration of geography. Evaluations typically focused on distinguishing the experiences of rural communities, and where this differed to the experiences of more urban areas. Evaluations tended not to distinguish between the experiences of different types of urban or rural communities.

During the pandemic, digital services and telemedicine were rapidly introduced or expanded. This had some positive impacts for people living in rural communities. Evaluations of: Youth work ^[19] and Healthcare (Near Me video consulting ^[15], telemedicine early medical abortion at home (EMAH) ^[34], Perinatal care etc.) are among those that demonstrate how interventions delivered remotely helped to maintain vital services during the pandemic ^[26]. While some interventions were being delivered remotely and making an impact pre-pandemic (e.g. Highland was the highest user of Near Me prior to COVID-19), in other cases the pandemic brought about new ways of delivering essential services. For example, in the evaluation of EMAH ^[34], staff noted that this way of delivering care was especially important in rural and remote areas - "a lifeline for rural services" - enabling them to provide high-quality care via phone/video, even when in a different geographic location.

However, evaluations also highlighted examples of where poor digital connectivity and digital exclusion in rural areas exacerbated existing social isolation and created difficulties in the implementation of, and access to, services. The qualitative evaluation of Connecting Scotland ^[37] showed that poor digital connectivity had an impact on some organisations participating in the programme. The evaluation of Near Me ^[15] showed that insufficient connectivity across rural and remote regions was one of the main barriers to patient uptake of Near Me, and also limited opportunities for some staff to run their video appointments from home^[38].

Programme implementation could be problematic if not set up to take account of geography (and particularly the challenges of delivering services in more rural areas). For example, the evaluation of the COVID-19 vaccination programme ^[39]found that the National Vaccination Scheduling System did not meet the scheduling requirements of some Health Boards, such as Island Boards and Boards covering large rural areas (e.g. patients on one island being given appointments on other islands).

Some interventions specifically focussed on addressing isolation for people living in rural communities. The review of Youth work ^[19] highlights how young people living in rural communities are affected by poor transport, poor digital connectivity, and inadequate housing. Despite these issues, before the pandemic, 72% of respondents felt optimistic about the future; in 2020, this reduced to 40%. Funding was available during COVID-19 to address social isolation and loneliness, reaching 842 rurally isolated young people and provided additional resources to improve the inclusivity of youth work provided and extra support hours to support young people in areas of high deprivation.

The issues affecting rural and urban communities differed during the pandemic. The evaluation of COVID Support in Low Income Households ^[19] showed that participants living in rural, and particularly remote rural or island communities were most likely to report severe problems with supply issues, whereas low income households in urban areas were more likely to be affected by lack of appropriate indoor and outdoor space. Evaluation of COVID-19 funding programmes show that in rural areas COVID-19 funding was often spent on improvements to infrastructure and supply lines, and reducing digital exclusion.

Several evaluations highlighted how stigma and pride also presented barriers to uptake of services in rural areas. The Supporting Communities Fund evaluation ^[21] and evidence review of the response of the third sector ^[14], both discussed numerous examples of how community-based organisations and government officials worked in partnership to find innovative solutions to this problem:

"[B]eing a small community we had to address the issue of stigma around food parcels. Food provision [became] part of our swap [s]hop initiative and this helped to break down that barrier and increased the use. Made food voucher scheme as anonymous as possible to reduce barriers to take−up. There are no written records of claimant names - they are known only by their voucher number".

Community anchor organisation, Highland.

Conversely, some services introduced as a result of the pandemic offered greater confidentiality which provided advantages for patients living in rural areas, e.g. EMAH. This evaluation shows that the telemedicine appointment may offer less 'visibility' for those patients who may be concerned about maintaining privacy and confidentiality of their care, which is particularly relevant for abortion services in small or remote and rural communities. ^[34]

The review of COVID-19 and the Centrality of Care ^[40] addressed the effects of rural factors in the gendered division of domestic labour. This study showed the significance of structural factors in rural contexts for interviewees during the pandemic - such as fewer childcare and adult/eldercare settings, job market opportunities and housing options - and how this influences and shapes gendered arrangements of family and work life.

The majority of the evaluations reviewed did not clearly report on differences in service uptake or funding between urban and rural areas.

As Covid restrictions eased, a hybrid model of working, combining remote and face-to-face services has emerged leading to some positive outcomes for staff and service users in rural areas. The qualitative evaluation of Connecting Scotland ^[37] cites positive impacts on the capacity of staff in some organisations, with time savings due to reduced travel time in rural areas, enabling more efficient use of staff resources.

In the evaluation of EMAH, NHS staff identified lasting benefits of changes to the provision of EMAH, including: improved access to care for patients; overcoming geographic barriers to timely care; reducing the need for patient travel; reducing need for multiple appointments and associated time required for these. ^[34]

The review of Youth work ^[19] shows how youth workers moved back to face-to-face youth work when restrictions allowed, all taking place outdoors in the local community. Many youth workers adapted their practice to make the most of the outdoor spaces, with young people and youth workers exploring new areas of their communities and operating in outdoor spaces not previously utilised, giving both parties a deeper appreciation of their local community. Organisations have continued to deliver some activities outdoors as they engage young people, in both rural and urban areas.

Policy and practice implications

• Travel times to access key services, including GPs, are longer in rural Scotland, and transport costs are higher ^[41]. Shucksmith et. al. (2023) stress the necessity of ensuring that people have a plurality of means of accessing services in and from rural areas – that is, providing a mix of face-to-face outreach, mobile, digital and phone services.
• The review of the needs of Scotland's migrant and minority ethnic population ^[8] also suggests that all sectors maintain a hybrid model of service provision with online provision alongside face-to-face support, depending on clients' needs and preferences. It notes: "The pandemic shows that remote service delivery can be very successful and in some cases is preferred by both the service providers and service users. Moving work meetings online has proved a convenient solution which saves time and travel costs for staff, and facilitates partnerships across geographies."
• Some evaluations recommended targeting any future iterations of programmes in rural areas where uptake was lower, for example the Connecting Residents in Scotland's Care Homes' Programme ^[24]. Programmes may need to be tailored for rural areas and needs, and greater attention to geographical diversity within rural health boards is required, e.g. journey times on and between islands.
• 'Legacy lessons' set out in the vaccination programmes in Scotland included that vaccination clinics should be held at a range of times in trusted local community locations that are person-centred and fully accessible to all. Furthermore, the technology – in this case the National scheduling systems - should be improved to meet the needs of both urban and rural Health Boards within Scotland.
• The diversity of Scotland's rural and island communities should be taken into account. Rural and island communities face specific challenges, from transport and affordable housing to service provision and ageing populations, but have shown resilience during COVID-19. Particular groups within rural and remote areas were more vulnerable to the impacts of the pandemic, such as low income households and older age groups.
• Work on the ongoing effects of COVID-19 on Scotland's rural and island communities ^[35] states that funding and grants should be accessible for those in rural areas recommending 'Retain a flexible, targeted and responsive approach to financial support, with simplified funding requirements.'
• The Verity House Agreement ^[6] offers an opportunity for greater place based leadership. It states that Scottish Government's Place Directors will act as ambassadors for Local Government and Community Planning Partnerships across portfolios and programmes and will build a strong understanding about place-based leadership and the local plans that focus on our shared priorities.
• The impact of place diversity needs further exploration. It will be important to acknowledge place and geography when developing place-based solutions for recovery, and recognise how and by whom these solutions were developed and how these might be transferred to different geographical contexts.
• The evaluation of the Covid Recovery Strategy could usefully consider the unique geographic challenges across Scotland and the impact these might have on recovery.

Equality and inclusion – There is a lack of evaluation evidence in Scotland on how and why COVID-19 interventions affected disadvantaged and equality groups

Key findings

The COVID-19 pandemic exposed and exacerbated existing inequalities. The Social Renewal Advisory Board report ^[2] states: 'COVID-19 has shone a light on structural inequalities, poverty and disadvantage and the disproportionate impact the pandemic has had on different people and communities. These inequalities were already limiting life chances and the pandemic made them worse.'

We know that the pandemic worsened inequalities, but there is a lack of detail from evaluations relating to how specific interventions introduced in response to COVID-19 affected different equality groups, disadvantaged people and communities.

Whilst some evaluations did look at uptake of interventions to address COVID-19 by disadvantaged and marginalised groups, very few of these evaluations provided any explanation as to the reasons why uptake differed between groups.

There were, however a small number of evaluations of COVID-19 interventions designed to mitigate inequalities that specifically evaluated impacts on different equality groups.These primarily focused on understanding the differences between the experience of different age groups, men and women, disabled and non-disabled people and different ethnic groups (to a lesser extent).

Interventions in response to COVID-19 were often introduced with considerable urgency, and a light touch approach was taken to monitoring to facilitate rapid implementation of emergency support measures. A lack of demographic data being collected or available, means that it has mostly not been possible to fully understand and evaluate the experiences of different groups and intersectional effects.

The Social Renewal Advisory Board report ^[2] highlights that disabled people, minority ethnic communities, people on low incomes, older people, younger people, and women may have been particularly disadvantaged during the pandemic.

This summary highlights some examples of the learning, where available, from evaluations of COVID-19 interventions about the impact of these interventions on marginalised and socio-economically disadvantaged groups. It does not provide a comprehensive overview of evaluation evidence and is primarily focused on Scottish Government evaluations. The themes of equality and inclusion are also highlighted in other thematic reviews produced as part of this series.

Evidence on socio-economic disadvantage

Overall, when looking across Scottish Government evaluations there is more evidence relating to socio-economic disadvantage than protected characteristics. Where socioeconomic disadvantage has been considered in evaluations, this is almost exclusively on the basis of SIMD, which looks at area deprivation rather than individual or household deprivation..

Evaluations suggest that demand for many COVID-19 interventions was typically higher in more deprived areas . For example, the Extended Distress Brief Intervention Programme evaluation ^[42] found that most individuals who interacted with the Level 1 service lived in SIMD 1 and 61% lived in the two most deprived quintiles in Scotland.^[43] The telemedicine early medical abortion at home intervention evaluation found that just under half of all patient respondents were from the most deprived areas. ^[34]

Evaluations also find that funding was targeted towards areas of higher deprivation. For example, monitoring information presented in the Wellbeing Fund Open Application Process evaluation ^[20] showed that awards to organisations working in a single local area were strongly directed towards areas with populations most vulnerable to the negative impacts of COVID-19. Among organisations for which SIMD data was available, over 72% of the funding went to organisations working in the two most deprived SIMD quintiles, and 48% to those working in the most deprived quintile.

However, evidence also suggests that for some population level interventions, levels of intervention take up were often lower in more disadvantaged areas and amongst particular equality groups. For example the evaluation of the COVID-19 vaccination programme ^[39] showed that the uptake of the COVID-19 vaccine was lower in younger populations, in most minority ethnic groups (but in particular Polish, Gypsy/Traveller and African groups), in more deprived areas, amongst pregnant women, and in larger, urban Health Boards.

Some evaluations of interventions designed to protect the general population from the virus, whilst beneficial, were found to have disproportionate negative effects on people living in areas of socio-economic disadvantage. For example, the summary of the Scottish evidence base on the COVID-19 mitigation measures aimed at children and young people in Scotland ^[11] found that those living in the 20% most deprived areas were more likely to feel anxious, or to find it difficult to understand teachers due to teachers' face coverings than those in less deprived areas. The evaluation of COVID-19 support study on experiences of and compliance with self-isolation ^[44] found that younger people, those with a household income of under £16,900, and those living in the two most deprived SIMD quintiles in Scotland were more likely to report that self-isolation negatively affected their employment and income than older people, those with higher incomes and those in other SIMD areas.

Evidence on disabled people

Disabled peopleexperienced a loss of access to healthcare, higher levels of anxiety, a decrease in general wellbeing, loss of employment and a greater risk of death from COVID-19 than non-disabled people. ^{[45] [46]}

The majority of evaluations of SG COVID-19 interventions did not examine the differential impact of these interventions on disabled people. Evaluations which did consider the impacts of interventions on disabled people did not generally distinguish between the experiences of people with different types of disabilities.

There were, however, a small number of studies that did provide information on how interventions affected disabled people and included the voices of disabled people through interviews (e.g. the Lockdown Lowdown report ^[47] and the evaluation of user journeys and experiences of Covid-19 and flu vaccination programme ^[48]). These evaluations provided examples of the barriers faced by disabled people (e.g. learning challenges for blind and partially sighted young people who rely on lip reading when socially distanced from teachers), and in relation to finding out about, and accessing services, e.g. practical barriers to engagement with the vaccination programme such as venues far from home, accessibility issues for those with autism or sensory disabilities. Evaluations also identified some issues relating to the quality of services some disabled people received.

Evidence on Minority Ethnic Communities

The majority of evaluations failed to look specifically at the experiences of minority ethnic communities. However, an expert review which did examine the needs of Scotland's migrant and minority ethnic population under COVID-19 ^[8] found that 'pre-existing inequalities affecting migrant and minority ethnic populations were exacerbated by the pandemic'.

Key inequalities identified in this review related to; access to food and essential products; digital exclusion; access to healthcare and public health information; housing conditions and homelessness; employment, income and access to social security benefits; education; and access to immigration information and support. ^[8]

Evidence on Gender

One of the few studies to look explicitly at the impacts of Covid interventions from a gender perspective ^{[40] [49]} showed that the loss of support structures and formal care provision, insufficient home environments and psychological stress all contributed to the increased emotional labour and burden carried by women caregivers during the pandemic. While the pandemic offered opportunities to redefine gender roles and improve work-life balance for some families, pre-existing gender inequalities and norms did not disappear.

Evidence on Age

There was limited detailed analysis of age within the COVID-19 evaluations included in this work (keeping in mind that this work was not intended to focus on the direct effects of the virus – which disproportionately affected older people – but instead the effects of interventions to address the pandemic). Reporting around age tended to be related to uptake of and compliance with interventions rather than a more qualitative and detailed understanding of how groups experienced interventions, barriers to take up and variations between age cohorts.

Evidence on compounding disadvantage and intersectional impacts

A small number of evaluations took a more intersectional perspective. For example, the evaluation of Perinatal experiences during the COVID-19 pandemic in Scotland ^[26] highlights that younger women and women from lower income households were more likely to feel concerned about having people come into their home due to COVID-19, and less likely to; have their physical and mental / emotional needs met; to feel included in their care planning; to have enough privacy and; to feel involved in planning their care.

Another example the evaluation of COVID-19 support in low income households ^{[27] [50]} which illustrates how greater challenges were experienced by low income households with additional needs, for whom difficulties accessing health/support services or benefits compounded the problems of poverty.

Further analytical work

An additional programme of work of the COVID-19 Learning and Evaluation Oversight Group is considering how we can improve our approach to evaluation during times of change. This work will report towards the end of the year and will be looking specifically at how we can better understand the effects of interventions on different groups through improvements to data, engagement and evaluation approaches.

Policy and practice implications

In order to better understand and address the needs of specific communities and help inform and shape policy and decision-making. The Social Renewal Advisory Board report ^[2] states that 'There will need to be a step change in the collection and consistent use of granular data on equalities and human rights to improve monitoring, alongside independent evaluation of how far investment is driving renewal and for whom.'

Recommendations of the Expert Reference Group on COVID-19 and Ethnicity in September 2020 included the need to have reliable and up-to-date data for minority ethnic communities, and to close evidence gaps and identify solutions. ^[51]

The Social Renewal Advisory Board report's ^[2] Call to Action 20 states that 'Decision-makers must commit to co-designing the means of gauging progress towards renewal'. One of the essential building blocks of this is: 'Consistent use of equalities and human rights data to improve capacity to monitor, alongside independent evaluation of how far investment is driving renewal and for whom.'

The Scottish Government Equality Data Improvement Programme (EDIP) and the Scotland's Equality Evidence Strategy 2023-2025 are responses to these recommendations.

Learning from the pandemic, and looking ahead, we need to better understand the take up of services, how take up relates to need and the reasons why particular groups did not take up services and what can be done to facilitate higher rates of uptake amongst those groups.

The Social Renewal Advisory Board report ^[2] also argues that 'our communities – particularly those most socially and economically disadvantaged – must be at the heart of this reconfiguration of where control over decision-making and resources lies and we must redouble our efforts to ensure that our reach includes groups with protected characteristics and others who feel excluded. This will require specific equalities approaches and consideration of human rights'.

Key Strategic Opportunities identified by the workshop participants included the Human Rights Bill. This would incorporate into Scots law the UN's International Covenant on Economic, Social and Cultural Rights ^[52], as well as three UN treaties on race ^[53], women ^[54] and disability ^[55]. It would also include environmental rights, rights for older people, and an equality clause including provision for LGBTI people. Although much will depend on the details, the Bill would bring about a major change to the human rights landscape in Scotland.^[56]

Another strategic opportunity identified in the workshops was the setting of new Equality Outcomes by the Scottish Government in 2025, which could be an opportunity to focus priorities. Lastly, workshop participants stated the importance of investment in prevention as a means of furthering equality and inclusion in the long term.