Coronavirus (COVID-19): Learning and Evaluation Oversight Group minutes: 5 February 2024

Attendees and apologies

• Linda Bauld, Chief Social Policy Adviser (Scottish Government), and Professor of Public Health (University of Edinburgh) (Chair)
• Nick Watson, Professor of Disability Studies, University of Glasgow
• Patricia Findlay, Professor of Work Employment and Organisation, University of Strathclyde
• Audrey MacDougall, Chief Researcher, Scottish Government
• Steven Marwick, Director, Evaluation Support Scotland
• Adam Hall, Improvement Service
• Sarah Skerratt, Chief Executive, RSE
• Louise Macdonald, Director General Communities, Scottish Government
• Vittal Katikireddi, Professor of Public Health, University of Glasgow
• Nicola Dickie, Director of People Policy and Health and Social Care, COSLA
• Pamela Smith, Public Health Scotland
• Andrew Connell, deputising for Mary McAllan, Director for Covid Recovery, Scottish Government
• Simon Wakefield, deputising for Alison Cumming, Director of Budget and Public Spending
• Tom Lamplugh, Head of Social Policy Unit, Scottish Government (Secretariat for the Covid-19 Learning and Evaluation Group)
• Graeme Wilson, Social Policy Unit, Scottish Government
• Fran Warren, Social Policy Unit, Scottish Government

 

Apologies were received from:

• Kay Tisdall, Professor of Childhood Policy, University of Edinburgh
• Alison Cumming, Director of Budget and Public Spending
• Gary Gillespie, Chief Economist, Scottish Government
• Mary McAllan, Director for Covid Recovery, Scottish Government
• Andrew Kerr, Chair of SOLACE
• Jim McCormick, Chief Executive, Robertson Trust

Items and actions

The Chair welcomed everyone to the ninth and final meeting of the COVID-19 Learning and Evaluation Oversight Group.

There were no actions arising from the last meeting. 

The Chair explained that the main purpose of the meeting was to share the key conclusions from the ‘Evaluation during times of rapid change’ work and then to provide a summary of the key achievements of the group and opportunities for further impact and dissemination, and to discuss together the prioritised messages from the group.

 

Evaluation during times of rapid change

The Chair welcomed Graeme Wilson to present on the work of the Evaluation sub-group.  The sub-group reviewed evaluation outputs and drew out lessons on 4 themes around building in evaluation during times of change, agreed with the group at a previous meeting. 

Following the presentation, there was a discussion around how the findings would contribute to learning from the pandemic, including whether this could be developed into a central resource or database that people could feed into. 

Key points raised during the discussion included:

• questioning the extent to which the types of evaluation undertaken during Covid-19 differed significantly to pre-Covid practice
• the significant value associated with framing this work around ‘what did pandemic evaluation tell us about evaluation?’
• linking this work to the new Scottish Government Evaluation Strategy, and potentially adding a strand to that on resilience and crises
• the applicability of this work to everyday policy development
• the role of the third sector in working directly with people with protected characteristics and carrying out small scale evaluation projects throughout the early stages of the pandemic
• the importance to look at the cost of not doing something as well as the cost of doing it, and that any evaluation framework needs to consider that too

The intention is to publish the report on the Covid-19 Learning and Evaluation Website.  The secretariat will also brief the Deputy First Minister and relevant Ministers on the report’s conclusions. The work will be used to inform the Scottish Government’s evaluation strategy.

Audrey MacDougall updated the group on the progress being made by the Equalities Data Improvement Programme. 

The Chair thanked all members of the sub-group (Kay Tisdall, Steven Marwick, Pamela Smith and Audrey MacDougall) for their support in taking this work forward.

 

Presentation on summary of key achievements of the group and opportunities for further impact and dissemination

The Chair welcomed Tom Lamplugh and Fran Warren to present on the present on the key achievements of the group.  Tom and Fran discussed the programme of grant funded expert reviews, the learning from the pandemic papers, including the synthesis of evaluations of Scottish Government and Public Health Scotland Covid-19 interventions, the published report on person-centred approaches, and workshops on Covid recovery that ran in later summer 2023 with around 50 senior participants. 

The group then discussed their reflections on the work produced by the group, and were positive about the range and quality of work the group has produced on time.

Some group members asked about the ways in which the momentum and learning can be maintained, following on from the discussion above.

 

Conclusion and thanks

The Chair invited Louise MacDonald to say some words of reflection. 

Louise thanked group members for their active participation in the group over the last two years with particular thanks to those members of the group who have been part of the two sub-groups, supported the expert reviews and participated in the workshops that ran last summer.

Louise recognised that the group has synthesised and published a large body of work which contains some important messages and learning. This work will be helpful in relation to informing ongoing work around public service reform, strengthening our approach to evaluation and enhancing our national resilience.

Louise welcomed the strong focus on equalities that cuts across this work and made recognition of the fact that the pandemic did not affect all groups equally and in some instances served to further widen pre-existing inequalities. One lesson that this work has repeatably highlighted is the need to systematically collect better disaggregated data so we are able to better understand how policy interventions affect particular groups such as women, disabled people and ethnic minorities, and to use the culmination of this work as a catalysing moment to address these gaps.

Louise highlighted that she was pleased with the transparent and accessible approach taken by the group with all key outputs published on the group’s webpages. And that numerous briefings on individual outputs had been provided to DFM, Cabinet Secretaries and relevant Ministers.

She thanked Linda Bauld for her role in chairing the group, and the secretariat (Tom Lamplugh, Fran Warren and Graeme Wilson) for driving forward the work of the group

 

Actions

Graeme Wilson to complete and publish the report on ‘Evaluation during times of rapid change’.

Office of the Chief Social Policy Adviser to discuss next steps for maintaining the learning from the group with the Chief Social Researcher, Audrey MacDougall, including how the work will be used to inform the Scottish Government’s evaluation strategy. 

Office of the Chief Social Policy Adviser to discuss next steps for dissemination of the work with the Scottish Leaders Forum.