Social Security Experience Panels: annual report 2019

2019: How we worked

This section will give a project update on what we have been up to in 2019. It will cover:

• Recruiting and welcoming new panel members
• Reaching seldom heard groups
• Our mid-programme peer review
• Undertaking more user testing
• Working with Client Insights researchers
• Sharing our approach

Recruiting new panel members

Recruitment to the Experience Panels closed in 2017 when the initial recruitment finished. Since that time we lost touch with a number of existing panel members, and others have chosen to leave the panels. There were also some specific groups that we wanted to boost, namely young people, ethnic minorities and people in remote rural and island communities.

So in July this year we reopened recruitment to refresh panel membership. We publicised the recruitment with the help of current panel members, a large number of partner third sector organisations and colleagues working in local areas for Social Security Scotland. We also circulated information about joining on social media. We tailored our approach to different groups, for example running a series of events with ethnic minorities.

As a result of all of this activity, we have now welcomed 536 new panel members, and we will soon set a closing date for the recruitment. We have begun sending our ‘About Your Benefits and You’ survey to new members to find out more about them. We’ll be reporting on the demographics of our new panel members, and of the panel as a whole, in the coming months.

Research with Seldom Heard Groups

We have been aware from the outset that there are a range of sensitive, marginalised and dispersed groups of people in Scotland who are less visible within or less likely to join a project like the Experience Panels, but who have particular experiences or characteristics that mean they may face barriers when engaging with public services.

This year we launched our programme of seldom heard research, which has five strands:

• Mobile populations: gypsy traveller populations, temporary EU migrants, refugees.
• Care Experienced and Carers: Care experienced people, foster and kinship carers, young parents, single parents.
• Vulnerable groups: people who have offended, people with experience of homelessness, veterans.
• End of life: people with terminal illnesses, bereaved families
• Survivors: survivors of domestic abuse and of childhood abuse.

We have commissioned external experts to conduct two waves of fieldwork with each of these groups over the next two years. This research will examine the particular experiences, needs and views of these groups, to ensure that Social Security in Scotland is as accessible as possible to these groups, and is sensitive to their specific needs.

Our Mid Programme Review

2019 marks the mid-point of the Experience Panels, which will run until all of the relevant benefits are live. We marked the occasion by commissioning a neutral expert from outside of the Scottish Government to review our processes and impact.

The peer reviewer examined a large number of our documents, and conducted a site visit to meet the team. We asked if our data management processes, processes to engage and communicate with panel members, research processes, and governance and stakeholder engagement processes were appropriate for the project, and if the research was impacting on relevant decision making. She found that the answer was yes for all of these and gave detailed and largely positive comments on what she had found. These included some suggestions for improvement, covering:

• Suggestions for minor clarifications around Data Protection
• Data management suggestions, including suggesting ways to file research materials.
• Suggestions for findings, including looking at how fast they are provided to colleagues, and compiling them in an accessible, searchable format.
• Advice to monitor the size of the internal Working Group within Scottish Government which supports researchers to run the panels.

We have looked at her findings in detail, and are putting in place resource to address all these points. Her overall conclusions can be found on the next page, and the full report is now on our publications page.

User testing work

The work we are doing with panel members is getting ever more specific as we move towards more and more benefits going live. This year there have been increasing opportunities for panel members to get involved in user testing work. This is different from our usual surveys, focus groups and interviews. It involves asking panel members to look at draft parts of a process, for example a piece of an application form, and see how easy to use it is.

This kind of work requires fewer participants than our wider research, and usually needs to be done face to face because of the hands on nature of the process. We still, however, make sure we do all we can to support panel members to take part in these sessions.

User testing like this feeds directly into design, and doesn’t have a research report like our other work. We do, however, include articles in our newsletters on how such testing has shaped processes.

Spotlight On: Mid Programme Review

“The work of the EP team is interesting and innovative for its type and I have been impressed by the evidence I have reviewed and learning about the origin and operation of the EP team. It is clear that after an initial period of ‘start up’ the team and its work are in a phase of establishment and with it now have the opportunity to maximise the impact and profile of the work with decision makers and think about what aspects of the programme (and the learning it has done) can feed into an exit strategy and/or overlap with the client insights team.

There are also aspects of panel management and administration that would be worth revisiting if Scot Gov were to adopt a panel as an ongoing research mechanism. For example:

• Whether you want to work with longitudinal data and if so an awareness of how that could change what is disclosive
• Issuing surveys via multiple modes to each panellist as standard and issuing unique survey links to panellists to more likely secure matched data
• Methods and approaches designed to increase response rates

Finally, whilst I appreciate that one aspect of leading work of this type could be requests from others on potential replication, as discussed with the team, this work is successful because of a particular constellation of context and opportunity which isn’t necessarily the case in other places and on other issues. However, I do think it could be useful to focus on the learning the team have achieved that specifies principles and considerations for user involved research in public service design and this could be an area that would be a sensible use of time and investment in sharing the work of the team any wider.”

Client Insights

Our partner researchers in Social Security Scotland have been busy this year establishing a programme of Client and Staff research. Their research is with those who use and deliver Social Security in Scotland, and will track their experiences and drive continuous improvement^[2].

To date this has included establishing feedback surveys for when people phone Social Security Scotland, or when they apply for a benefit online, over the phone and using a paper form.

They are currently preparing for their first All Client Survey, which is due to launch during the first part of 2020. The survey will include an opportunity for clients to volunteer to take part in further research. This will result in a longer term Social Security Scotland version of the Experience Panels. We have worked together closely on this and will use everything we have learned from the Experience Panels to shape how they work with clients.

The existing Experience Panels and client research will therefore overlap for a couple of years. Panel members who are also now clients of Social Security Scotland may wish to participate in both. We will work together to make sure that is straightforward. For example we will avoid scheduling events at the same time, and we will make sure it is clear in invitations which research programme each project is part of.

Sharing what we have found

At this mid-point of the Experience Panels work, we have focused this year on increasing the time we spend communicating with others.

For panel members, this has meant rolling out a newsletter, and adding lunchtime update sessions to our focus group events. Both of these allow us to keep panel members up to date with the Experience Panels programme, and wider developments in the design of Social Security in Scotland.

We have continued to run twice yearly events for stakeholder organisations outside of the Scottish Government to keep them up to date with what we are doing. We have also presented on our work to academic and professional audiences.

And for colleagues within government, as well as our usual dissemination of findings, we have presented on our findings and approach to colleagues across the Scottish Government, and responded to many requests for information about what we do from a range of UK government departments.

On the next page we explore further the reflections we have made on the Experience Panels approach when telling others about the work.

Spotlight On: Reflecting on our Approach

We are sometimes asked about the Experience Panels by others considering beginning similar work. This is what we tell them:

Challenges: Resourcing: The Experience Panels are very resource intensive, requiring a team of researchers and support staff, and a budget for events and publications.

True accessibility: it can be challenging to provide all adjustments participants require, especially when booking venues across Scotland.

Complexity of social security: Social security is a large and fast paced area, we have to work hard to ensure research is well timed.

Strengths: Quality and value of ‘lived experience’ input: put simply, panel members give us insights we could not get any other way.

Buy in across social security: Doing the research ourselves, and the fact that it is part of the wider culture of dignity, fairness and respect, means colleagues across social security are seeing the value of the research and using it to inform decisions.

Positive experience for panel members: Participants tell us they enjoy taking part and appreciate the opportunity to shape a system that works for them.

Transferability:

There are many different ways that the Scottish Government can engage members of the public, and the Experience Panels are one model among many. Anyone considering the approach should think carefully about the scale, depth and range their situation requires, and should focus only on questions that have potential to shape decisions. Finally, the administration and logistical requirements for this approach should not be underestimated.