Social Security Experience Panels: legacy report

4. Feedback Survey

A survey was sent to all panel members to collect feedback on their experience of being part of the Experience Panels. The survey ran from December 2023 to February 2024, and a total of 206 responses were received.

4.1 Why people joined the panel

Over 2,400 people joined the Experience Panels over the six years they ran and although panel members were not paid to participate many remained engaged throughout.

The feedback survey asked panel members what had motivated them to join the Experience Panels. The most common responses were to ‘Design Scotland’s social security system’ and to ‘Share my experience’, with 80 per cent and 79 per cent respectively. 25 per cent of respondents said that they were motivated to join to ‘Meet other people’ (Table 4.1). ‘Other’ responses could generally be grouped under these same three areas but provided more detail.

Some respondents provided additional detail to their answers. A few stated that they wanted to make sure that the new social security system in Scotland was designed with respect and fairness in mind. This speaks directly to the core values of Social Security Scotland and reflects how the inclusion of people with lived experience was vital to the direction of developing the new benefits system.

“I wanted to help shape a fairer system in Scotland and using my experiences helped.” (Feedback survey respondent)

One respondent said that they were someone who was on the ‘edge’ of eligibility, and they wanted to make sure that people like them were considered during the design of new benefits. This is consistent with the Experience Panels’ commitment to inclusivity and ensuring that there was representation of all groups of people who may need to access social security, in the design of the new social security system.

“I wanted to be sure someone like me was considered [in the design of social security in Scotland]” (Feedback survey respondent)

Another respondent said that they wanted to speak for future generations, which aligns with Principle 1 in ‘Our Charter’, which states that ‘social security is an investment in the people of Scotland’.

“I wanted to be a voice for the future generations.” (Feedback survey respondent)

*Figures do not add to 100 per cent due to rounding, and because respondents could select more than one option.

4.2 Accessibility of participation

As a primary objective of the Experience Panels was to gather a wide range of different lived experiences, a key consideration when conducting research was to ensure that this was accessible for all participants, and that nobody who wanted to participate was excluded on the basis of individual accessibility needs.

The feedback survey included several questions about accessibility relating to in-person research, phone/video research and surveys to gauge the extent to which this aim had been achieved.

For in-person research, 94 per cent of respondents said that they had enough choice about when and how they took part. 95 per cent said that the research events were in a venue that was accessible for them, and 85 per cent reported that the location was convenient for them. No respondent disagreed with the statement ‘I was able to get support to take part if I needed it’.

However, some respondents raised accessibility concerns related to the availability of disabled parking at in-person research venues and the accessibility of venues for wheelchair users.

“Most of the in-person meetings were at venues without blue badge parking and some had accessibility issues.” (Feedback survey respondent)

“Ensure venues for in person events are easily accessible for wheelchair users and have adjacent parking.” (Feedback survey respondent)

For phone and video interviews, 93 per cent of respondents said that they had enough choice about how to take part and 80 per cent of respondents said they were able to get support to take part if they needed it (Table 4.2). One respondent strongly disagreed with the statement ‘I was able to get support to take part if I needed it,’ but no further detail was provided on what could have been improved. 18 per cent said that they ‘neither agreed nor disagreed’.

*Figures may not add to 100 per cent due to rounding

For surveys, 97 per cent of respondents said that they had enough time to complete the survey, 72 per cent said that they were able to access support to complete surveys if they needed it. 3 per cent of respondents disagreed with the statement ‘I was able to access support if I needed it’. 25 per cent said that they neither agreed not disagreed (Table 4.3). Of those who said they disagreed that they were able to access support if they needed to, none of these respondents provided further insight into that response when asked for any further information.

*Figures may not add to 100 per cent due to rounding

These figures suggest that for the majority of respondents, participation in the research was accessible. A later question asked panel members to provide further comment on what research methods should be continued or improved on in future research. Some respondents provided more information here that related to the accessibility of research, including that surveys and online video interviews were easy to access, and that more dates should be provided for research events.

“More surveys would be good, they are easy to complete.” (Feedback survey respondent)

“Keep it all but add more dates for in person panels.” (Feedback survey respondent)

“Please keep the video research studies with interviewers.” (Feedback survey respondent)

4.3 Communication with panel members

During the course of the Experience Panels, all panel members were contacted at least four times a year via the quarterly newsletters. These newsletters provided an update on research over the preceding quarter and often a ‘spotlight article’, for example an interview with somebody involved in using Experience Panels research. Panel members who participated in research were also sent a copy of any relevant research outputs related to this, such as a report or visual summary.

The feedback survey asked panel members for their thoughts on these communications and how accessible they were. 88 per cent of respondents said that the communications from the Experience Panels team had kept them up to date with the research and 96 per cent said that the reports and newsletters were in a format that was accessible to them.

For reports, which were sent to panel members who had specifically engaged in that piece of research, 93 per cent of respondents said that they were clear and easy to understand and 90 per cent said they found it useful to receive the reports.

For the quarterly newsletters, 93 per cent of respondents said that they were clear and easy to understand and 91 per cent said that they were useful to receive.

Respondents were also asked if there was anything else it would have been useful to have included in the reports or newsletters. Suggestions included: featuring more views and experiences from Social Security Scotland clients, a reduced length and/or increased frequency of newsletters, using different forms of media to communicate with panel members such as online platforms, and more detail on how the research findings had fed into changes within benefits and Social Security Scotland itself.

“Personal experiences of new Social Security Scotland.” (Feedback survey respondent)

“I may have liked a more frequent short newsletter to feel more connected to the project all the time.” (Feedback survey respondent)

“Better use of Zoom, YouTube and other online platforms.” (Feedback survey respondent)

“More of the intensely practical stuff like how the agreed ideas translate into forms, helplines, policy, complaints procedures etc.” (Feedback survey respondent)

This suggests that communicating with panel members via reports and newsletters was a valuable way to keep panel members up to date, but that the newsletters and reports could have been further developed as the Experience Panels progressed, to provide more information on how the research was connected to concrete changes within policy areas and Social Security Scotland.

4.4 Panel members’ impact on decision making

The feedback survey also asked panel members about how they thought their contributions during research had been used.

82 per cent of respondents said that it was clear how Experience Panels research was used to inform the design of the new social security system in Scotland (Table 4.4), 81 per cent said that they felt being part of the Experience Panels had allowed them to shape the new social security system (Table 4.5) and 87 per cent said that the Experience Panels were a good way to ensure people with lived experience helped to design the new social security system (Table 4.6).

* Figures may not add to 100 per cent due to rounding

* Figures may not add to 100 per cent due to rounding

* Figures may not add to 100 per cent due to rounding

4.5 Reflections on Experience Panels research from the feedback survey

Panel members were also asked for their views on what worked well during the Experience Panels and what could have been better.

What worked well?

When asked what was good about the research activities, panel members responses largely focussed on five areas: the value of using lived experience in research, opportunities to share, hearing from others, the Experience Panels team, and the accessibility of research.

Many respondents commented on the importance of involving people with lived experience in the design and delivery of social security. Many said that they were pleased to be able to use their experiences constructively and that it made them feel part of the process of developing Social Security Scotland, with several respondents stating that they felt their experiences were valued.

“I felt as if I was part of the design for devolved benefits in Scotland.” (Feedback survey respondent)

Many participants talked about being able to share their experiences as being a positive part of contributing to the research. Participating was described as being “cathartic” and they said it helped them to feel listened to.

“For me it was a cathartic exercise being able to put my experience to good use.” (Feedback survey respondent)

Respondents also said that hearing from other people and hearing their similar experiences was a valuable part of the research. A few respondents said that hearing from other people made them feel less alone.

“I enjoyed meeting other people and hearing their experiences and points of view.” (Feedback survey respondent)

Many of the respondents spoke very positively about the researchers in the team and the organisation of the research overall. Respondents made various comments, including that the researchers were friendly, put them at ease, made them feel valued and that throughout the research activities they felt able to speak openly.

“In person groups were the right size, and allowed people time to tell their experiences or answer questions. The interviewers were very good at putting us at ease, and making sure everyone was ok throughout the session.” (Feedback survey respondent)

When reflecting on accessibility, panel members commented on survey and interviews questions being clear and easy to understand, the visual display of surveys, the range of modes for research participation, the range of times available to participate and being able to take breaks.

“I found it all easy to do and take part in and I felt supported throughout the time.” (Feedback survey respondent)

“…appreciate the yellow backgrounds for text. This is good for dyslexia.” (Feedback survey respondent)

“Video conference meant I could take part.” (Feedback survey respondent)

What could have been better?

When panel members were asked what could have been better, the responses generally related to accessibility of research locations (parking was reported as an issue for wheelchair users), feedback on the research and a desire for more in-person research. These responses related specifically to the research activities, but a later question in the survey asked panel members what should be kept, stopped and improved for any potential future research. The responses for that question were broadly consistent with these themes, but also included the suggestion to improve the diversity (age and ethnicity) of the panel.

Survey respondents noted that they would have liked to have heard more about what happened to their contributions after the research was complete. Some respondents said that they had been able to see the impact of what they had said, but others said they would like to know what had been implemented as a result of the research they had participated in.

“More thorough explanations of how panel members’ ideas were applied in the actual processes of using the new Scottish system would have been better.” (Feedback survey respondent)

4.6 Impact for panel members

Being part of the Experience Panels represented a long-term commitment for many panel members, and their consistent engagement represented a personal investment of time and energy into the panels.

The feedback survey asked panel members if they felt that they had achieved what they hoped through their participation. Many responded with a simple ‘Yes’ or ‘No,’ while others provided more detail on how they felt about their time on the panel. Many panel members expressed hope that they were part of something that had created a positive change for Scotland’s social security system. Some said that being involved in the panel had helped them feel active and involved in something, indicating that there was value for some participants beyond research outcomes.

“I loved being a part of the panel, I hope that this has helped to create a fairer system for all.” (Feedback survey respondent)

“I also achieved a sense of worth which I have not felt for a long time. I now actively look for ways to become involved, to make a difference, which I have not done before.” (Feedback survey respondent)

Other panel members expressed disappointment. A few felt that their time on the Experience Panels had not covered topics important to them. For example, some panel members wanted more opportunities to talk about mental health. Others were disappointed as they felt that their contributions and suggestions had not been implemented.

“I don't feel I had enough opportunity to talk about mental health.” (Feedback survey respondent)

“No, I am extremely disappointed that what information was given by myself and others was not heard when I came to deal with the system.” (Feedback survey respondent)

However, other respondents felt they had made a difference and enjoyed being able to talk to others and help with the area they had most experience in.

“I feel we made significant progress in Carer's Allowance.” (Feedback survey respondent)

Many panel members were undecided about whether they had achieved what they had hoped to, as they did not yet have experience of the new social security system. Others wondered about how much of their contributions would be implemented.

“It remains to be seen. I hope what we have said will be taken on board and it will be clear when benefits are up and running.” (Feedback survey respondent)

4.7 Overall experience of being on the panel

Panel members were asked some questions around their general experiences when taking part in the panels. The vast majority (91 per cent) of respondents said that they had enjoyed taking part in the Experience Panels research, with more than half (55 per cent) strongly agreeing (Table 4.7). Similarly, more than four-fifths (81 per cent) of respondents said that they had felt listened to, with almost half (48 per cent) strongly agreeing (Table 4.8).

Although only a few panel members stated that they did not enjoy taking part, their comments referenced various issues. These included feeling that contributions to research had not been implemented in service design, that there had been no improvement in service delivery from Social Security Scotland compared to DWP, or that the panel member had not been invited to take part in enough research.

*Figures may not add to 100 per cent due to rounding

*Figures may not add to 100 per cent due to rounding

Drawing on the aspirations of Social Security Scotland, panel members were also asked if they agreed or disagreed that they had been treated with dignity, fairness and respect while participating in the Experience Panels. Almost all (95 per cent) respondents agreed with this, with almost three quarters (72 per cent) strongly agreeing (Table 4.9).

*Figures may not add to 100 per cent due to rounding

Finally, panel members were asked if they would take part in research organised by the Scottish Government again. Almost all (93 per cent) respondents said that they would, with only 2 per cent responding ‘No’ (Table 4.10)

“I wanted to make sure others didn’t have to experience the same. The panel made me feel listened to and our experiences and suggestions to make things better were heard. I would, without hesitation, volunteer for anything similar.” (Feedback survey respondent)

* Figures may not add to 100 per cent due to rounding