Social security experience panels - About Your Benefits and You: qualitative research findings

Annex A: Methodology

The Survey

The ‘About Your Benefits and You’ survey was carried out between July and September 2017. The survey was multi mode with panel members being able to complete the survey online, in hard copy or over the phone. This included offering respondents a choice of language to complete in and a range of other accessible formats. The survey was for adults, over the age of 16.

The experience panels are longitudinal so information from the ‘About Your Benefits and You’ survey was added to information collected during the registration phase ‘Have Your Say’, including postcode, accessibility requirements and information about the benefits that respondents have experience of.

‘About Your Benefits and You’ asked respondents about their present and past experience of applying for, getting, challenging and appealling:

• Disabilty Living Allowance ( DLA)
• Personal Independence Payments ( PIP)
• Attendance Allowance ( AA)
• Severe Disablement Allowance ( SDA)
• Industrial Injuries Disablement Benefit ( IIDB)
• Carer’s Allowance ( CA)
• Funeral Expenses Payments ( FEP)
• Sure Start Maternity Grants ( SSMG)
• Cold Weather Payments ( CWP)
• Winter Fuel Payments ( WFP)
• Discretionary Housing Payments ( DHP)
• Scottish Welfare Fund ( SWF)
• Universal Credit ( UC)

Respondents answered both in relation to their experience of applying themselves, and their experience of helping others. Responses also included reference to wider benefits, for example ESA, and this is noted where applicable in the text. Respondents were asked to rate their overall experience from ‘very good’ to ‘very poor’, and were asked to provide their top three priorities for Scottish Government to improve in the new social security system. They were also asked a series of qualitative questions relating to their experience which are analysed in this report.

Respondents were also asked basic demographic information about their age, gender, disabilties and long term health conditions, caring responsibilities, and the make-up of their household.

Following data cleaning, 1,144 experience panel members completed the ‘About You and Your Benefits’ survey.

Analsysis of the quantitative data (from the tick box questions), including a breakdown of respondents, was published in November 2017 and is available at: http://www.gov.scot/socialsecurity-experiencepanels-aboutyou.

The focus groups

35 focus groups were held in locations across Scotland, meeting with more than 250 panel members. These focus groups were structured around a discussion of respondent’s priorities for improvement in the new social security system and their reasons for this.

Analysis and reporting

The combined dataset from the survey and focus groups is reported here after analysis by Scottish Government researchers. Survey and focus group respondents were asked about their experience of the current benefits system in general, rather than in relation to their experience of specific benefits. This analysis is therefore based on key themes relating to the system generally, and draws out detail relating to individual benefits where responses explicitly mentioned a specific benefit.

This data was analysed using NVivo 11 software. For each open-end question 10 per cent of responses were manually coded in full to identify reoccurring themes. Automated coding and analysis tools, including word frequency analysis, coding query and text search tools, were then used to support the analysis of the full dataset. The same approach was taken to analysing the focus group reports.

This approach to analysis allowed researchers to identify key themes relating to priorities for the Scottish Government to consider in the new system, what currently works well and areas for improvement. The report has used a large number of quotes from respondents and focus group participants to illustrate themes that were identified through this analysis. Evident spelling mistakes and typos have been corrected in this report, but use of colloquial language and expressions such as use of capital letters for emphasis has been kept.