Student Finance and Wellbeing Study (SFWS) Scotland 2023-2024: technical report

1. Background to the study

This is a report on the research methods used in the Student Finance and Wellbeing Study for Academic Year 2023-2024 (SFWS 2023-2024) in Scotland. The study was carried out on behalf of the Scottish Government (SG). It used a mixed-methods approach comprising a survey and qualitative focus groups and interviews with full- and part-time students in colleges and universities across Scotland. The SFWS survey collected up to date and detailed information on the income, expenditure and debt levels of students, as well as their financial wellbeing and the impact of finances on their lives and study choices.

The survey part of the study was based on the UK Government Department for Education (DfE) Student Income and Expenditure Survey (SIES). This survey was last carried out with students in England and Wales in the academic year 2021-2022. In Scotland, the last study of this type was the SIES 2007-08^[1]. The SFWS Scotland 2023-2024 web survey was a 30-minute questionnaire, taking a similar approach to the last three waves of SIES in England and Wales.

This study, the SFWS, was developed to be applicable to the context of student funding in Scotland and in doing so, it extended the scope of the SIES in England and Wales to include further education (FE) and postgraduate (PG) students in addition to higher education students. SFWS did not include a 7-day expenditure diary, which was included in England and Wales’ SIES 2021-2022. Due to these differences in methodology, scope and policy contexts, care should be taken when making any comparisons between the two sets of data – both over time and between places (see ‘Comparisons with SIES’ section below).

The Scottish Centre for Social Research (ScotCen) carried out all elements of SFWS 2023-2024 including the sample design, liaising with colleges and universites, questionnaire design, delivery of the survey and qualitative research, data preparation and analysis.

Overview of the methodology

This SFWS 2023-2024 technical report provides detailed descriptions of all aspects of the survey and qualitative data collection, including the development phase. To give an overview of the research process, the key activities are outlined within the project timetable in Table 1.1.

Table 1.1 Project timetable

March 2023

• Start of the SFWS 2023-2024 contract and project inception

March-May 2023

• Design of sampling plan
• Development of online survey pre-pilot
• Pre-notification letter sent in May to Principals of universities and colleges in Scotland from the Head of Education Analytical Services at the Scottish Government.

June 2023

• Survey pilot fieldwork – 66 students completed the full survey across 17 institutions (6 colleges and 11 universities)
• Opt-in invitation email sent to institutions

July-October 2023

• Survey amends – post-pilot to capture learning and feedback
• Reminder emails and calls to institutions that had not yet opted in or out (July-August)
• Instructions issued to participating institutions on how to randomly select a sample of students and invite them to take part
• Support from ScotCen researchers was given to institutions with queries on sampling during this period
• Review of profiles of sample compared with institution and any discrepancies followed up with key contact at institution

October–December 2023

• Survey fieldwork – 2,429 students participated in the survey across 32 institutions (16 colleges and 16 universities). Invites were sent to students directly from their institutions
• Up to 4 reminder emails were sent by institutions to all selected students in the sample, as well as general communication on the study survey using existing social media channels/ websites
• The fieldwork period was extended and the number of reminder communications increased due to lower than expected responses from FE and part-time students
• Development of the qualitative (interview and focus group) topic guides informed by early data from the survey

November 2023-February 2024

• Qualitative fieldwork – 53 students participated in either a follow-up interview (n=30) or focus group (n=23; 5 focus groups held)

January-June 2024

• Survey analysis, including data checking for inconsistencies and errors, and coding and editing of data
• Development of summary measures for sources of income, expenditure, debt, and savings for use in analysis
• Qualitative analysis, including thematic coding of data, and chapter reporting

July-October 2024

• Survey reporting, including running and checking of data tables and drafting analysis of data
• Synthesis of qualitative and quantitative elements of data analysis for final report and development of research findings
• Preparation and finalising of the study report, research summary, infographic, this technical report and background data tables

The survey fieldwork ran from 30^th October to 22^nd December 2023. This was timed to allow colleges and universities to complete the matriculation process of new students beginning in the academic year 2023-2024. The sample for the qualitative research was drawn from the survey sample. The qualitative data collection was timed to take place later in the first term and into the second term of the academic year, to ensure first year and new students had more time to experience the impact of finances on student life. Interviews and focus groups took place between 20^th November 2023 and 14^th February 2024.

Questionnaire design

The survey was developed based upon the key questions covered in the Department for Education (DfE) Student Income and Expenditure Survey (SIES) 2021-2022 which were relevant to the Scottish context. The survey was adapted to be relevant to Scotland’s student funding and support system, which differs to that of England and Wales. The Scottish survey covered students enrolled on FE, HN/undergraduate and postgraduate courses – more information on the students in scope in the study is in Section 2 on sampling.

There was a comprehensive review of the England and Wales SIES 2021-2022 questionnaire to ensure the questions were suitable and relevant for the Scottish context. This led to the addition of individual questions to capture information on each of the bursaries, allowances and grants that are available to Scottish students.

The current cost-of-living crisis and recent findings from NUS Scotland’s Broke report^[2] provided a case for expanding the questions on financial wellbeing to greater explore the impact of financial hardship on students.

In addition, terminology, definitions or references were amended to be in line with current Scottish policy. Where possible maintaining the potential ability in the future, with further analysis, to understand any differences in student’s financial experiences between Scotland and England and Wales.

Following agreement of the final questionnaire, the online survey was scripted and the electronic questionnaire was then tested. Questionnaire content, question wording, routing, internal consistency checks and any text substitution, was thoroughly tested by ScotCen staff.

Before issuing the full sample, ScotCen conducted a pilot with 66 students from 6 colleges and 11 universities to test survey administration, length of the survey, questionnaire links and access codes. Respondents were asked to provide written feedback on: questionnaire length, accessibility of the survey tool, technical issues, question phrasing especially in relation to sensitivities for particular student populations, and overall views of the questions. The pilot respondents were from FE, HN/undergraduate and postgraduate courses and from a range of widening participation groups.