Student Finance and Wellbeing Study (SFWS) Scotland 2023-2024: main report

2. Methodology

2.1. Research aims, objectives and research questions

The purpose of this study was to provide up to date information on the financial experiences of students in further and higher education in Scotland. More specifically, the aims of the study were to:

1. Explore the income, expenditure, debt and savings of students in the academic year 2023 to 2024.

2. Gather information on students' experiences of, and attitudes towards, student finance, financial management and the support available whilst studying.

3. Understand the impact that students' finances have on their experiences of studying for a higher and further education course, including on their physical and mental wellbeing.

4. Identify groups of students, including those with protected characteristics, who are in, or are at risk of, financial hardship.

5. Form a baseline for any future similar surveys and provide information on student finance in Scotland prior to the introduction of any future student support policies or policy development.

The objectives of the research were to:

• Provide an up-to-date picture of the financial experiences of the student population in further and higher education in Scotland.
• Inform understandings of how students experience and perceive finance and financial management whilst they are studying.
• Provide insights, where the data are available, on the experiences of particular populations, including students who are: care experienced, estranged, disabled, parents and/or carers.
• Develop an understanding of the impact of current student support policy on students and the difference it makes, including any factors that negatively influence financial experiences.
• Provide evidence to inform the future delivery and development of policy on student support in Scotland.
• Fill a long-standing 15 year evidence gap in this area.
• Provide learning and insights for the development of future research and surveys in this area.

2.2. Research design

To address the research aims and objectives, ScotCen conducted: a targeted literature, documentary and policy review; a representative web survey of students; and qualitative research with students in the form of five focus groups and 30 in-depth interviews. This project was called the Student Finance and Wellbeing Study (SFWS).

The survey was live between 30^th October to 22^nd December 2023. Interviews and focus groups took place between 20^th November 2023 and 14^th February 2024.

To ensure the research was designed and implemented in an ethical manner, an application was submitted to NatCen Research Ethics Committee (REC) who critically review all NatCen and ScotCen projects to ensure they meet high ethical standards. Ethical approval for the research was granted by NatCen REC in May 2023.

A Research Advisory Group (RAG) formed of public and third sector organisations who lead on policy and/or who engage with students provided support and expert advice to the project. The RAG offered feedback on language/ terminology, advised on the content of research tools, encouraged participation in the study via networks and provided information on the current policy and financial support available to students.

2.2.1. Review of policy literature

To ensure the research built on existing evidence, a rapid literature, documentary and policy review was conducted to inform the development of the survey and qualitative topic guides. This included a review of published evidence on contexts and experiences of student finance, including student financial support, focusing on further and higher education students in Scotland. The majority of literature on student finances and students' experiences of funding and financial wellbeing is based on UK-wide or England and Wales-only data. This review focused on the experiences of Scottish-domiciled students studying in Scotland rather than in the rest of the UK to remain in line with the scope of the study. The following inclusion and exclusion criteria were established.

Inclusion criteria:

• Population: Scottish-domiciled students who are residentially eligible for fees and/or student support and are enrolled in a Scottish college or university.
• Scope: Research and policy studies within the UK.
• Date/language: published in English from 2014 to the present.
• Data sources: peer-reviewed articles published in academic journals; official published government (UK and Scotland) statistics and research reports; third-sector organisations and non-governmental research agency/social change organisation published reports.

Exclusion criteria:

• Population: Non-Scottish domiciled students; Scottish domiciled students attending colleges and universities in the rest of the UK; students domiciled in Scotland who are not eligible for fees and/or student support in Scotland (such as some asylum seekers, and students whose families are living in Scotland on certain work visas); Scottish students at private training providers; international students.
• Scope: Studies undertaken outwith the UK.
• Date/Language: published in any language other than English and/or published before 2014.
• Data sources: Opinion pieces, editorials, blogs, unpublished statistics or reports.

The literature search was conducted between February 2023 and February 2024. To identify peer-reviewed evidence, searches were conducted using academic catalogues and Google Scholar. Grey literature searches were conducted in the search engine Google to identify Scottish Government statistical data and research reports, and third sector and non-governmental organisation research reports. Additionally, researchers drew upon existing literature searches conducted by the Scottish Government, as well as literature identified by stakeholders and members of the Study's Research Advisory Group. In total, 67 documents were reviewed. These included journal articles, grey literature, policy documents and relevant websites, including those of government bodies.

Evidence was reviewed and synthesised using a framework approach, which involved summarising the evidence thematically so that the review systematically captured the information needed to address the core research questions.

2.2.2. Survey of students

Quantitative data was collected via a 30-minute online survey issued to a randomly selected representative sample of 52,812 Scottish-domiciled students at 32 (16 colleges and 16 universities) institutions in Scotland. Survey responses were received from 1,734 full-time and 694 part-time students (2,429 students in total).

Questionnaire development

A 30-minute web questionnaire was developed, covering key topics of interest and a range of socio-demographic information.

Specific elements of the Department for Education (DfE) Student Income and Expenditure Survey (SIES), which were relevant to the Scottish context, were integrated into the SFWS survey. However, because Scotland's student finance system differs to England and Wales, some questions had to be adapted, or new questions added, to ensure they were relevant to the Scottish system and current policy. The current cost-of-living crisis and recent findings from NUS Scotland's Broke report provided a case for expanding the questions on financial wellbeing to greater explore the impact of financial hardship on students. In addition, terminology, definitions or references were amended to be in line with current Scottish policy.

Before issuing the full sample, ScotCen conducted a pilot with 66 students from 6 colleges and 11 universities to test survey administration, length of the survey, questionnaire links and access codes. Respondents were asked to provide written feedback on questionnaire length, accessibility, technical issues, question phrasing and overall views of the questions. The pilot respondents were from FE, HN and undergraduate and postgraduate courses and students from under-represented groups.

Survey sampling and recruitment

The population and subgroups of interest in SFWS were similar to those of the DfE SIES, but the scope was widened for the SFWS to include FE and postgraduate students, who were not included in the DfE SIES.

A census approach to sampling was implemented, with all eligible universities (n=19) and colleges (n=26) in Scotland invited to participate in the survey. Initially, 16 universities and 20 colleges agreed to take part, with 4 colleges subsequently opting out. Therefore, 16 universities and 16 colleges participated in the research. For anonymity none of the universities and colleges will be identified in this report or the dataset.

Participating institutions were then asked to select a random sample of their students, and were sent a detailed protocol on how to do this. The protocol carefully specified which students were eligible, then requested a systematic random sample be drawn. ScotCen specified the number of full-time and part-time students to be selected.

The following information on students eligible to take part in the survey was supplied to ScotCen by the participating universities and colleges:

• Full-time or part-time study
• Sex
• Age group
• Further or higher education level of study
• Undergraduate or postgraduate study.

This information on the profile of eligible students was then used in the weighting strategy. This allowed adjustments to be made to the profile of responses to ensure that they more closely match eligible student population estimates ensuring that the responses to the survey are representative of students from the three different study levels: FE, HN/undergraduate and postgraduate. Full details on the weighting strategy, as well as on sampling and recruitment, are provided in the Technical Report.

Sampled students were sent an email directly from their institution, designed by ScotCen, inviting them to participate in the survey. Two reminders were scheduled to be sent to students by the institutions after the first invitation. The number of reminders issued was increased due to lower than expected responses, particularly from college students. Therefore, many institutions sent 3-4 reminders. Institutions and other stakeholders publicised the study on their websites and social media channels to encourage response amongst students in the sample. The content of social media and website posts was provided by ScotCen.

The survey invitation and subsequent reminder emails sent to students provided information about the survey and how to take part including the web survey link. Also included were details of how data will be used in compliance with GDPR, a link to the study's website and Privacy Notice, and ways to find out more about the survey or to opt out from further communication. Sampled students were informed that participation was voluntary and that the survey was confidential. It was also explained that respondents could skip any questions they did not wish to answer, and all respondents were provided with a link to a range of relevant organisations offering support or information. Survey participants were given the chance to enter in a prize draw as a thank you for their time and as an acknowledgement of their contribution to the study. They were asked if they would like to be entered into a prize draw with a chance to win an iPad or one of 6 Love2Shop vouchers worth £50 or £25, with 95% of students who took part entering into the prize draw. Communications were developed in line with standard accessibility guidelines and were tested during the survey pilot and adapted in response to feedback.

Survey analysis

Throughout the report, data is reported separately for FE, HN/undergraduate and postgraduate students. Postgraduate students' data includes Masters, Postgraduate Diploma and PhD students. Further breakdown by these levels of study is not possible due to low sample sizes. However, HN/undergraduate data is reported by full-time and part-time status as sample sizes allow for this additional breakdown.

In addition, the report includes analysis of five under-represented groups of students: care-experienced, parents, estranged, unpaid carers, and disabled students. It was not possible to split these into FE, HN/undergraduate, and postgraduate students due to low numbers. However, analysis was undertaken on all student respondents in these five groups of interest regardless of their level of study to provide some initial and high-level insights into their financial experiences whilst studying.

Findings are presented for the 2023 to 2024 academic year, which is assumed to be 39 weeks for all students. The report uses median average values for income and expenditure data (continuous data), in line with the reporting conventions for the England and Wales Student Income and Expenditure Survey (SIES). Median amounts are calculated only for those students who have incurred the particular costs or received the particular sources of income being discussed, rather than showing the medians across all students.

The accompanying data tables for the median amounts also present means and the standard error of the mean. The standard error is a variance estimate that measures the amount of uncertainty (as a result of sampling error) associated with a survey estimate. Statistical testing of differences between medians has not been conducted, so most medians are reported without reference to amounts being higher or lower than each other. Where a median is cited as being higher than another median, this is based on whether the mean amounts are statistically significantly different, reviewing the standard errors and the size of the difference between the two medians.

Throughout the report where categorical data is reported showing differences in proportions between subgroups, the analysis included chi-square tests of statistical significance.

Where differences between subgroups, such as male and female students, are described in the text of the report, these are statistically significant at the 95% level or above, unless otherwise stated. This means that the probability of having found a difference of at least this size in the sample, if there was no actual difference in the population, is 5% or less. Where differences are described as marginally significant, this means that they are significant at the 90% level or above - this is included in a footnote. Not all subgroup analysis is presented in detail in this report. A full set of data tables are available as separate supporting documents.

Very small base sizes can produce uncertainty in survey estimates. As a precaution against this, the minimum subgroup size for which percentages and medians could be quoted was set at 30. Figures based on subgroup sizes of less than 50 but equal to or more than 30 have an accompanying footnote to show that these figures should be treated with caution due to the low base size. This is the same approach taken with both the England and Welsh SIES data tables.

Findings are compared according to differences in sex, age, parental experience of Higher Education, area deprivation and whether students live independently or with their parents, carers, or guardians.

Further detail is provided in the Technical Report.

2.2.3. Qualitative interviews and focus groups with students

Fifty-three students participated in either an interview (n=30) or focus group (n=23).

The qualitative research sought to understand more about the relationship between students' financial experiences and their physical and mental wellbeing. It also focused on learning more about the financial experiences of students from under-represented groups (care experienced students, students estranged from their family, student parents, student carers and disabled students).

Sampling and recruitment

Students were recruited from the survey to take part in either an interview or focus group. Towards the end of the survey respondents were asked if they would be willing to be contacted about taking part in follow-up qualitative research. If they did, they were asked to leave contact details. Respondents were informed that they had the choice whether or not to take part when the research team invited them for an interview or focus group.

To ensure that a breadth of experience was represented in the qualitative research, students were purposively sampled from the survey to recruit a diverse range of participants in terms of: level of study; full-time and part-time study; year of study; sources of income; institution; age; sex; ethnicity; accommodation type; and other student groups of interest to the study, namely students who were care experienced, estranged, carers, parents and/or disabled. Qualitative participants were purposively sampled based on these criteria. In addition, the focus groups were sampled for five specific groups of students: care experienced; estranged; parents; postgraduates (Masters and PhD), and those studying in colleges.

Those invited to take part in an interview or focus group were sent an invitation by email with an attached information sheet with an embedded link to the privacy notice. The information sheet was written in plain language and formatted to be screen reader accessible, whilst the privacy notice included information on how any data collected for the study would be used in line with GDPR. Those interested in participating in an interview or focus group were asked to contact the project team by email or phone. A time and date for the interview or focus group was then set up.

Conducting the research

Interview participants were given the choice of a telephone or video interview. All focus groups took place online using MS Teams. The interviews took an average of 50 minutes to complete. The focus groups took an average 96 minutes to complete. At the end of the interviews and focus groups, participants were made aware by email of available support organisations. With the consent of participants, all interviews and focus groups were audio recorded and transcribed for ease of analysis. All participants were given a one-off £30 Love2Shop voucher as a thank you for their time and for sharing their experiences with the study.

Research materials

The interview and focus group participant information sheet and topic guides were developed in consultation with the RAG and agreed with the Scottish Government. The interview guide covered a range of topics, including tuition fees (where applicable); sources of income; experiences and impact of sources of income, including paid work; expenditure; financial management; impact of finances on student experience; and suggested improvements to the provision of financial support for students. The focus groups topic guide covered student views and experiences of current financial support available, different types of expenditure (including student accommodation), and suggested improvements to the provision of financial support for students.

Analysis

The transcribed interview and focus group data were managed and analysed using qualitative analysis software NVivo. Analysis involved several stages. First, the key topics and issues which emerged from the research objectives were identified through familiarisation with transcripts by at least two members of the research team. A draft analytical framework was drawn up and an internal meeting was held to agree the initial coding framework. The analytical framework was then set up in NVivo and piloted with a few transcripts by two researchers and any amendments to the framework made. Each transcript was coded, so that all the data on a particular theme could be viewed together. Through reviewing the coded data, the full range of views were systematically mapped, and the accounts of different participants compared and contrasted. Emergent patterns and explanations for individuals and categories of respondent holding particular views were also explored and tested. All qualitative data was anonymised for analysis and reporting.

2.3. Strengths and limitations of the research

There are a number of strengths and limitations to this research study.

2.3.1. Strengths

The study used a mixed method design. The rapid literature review ensured that both the survey and qualitative research tools were informed by up-to-date evidence on student support in Scotland. Findings from the rapid literature review have also been incorporated into the report to provide context and insight for the reader. Combining a survey with follow-up qualitative research has enabled this study to capture more nuanced and in-depth information on students' experiences of, and attitudes towards, student finance, financial management, and the support available whilst studying. The survey provided high-level data on student income, spend and debt, while the interviews and focus groups captured rich, in-depth data which provide insights into a wide range of views and experiences of further and higher education students in Scotland.

Finally, a strength of the study was its inclusion of students studying FE, HNC, HND, undergraduate and postgraduate courses. While the inclusion of all study levels was technically challenging, this is the first time that a study has attempted to do this. As noted in the literature review, there is a lack of research in Scotland on the financial experiences of students at all levels, but particularly those enrolled in colleges and at a postgraduate level. The study provides insights into the financial experiences of further and higher education students which can be built on in further research.

2.3.2. Limitations

While the census approach to sampling had many strengths, it also had some limitations. Overall, a large number of institutions engaged with the study, though a smaller proportion of colleges participated in the study than universities. As a result, fewer than anticipated FE students took part. This limited the level of sub-group analysis that could be conducted for this group. The four colleges that dropped out after initially opting in to the study reported that it was due to a lack of staff capacity to undertake these tasks. To participate in the study, colleges and universities had to provide staff to draw the sample, with the support of the ScotCen Research Team, as well as send out invitations and reminders to students. If the survey is repeated in the future, the study would benefit from additional time and budget to undertake engagement with colleges, discuss any solutions to staff capacity, ensure that they understand what they are being asked, and try and further improve the numbers agreeing to take part.

Another limitation of the study was the timeframe in which the survey was implemented. Due to the project timescales, the survey was conducted in the first semester at the start of the academic year. This may have affected student engagement in the study and student awareness of their finances, in particular for first year students. The timeframe for the survey may have also impacted college and university engagement in the study as the period following matriculation is a very busy time for institutions.

If the study was to be run again in the future, it would be worth trying the survey element in the second semester. The use of survey incentives for all those completing the survey could also improve survey response rates. While the survey questionnaire was piloted before being launched, and amendments made based on the feedback received, the survey responses indicate that there were inconsistent understanding of some questions. We would recommend that if this study was to be repeated in the future, that time and budget are provided to enable cognitive testing of the Scottish focused questions to ensure students are fully comprehending questions, and further exploration with potential respondents on the overall flow of the survey.