Student Finance and Wellbeing Study (SFWS) Scotland 2023-2024: technical report

7. Conducting analysis and reporting

In conducting analysis of SFWS survey data for the report, all percentages, medians and means quoted are based on weighted data. The weighting controlled for the effects of sampling and allowed adjustments to be made to the profile of responses to ensure that they more closely match eligible student population estimates, ensuring that that the responses to the survey are representative of students from the three different study levels: FE, HN/undergraduate and postgraduate.

The unweighted number of cases on which figures are based is also included.

In line with the SIES for England and Wales, thirty was taken as the minimum subgroup size for which percentages and means could be quoted. Figures based on subgroup sizes of less than 50 but equal or more than 30 are shown in brackets.

All cross-tabulation, description of medians and significance testing for the report was carried out using SPSS software. Commands from the Complex Samples procedures were used, to ensure the correct treatment of complex sampling weights in these procedures.

Statistical significance

The findings presented in the report which relate to differences in percentages between student groups are statistically significant at the 95% confidence level and where the findings are marginally significant, at the 90% confidence level, this is captured in footnotes linked to the finding being reported.

Making comparisons with the England and Wales SIES

The methodology for the 2023-2024 Scottish SFWS survey was the same as for the 2021-2022 England and Wales SIES survey, as far as possible.

However, the SFWS was expanded to include both further education and postgraduate students, as well as including HE students (HNC/D and other undergraduate students). There are several caveats that should be borne in mind if seeking to explore any comparisons with SIES data for 2021-2022 for England and Wales.

Questionnaire differences

The questionnaire for the survey element of the research was based on the questionnaire used in the England and Wales SIES survey in 2021-2022. Where possible wording for the questions remained the same for both surveys. However, differences between the student funding landscape in Scotland and that in England and Wales, means that the questionnaire had to be amended to reflect these differences and some new questions added. The full SFWS questionnaire is available in the Appendices to this technical report. Questions that were amended from the England and Wales SIES or new questions added in are highlighted in this document.

Before seeking to explore any comparisons between SIES 2021-2022 and SFWS 2023-2024, close attention should be given to the different wording used and where the data was collected in different ways between the two surveys, as well as differences in the policy context on student support/ funding.

Tuition fees

In the 2021-2022 England and Wales SIES report, expenditure was presented minus tuition fees, and income was presented excluding tuition fee loans, whereas the data tables presented both data including and excluding tuition fees and including and excluding tuition fee loans.

As tuition fees are utilised less frequently in Scotland, due to the majority of students being eligible for free tuition fees, a different approach has been taken for the Scottish SFWS. For the Scottish report, the fees and use of tuition fee loans are reported in a set of separate tables and are reported in the Fees chapter of the report, rather than being included in either the income or expenditure data tables or reporting sections. This means that care needs to be taken if comparing data from SIES 2021-2022 and SFWS 2023-2024. When comparing both sets of data, the SIES data excluding fees and tuition fee loans must be used.

Use of medians

In 2021-2022 the England and Wales SIES report moved to reporting median averages, rather than means, as these are less likely to be skewed by outliers and so give a more reliable indication of the ‘typical’ value when reporting on financial data. Means were still presented in the tables. SFWS reporting has also adopted this approach for consistency between SIES and SFWS.

All reported medians have been calculated using the ‘CTABLES’ function in SPSS. Different SPSS functions may produce slightly different median amounts. This is due to different functions employing different default methods for calculating percentiles. CTABLES uses the ‘aempirical’, whereas the frequencies command uses Waverage (a weighted average).

The study has not sought to assess whether the differences between median amounts of income, spend and debt between the different student groups in the study are statistically significant. It has instead, as set out in the study’s aims, focused on gathering up to date information on student’s financial experiences and understanding the impact of these on student’s experiences and wellbeing. Potential research into the survey data in the future could explore the feasibility and usefulness of any further work on the differences between medians.