2015 Scottish Maternity Care Survey Technical Report

8 Analysis and Reporting

Introduction to analysis

8.1 The survey data was collected and coded by Quality Health Ltd. Quality Health Ltd produced the analysis for the hospital reports and produced the hospital and NHS Board reports. Data quality checks were carried out by Quality Health Ltd and Health ASD. Health ASD also undertook the NHS Board and National analyses. The national report was produced by the NMAHP-RU with support from Health ASD.

8.2 Given that the survey is based on only a sample of mothers, the figures included in the report are estimates for the ‘true’ figures that would have been found if we had surveyed every eligible mother in Scotland. Confidence intervals, which are included in most tables of the national report, provide a way of quantifying this sampling uncertainty. A 95% confidence interval means that, over many repeats of a survey under the same conditions, one would expect that the confidence interval would contain the true population value 95 times out of 100.

Number of responses analysed

8.3 The number of responses that have been analysed for each question is often lower than the total number of responses received. This is because not all of the questionnaires that were returned could be included in the calculation of results for every individual question. In each case this was for one of the following reasons:-

• The specific question did not apply to the respondent and so she did not answer it. For example if a woman gave birth at home she would not have completed the section on giving birth at hospital.
• The respondent did not answer the question for another reason (e.g. refused). Women were advised that if they did not want to answer a specific question they should leave it blank.
• The respondent answered that she did not know or could not remember the answer to a particular question.
• The respondent gave an invalid response to the question, for example she ticked more than one box where only one answer could be accepted.

8.4 The number of responses that have been analysed nationally for each of the per cent positive questions is shown in Annex A.

Weighting

8.5 With the exception of the ‘Women who responded to the survey’ section, the percentages presented in the report have been weighted in order to increase the representativeness of the results. Weights were applied to all survey responses based on the number of eligible mothers who gave birth in each hospital (or gave birth at home). This means that the per cent contribution of each hospital to the NHS Board and Scotland results is proportional to the number of eligible mothers that gave birth there.

8.6 Weighting the results in this way provided results more representative of the population (at Scotland, NHS Board or hospital level) than would have been the case if all hospitals (large and small) had been given equal weighting in the calculation of aggregate results.

Percentage positive and negative

8.7 Per cent or percentage positive is a term frequently used in the reporting. This means the percentage of people who answered in a positive way. For example, when people were asked if they were involved enough in decisions about their care, if people said ‘Yes, always’, these have been counted as positive answers. Similarly those women who said they ‘No’ have been counted as negative. Women who said ‘Yes, sometimes’ were counted as partially positive. Annex A details which answers have been classed as positive, partially positive and negative for each question.

8.8 Percentage positive is mainly used to allow easier comparison rather than reporting results on the three or five point scale that patients used to answer most of the questions. There is also a belief that differences between answers on a five point may be subjective. For example there may be little or no difference between a person who “strongly agrees” and one who “agrees” with a statement. In fact some people may never strongly agree or strongly disagree with any statements.

Analysis by Health Board

8.9 A number of mothers gave birth in a different NHS Board from the one in which they live. In such cases, the mothers’ responses for questions relating to hospital care, labour and the birth have been included in the results for the NHS Board in which they gave birth. Responses to other questions, such as those relating to antenatal care and ‘care at home’, have been included in the results for the Health Board in which they live.

Analysis of the free-text comments

8.10 For the first time the comments which women returned in answer to the open questions in the Scottish Maternity Survey have been analysed for inclusion in the main report. These comments were provided in answer to four ‘free text’ questions. The number and spread of the comments is described below:

Survey Question	Number of Comments
If there is anything else you would like to tell us about your antenatal care, please do so here	620 comments
If there is anything else you would like to tell us about your labour and birth, please do so here	755 comments
If there is anything else you would like to tell us about your care in hospital after the birth, please do so here	757 comments
If there is anything else you would like to tell us about your postnatal care, please do so here	535 comments

8.11 In total 1,244 of the returned surveys included ‘free text’ comments, meaning that just over 61% of the women who completed this survey chose to provide comment in response to at least one of the above open questions. A total of 2,667 comments were provided by women, meaning that some survey participants answered more than one ‘free text’ question.

8.12 The spread and content of the comments suggest that women were most motivated to tell us about their experiences in hospital; before, during and after the birth of their babies. However a sizeable number of women did choose to make comment on the care they received in the community both during pregnancy and in the post-natal period. As women chose whether or not to provide comment on their experiences, the sample is necessarily unrepresentative but nevertheless was found to contain a broad range of opinion. We are grateful to the women who shared their views with us in this way, as their comments provide real insight into recent experiences of having a baby in Scotland.

8.13 The overall aims of our qualitative analysis were as follows:

• To listen to and learn from women’s accounts of their experiences
• To better understand the overall ratings of their maternity care provided by women in response to this survey
• To illustrate the main findings from the survey.
• To uncover areas for learning and improvement in Scottish maternity care.

The researchers worked together to fit the statistical and qualitative analysis of the survey data together in order to provide a report on all the responses to the survey. We were also interested in whether the free text comments contained suggestions for the future development of the Scottish Maternity Survey design.

8.14 The full data set of 2,667 comments was read and re-read and initial notes on overarching themes were taken. The complete data set was then divided up by question to form 4 smaller sets of data, one for each open question in the survey. Initial ‘high level’ coding by ‘positive’ and ‘negative’ was completed in order to discover the overall balance of opinion in each section. However it should be recognised that the richness of the ‘free text’ comments returned mean that many included reference to both positive and negative experiences of maternity care.

8.15 Following the ‘high level’ analysis, the content of the comments was then analysed for emergent themes (Miles and Huberman, 1994; Garcia et al, 2004), which were then developed into codes. The data was coded using a process of ‘constant comparison’ (Glaser & Strauss, 1967; Charmaz, 2014). Some major and a number of more minor themes emerged from each section of the data. To provide an example, many comments returned in relation to community antenatal care referred to whether women had experienced ‘continuity’ in their care. Therefore, coding was thematic in nature and was derived from the major concerns expressed within the data itself.

8.16 The entire data set was then coded line by line in detail. For the comments on antenatal care, care in hospital after the birth, and postnatal care in the community, the data was coded in Excel spread sheets. For the comments on labour and birth, the data was coded using the QSR International qualitative data analysis software NVivo (version 10). The use of a software package specifically designed for qualitative analysis allowed for more detailed coding of the comments regarding women’s labours and births (Bazeley, 2007). This was helpful as women often provided the wider context of their birth experiences in order to explain their views, meaning that comments in this section were generally longer and referred to more than one aspect of care received during labour and birth.

8.17 Finally, the comments were grouped by code into word documents and were then used to write a thematic analysis of the data. The analysis drew on previously undertaken, peer-reviewed, research on experiences of maternity care in order to situate the Scottish experience in context, and consider similarities and differences in experience. This qualitative analysis and a selection of the original comments provided by women were included within the main Scottish Maternity Survey report under the ‘what women said’ headings for each section.

8.18 We hope that the inclusion of both a full analysis of the free text data and a selection of women’s comments helps to illustrate the main findings from this survey and brings women’s experience of maternity care to life. We further expect that it may provide ideas for ways in which maternity care can further develop to meet the Scottish Government’s commitment to providing women and babies with the best quality of health care and to ever improving outcomes for Scotland’s children and families.

Quality assurance of the national report

8.19 A small group of Scottish Government policy leads were sent a draft version of the national report for quality assurance. In addition Health ASD carried out quality checks of all figures used in the report.

8.20 A statement on data quality for all of the patient experience surveys is available at www.gov.scot/Resource/0049/00490162.pdf.

Revisions to previous publication

8.21 A copy of our revisions policy is available at www.gov.scot/Resource/0049/00490163.pdf.

8.22 During the analysis of the 2015 data, some small revisions were made to the 2013 results. These revisions were:

• Corrections to the weightings for the “percentage positive” questions and
• Reanalysis of some of the questions in the 2013 local reports to bring them into line with the methodology used in the 2013 Main Report. (questions B18 and F14).

8.23 The change to the data in the 2013 National Report are negligible but are slightly larger for some of questions at a Health Board level. Where figures have been revised these have been marked with an “R” in the 2015 Local Board reports.