Primary care - public understanding and perceptions survey: analysis report

Annex 2: Technical methods and sample demographics

Recruitment

Set up in 2015, the ScotCen Panel is a part of the NatCen Panel with a specific focus of research with people living in Scotland. The Panel allows the efficient gathering of robust survey data from a nationally representative sample of adults living in Scotland. ScotCen Panel members are recruited through the Scottish Social Attitudes survey, for which participants are selected at random from the general population using the Postcode Address File (PAF) as sample frame. The ScotCen Panel is not based on an opt-in approach, reducing the risk of bias in the sample.

The ScotCen Panel is recruited through face-to-face fieldwork, rather than 'push-to-web' or random digit dialling approaches. This increases the sample quality in several ways, including:

• Increasing the overall response rate (going back to the original issued sample) and therefore reducing the risk of bias
• Allowing low internet users to join, meaning the sample is not biased at the point of recruitment
• A larger proportion of non-response occurs after the participant is recruited, meaning it can be more effectively modelled and adjusted for in weighting;
• Allowing for the proper enforcement of probability-based selection of individuals within the household.

1,136 surveys were completed between the 4^th of February and the 7^th of March 2022 against target of at least 1,000 responses. Invitations to participate were issued to c.2,300 members of the ScotCen Panel. As all eligible panel members were invited to take part and no quotas were used, the population had a known and non-zero chance of being selected, thus utilising a random probability design.

Data weighting and demographics

The ScotCen Panel's fieldwork design helps to ensure a representative sample is recruited. However, as with all quantitative surveys, the risk of bias is never fully eliminated. To further minimise the impact of bias on results, a post-fieldwork weight was implemented.

The use of face-to-face fieldwork for panel recruitment meant that much of the non-response could be effectively modelled using the wealth of data available from the recruitment survey (in this case Scottish Social Attitudes or SSA), accounting for much of the bias that occurs during the panel survey process.

As a random probability sample, estimates are affected by non-coverage and non-response. In order to ensure the sample is representative of the population, a set of non-response weights has been computed. Non-response to one of NatCen's probability panel surveys can occur at any one of three stages: (I) refusal to take part in the recruitment survey, (II) refusal to join the Panel, and (III) refusal to take part in a survey issued to panel members (including through attrition). To account for this, ScotCen produced a final weight that is the product of three separate weights, each of which is designed to adjust for non-response at the three stages described above.

The sample for this survey was recruited from Scottish Social Attitudes Survey 15, 16, 17, and 19, and British Social Attitudes (BSA) Survey 21. All recruitment surveys used weights to adjust for differential selection probabilities (design weights), non-response at household level (non-response weights) and weights to adjust the profile of respondents to match population estimates (calibration weights). You can find more information about the weighting approach for SSA https://www.gov.scot/collections/scottish-social-attitudes-survey/.^[11]

The subsequent two weights were developed using separate logistic regression models to compute the probabilities of response of each participant, with the weight being equal to the inverse of the probabilities of response. The model is based on a number of variables, including: age and sex groups, region, BSA/SSA year, household type, household income, education level, ethnicity, tenure, social class group, economic activity, political party identification, and interest in politics. The weighted and unweighted sample size for key demographics is shown in table 1.

As a nationally representative sample, the ScotCen Panel provides sufficient sample sizes for sub-group analysis shown in Table 2.

Data Analysis

The data collected was cleaned and checked for data quality by a third party and provided to the ScotCen and Scottish Government research teams. Analysis was undertaken by ScotCen to produce a set of data tables, which included a table for each survey question along with ten different crossbreak variables (some demographic and some based on survey data), as well as NET rows (where applicable) and indicators of significance at the 95% confidence level (using a weighted equivalent of a t-test with pooled variance).

In addition, a dataset was provided in Excel and SPSS formats. This dataset included data for all participants who have completed the survey and for all funded questions. In addition, it includes sampling and weighting variables for analysis, design variables (mode and date of completion) and socio-demographic information.

These data outputs were accompanied by a summary slide deck in PowerPoint with graphical representations of the survey questions, which formed the basis of a presentation to the Scottish Government Research Advisory Group (RAG).

Scottish Government analysts then conducted further analysis and interpretation of the data (including additional crossbreaks) to compile this report.

Comparison with Health and Care Experience (HACE) survey methodology

• HACE has been run every two years since 2009 and can provide insight into changes over time. The most recent data was collected earlier than this survey (November 2021-January 2022), asked about experience during the previous 12 months and was published in May 2022.
• The sample size for HACE is much larger, with over 130,000 respondents in the 2021/22 survey.
• HACE is also representative, but uses a different methodology. Respondents are selected from the Community Health Index using probability sampling, whereas the panel used in this survey selected people at random from the general population using the Postcode Address File (PAF).
• HACE samples people who are registered with a Scottish general practice, whereas this survey includes a very small number of people not registered with a general practice.
• HACE includes those aged 17.
• HACE asks primarily about general practice (in and out of hours) and caring experiences, whereas this survey asked questions about general practice and other primary care services such as community pharmacy and dentistry.

How to access background or source data

☒ may be made available on request, subject to consideration of legal and ethical factors. Please contact socialresearch@gov.scot for further information.