Understanding extremism in Scotland: public sector practitioner perceptions and views

2. Methodology

2.1. Overview

A mixed-methods approach was adopted for this research, incorporating both qualitative and quantitative elements. The qualitative research was undertaken first in order to generate depth of insight into how public sector practitioners understood the topic, and explore differences in this understanding between types of practitioner. The findings from this stage informed the design of the quantitative survey, ensuring that the questionnaire covered key topics identified by public sector practitioners themselves and used language that would be fully understood by all respondents.

Ethics

The Scottish Government carried out an ethics assessment prior to the research commencing, highlighting particular sensitivities that would need to be taken account of during the research. This assessment was further developed with Thinks Insight & Strategy during project inception.

To ensure the identity of participants was protected, the research was designed and conducted in full compliance with EU General Data Protection Regulation (GDPR) legislation. Informed consent was obtained from all participants.

All fieldwork was carried out by members of the Thinks Insight & Strategy project team, who are trained to meet ethical standards and are experienced in conducting research on sensitive topics. Recognising that the sensitivity of the topic might create discomfort or distress for participants in the qualitative research in particular, steps were taken to minimise this, including offering relevant support following participation in the discussion. In view of the fact that safeguarding concerns might potentially arise as a result of the discussions, there was also a comprehensive safeguarding policy in place which informed all interviewers of their safeguarding responsibilities, including guidance on what steps to take in the event that any unreported Prevent concerns were raised.

A full breakdown of the ethical issues that were identified, and the measures put in place to mitigate them, can be found in Appendix B.

2.2. Qualitative research

Overall design

The qualitative research was conducted between 10 May and 23 June 2022. The initial design included in-depth interviews (a detailed, individual discussion designed to elicit depth on a topic of interest) with public sector practitioners with a Prevent-related role – in order to capture the detail of their roles and experience – along with mini focus groups (an organised discussion with a small number of participants) with public sector practitioners without a Prevent-related role. The project team felt that focus groups were more appropriate for practitioners without a Prevent-related role on the assumption that they might feel less confident in their knowledge of the topic and that the interaction and sharing of experience arising within a peer group setting would facilitate discussion. However, due to issues aligning practitioner availability, some focus groups were replaced with in-depth and paired in-depth interviews (meaning that two participants were interviewed at the same time).

Therefore, the qualitative research was ultimately structured in the following way:

• 12 x 45 minute, 1-on-1 in-depth interviews were conducted with public sector practitioners with a designated Prevent aspect to their role.
• 4 x 1 hour mini focus groups (3-4 public sector practitioners), 6 x 45 minute in-depth interviews (1-on-1) and 1 x 45 minute paired in-depth interviews (2 public sector practitioners), were conducted with public sector practitioners without a designated Prevent aspect to their role but who have a statutory obligation to deliver the Prevent duty.
  • The 4 mini focus groups were conducted with public sector practitioners from the health and social care, schools, Further / Higher education and local authority sectors.
  • 4 of the in-depth interviews were conducted with public sector practitioners from the prison sector; while 2 in-depth interviews and the paired in-depth interview were conducted with practitioners from the police sector.

Sample

The research aimed to recruit public sector practitioners working across a range of professions and areas of Scotland. It also sought to ensure there was diversity of coverage across variables such as gender, rural or urban area and job role. A screening questionnaire was used at recruitment stage, and quotas were set for the in-depth interviews and mini focus groups across those variables to ensure this diversity was met.

To identify and recruit participants for the interviews, the Scottish Government contacted public sector practitioners with Prevent-related roles and asked if they were interested in taking part. The contact details, alongside information about job role and area of work, of those who wished to, were shared with Thinks Insight & Strategy with their consent. These contacts were then screened by Thinks Insight & Strategy to ensure a spread of participants were recruited in terms of region and rurality.

To identify and recruit participants for the mini focus groups, existing contacts of the Scottish Government were provided and utilised (with their consent). This involved asking these contacts to advertise the opportunity to participate in the research to their colleagues. This was combined with a free-finding method, which involved recruiting individuals through a recruitment agency (Taylor McKenzie) using a screening questionnaire. This was designed to ensure the recruitment of participants would reach a broad and diverse set of practitioners with a range of views and experiences.

The final sample for the qualitative research component included 34 public sector practitioners, broken down in Table 1 as follows:

Table 1. Breakdown of qualitative research participants, by sector

	Health and social care	Primary / secondary education	Further / Higher education	Local authorities and social work*	Police Scotland	Scottish Prisons Service
Total	6	5	4	7	6	6
Practitioners with Prevent-related role	2	2	1	3	2	2
Practitioners without Prevent-related role	4	3	3	4	4	4

*’Local authorities and social work’ is often referred to as ‘local authority’ in the report for ease.

The spread of public sector practitioners participating in the qualitative research by region is as follows:

• 3 x Aberdeen and North East
• 1 x Highlands and Islands
• 2 x Tayside, Central and Fife
• 9 x Edinburgh and Lothians
• 15 x Glasgow and Strathclyde
• 4 x Scotland South

Topic guide development

Topic guides for the interviews and focus groups were developed in partnership with the Scottish Government. The wording of the questions varied depending on whether the participants were public sector practitioners with or without Prevent-related roles, in order to make the questions relevant. For example, public sector practitioners without a Prevent-related role were asked if they were aware of other roles within their organisation that might deal with Prevent referrals. Public sector practitioners with Prevent-related roles were not asked this.

A broad outline of the topics covered in the guides can be found in Appendix B.

Analysis

A qualitative analysis framework was developed at the start of fieldwork to organise key points from the fieldwork into themes, and draw insight from the interviews by organising notes into a grid structure, filled with verbatim quotes from participants. This analysis framework was used throughout the fieldwork period to identify key themes and insights early on, and to help shape the ongoing qualitative fieldwork and subsequent quantitative fieldwork.

Full analysis of the qualitative framework, alongside quantitative analysis of the survey, shaped the report findings. Quotes from participants in the qualitative research are used throughout the report to illustrate key points.

A key difference of relevance to the research in terms of practitioners’ job roles was the extent to which their role included an additional designated Prevent element. Some had been assigned Prevent-related roles or were responsible for managing and coordinating responses to Prevent concerns, while others had no additional responsibilities beyond fulfilling the Prevent duty. A distinction has therefore been made between these types of role in the analysis and throughout the report, which is explained in more detail in section 2.5.

2.3. Quantitative research

Overall design

An online survey (5-7 minutes in length) was distributed through an open link to public sector practitioners involved in delivering Prevent in Scotland. This was available for public sector practitioners to complete between 8 July and 30 September 2022.

The survey was disseminated through several methods in order to maximise the overall sample size. Firstly, partners of the Scottish Government helped to distribute the link. This included the sector leads on the Prevent Sub Group^[7], the Prevent Single Points of Contact (SPOCs)^[8] and Prevent Multi-Agency Panel (PMAP)^[9] leads in each local authority. Secondly, participants who had already taken part in the qualitative research were also encouraged to engage with the quantitative research. Thirdly, participants were asked to forward on the email to their colleagues to enable the link to reach a wider range of practitioners in their organisation. Finally, contacts in relevant organisations were identified through desk research using publicly available information and asked to participate. Follow-up telephone calls were made to ensure that the invitation had reached the appropriate individual.

Sample

Quotas of at least 50 respondents per sector were put in place prior to the launch of the survey. This threshold was set to allow for indicative significance testing at the analysis stage. These quotas were met, meaning that the sample was large enough to provide a sufficient spread of views and experiences of public sector practitioners working in different sectors and areas of Scotland. Despite this, due to the opt-in approach of the survey, the findings cannot be considered representative of the whole population of public sector practitioners working in the relevant sectors in Scotland.

The final survey sample, by sector and region, is shown in Tables 2 and 3 below.

Table 2. Breakdown of survey sample by sector

Total	Health and social care	Schools	Further / Higher education	Local authorities (and social work)	Police Scotland	Scottish Prisons Service
492	141	62*	31*	99*	74*	85*

Table 3. Breakdown of survey sample by region

Total	Aberdeen & North East	Highlands & Islands	Tayside, Central & Fife	Edinburgh & Lothians	Glasgow & Strathclyde	Scotland South
492	118	20**	145	52*	148	6**

* small base (under 100); ** very small base (under 30) meaning ineligible for indicative significance testing

Questionnaire development

The questionnaire was developed in partnership with the Scottish Government. The survey questions were informed by the qualitative findings, to ensure all questions were relevant to public sector practitioners and reflected the language they used.

The majority of the questions were closed, with answer categories provided. However, two open questions were included to allow public sector practitioners to explain their experience, if applicable, in more detail.

A summary of the broad topics covered in the survey is provided in Appendix B.

Analysis

The survey was scripted and hosted by a third party provider, Yonder Data Solutions. The qualitative findings informed the tabulation specification, which is a set of instructions for setting out the data tables, including outlining the variables that the data will be analysed by (e.g., by job sector, by geographical location, by age). This allowed for analysis and comparison of subgroups. The specification evolved throughout the analysis process, with additional variables requested as the analysis progressed.

Significant differences were reported for all groups with a sample size of greater than 50, however it should be noted that any base size of less than 100 is considered a small sample. See note on limitations below in relation to reporting on statistical significance.

Due to some regions not achieving a base size of 50, regional differences were treated with caution as they could not be compared across all regions of Scotland. Therefore, regional differences have been reported on where the difference appeared relevant and was supported by evidence in the qualitative research. In contrast, as each job sector achieved at least a base size of 50 (once the education sectors are combined), these differences have been reported on with more confidence.

2.4. Limitations of the research

There were several limitations to this research, which are explored in this section.

One limitation was that this research was relatively small scale, in terms of the overall number of participants, when compared to the size and breadth of the police, local authority, health and social care, prison and education sectors in Scotland. While a diverse range of opinions were included, this sample cannot be considered representative of the population of interest (i.e., all employees in each of these sectors who have an obligation to carry out the Prevent duty).

In addition to this, some participants were recruited through existing Scottish Government contacts, so there is a possible risk of bias given that the work was commissioned by the Scottish Government.

A broader limitation is that the research relies on participants’ own recollections, assessments and descriptions of their knowledge, thoughts or experiences, rather than an observation of their behaviour or an objective view of their experiences (e.g., they might make incorrect assumptions about a person they encounter, but the research can only explore what the perception was, rather than the reality).

This means that participants may sometimes overstate or understate their true knowledge, thoughts or experiences, as established by research into cognitive biases and heuristics. Some relevant cognitive biases include the overconfidence effect, whereby people tend to report higher levels of confidence in their knowledge than their objective level of knowledge (Fischhoff, Slovic, Lichtenstein, 1977), the Dunning-Kruger effect, whereby people with the lowest level of knowledge are most likely to overestimate their level of knowledge (Kruger, Dunning, 1999), and social desirability bias, whereby due to societal norms people tend to overreport socially desirable activities and underreport socially undesirable ones (Krumpal, 2011).

To address these cognitive biases and heuristics, this report has referenced where there was a difference between participants’ self-assessments and their other responses, and caveated findings where biases might be relevant. It has also been made clear in the body of the report that findings are based on participants’ perceptions, which may not always be accurate.

There were also some limitations which were specific to the qualitative research. Incentives are often given to qualitative research participants, to compensate them for their time and to encourage participation from individuals who may not otherwise take part, either due to logistical barriers (i.e., not having sufficient time to participate) or perceived lack of benefit (e.g., due to lack of interest in the topic area). For this research, it was decided that incentives would be used because the audience is typically time-poor, and could therefore be difficult to recruit. It was also deemed to be important to ensure that the sample was not solely comprised of individuals who had strong pre-existing views of the topic (and would therefore be willing to take part without an incentive) and whose views may therefore be atypical. This was considered as part of the ethics assessment (see Appendix B).

However, Police Scotland and the Scottish Prisons Service have organisational policies that do not allow staff acceptance of incentives for participation in research. This presented two key challenges for this research. The first of these challenges related to recruitment of practitioners from these sectors, where the inability to provide an incentive meant that interest in participation was initially lower than in the other sectors. Nevertheless, the sample size achieved for these sectors was ultimately similar to the other sectors. Secondly, it is possible those who did take part were particularly motivated by the topic of extremism, and had views they wished to voice in research, as opposed to being motivated to take part because they were being compensated for their time. This higher level of motivation on the topic of extremism may have meant that their views were atypical compared to others in their sector. This limitation is to some extent mitigated by the mixed-methods approach, as a much greater number of public sector practitioners in these sectors gave their input to the research through the survey.

Other limitations specifically impacted the quantitative research. There were some challenges when distributing the survey, for example, email addresses which were out of date, meaning it was not possible to reach all potential participants. In addition, when sending the survey to organisations, it was difficult to ensure that the survey had reached public sector practitioners at all levels of that organisation, since this was dependent on the survey being circulated internally. Furthermore, as the survey was self-selecting, it is possible the views of these practitioners may be atypical compared to others in their sector. Lastly, the timing of the survey – being distributed during the summer – may have impacted the participation of some sectors more than others, such as those in the education sector. For this reason, the survey period was extended through the autumn months to ensure sufficient responses were received across sectors, and additional work was undertaken to raise participation from education practitioners.

Particular regions of Scotland had much greater representation in the survey than others. For example, Tayside, Central and Fife, and Glasgow and Strathclyde were much more strongly represented in comparison with areas such as the Highlands or South Scotland. Although some of this variation reflects population differences between regions, as mentioned above, this limits the ability to make regional comparisons.

Significance testing was used on the data, where sample sizes were sufficient, to report key between-group differences, such as between sectors or between practitioners with different levels of Prevent knowledge. Significance testing allows for a consistent threshold to determine whether or not between-group differences are highlighted and reported on. Triangulating these differences with the qualitative findings creates an additional factor to determine whether the group differences warrant highlighting in this report. However, as representative sampling methods were not used, strictly speaking, statistical significance should not be applied, and has only been used in the analysis of this survey data as an indication of differences that are likely to be of importance. Due to the non-representative nature of the sample, these differences should be treated with caution.

2.5. Methodological notes

Practitioners’ roles in relation to Prevent

A key difference of relevance to the research in terms of practitioners’ job roles was the extent to which their role included an additional designated Prevent element beyond the Prevent duty.

All the public sector practitioners in the qualitative sample held roles which are expected to comply with the Prevent duty. A subset of the sampled practitioners had additional designated Prevent-related responsibilities in their role, such as managing Prevent concerns made by colleagues. These practitioners are referred to as having 'Prevent-related roles' in the report text (12 participants). The other 22 participants in the qualitative research did not have a designated Prevent role beyond complying with the Prevent duty.

In the survey, respondents were asked whether managing or coordinating a response to Prevent concerns was part of their role. In analysis of the survey findings, those who did (29% of participants) were categorised as having ‘Prevent managerial roles’, while those who did not (71% of participants) were categorised as having ‘Prevent non-managerial roles’, meaning they did not have these responsibilities.

In both the qualitative and qualitative research, consistent differences were found between the two groups in respect of their views and experience of Prevent and extremism. Therefore, throughout the report, differences will be reported by job sector, ‘Prevent-related roles’ compared with roles without a designated Prevent element (when referring to the qualitative discussions) and ‘Prevent managerial’ compared with ‘Prevent non-managerial’ roles (when referring to the survey). Additionally, differences are also reported by job sector, experience with Prevent and whether they saw Prevent as part of their role.

Participant priming

Participants in both the qualitative and quantitative research were told at the recruitment stage that the research was about extremism and Prevent. This report refers to ‘initial’ or ‘unprompted' views, which are therefore not entirely unprimed, but were those given before extremism was discussed in more detail and before public sector practitioners were given an explanation of Prevent.