Stop and Search code of practice: twelve month review - quantitative report

7. Predicting positive search outcomes

7.1 Introduction

The analysis conducted so far has involved a simple descriptive examination of change in the relationship between positive outcome and protected characteristics (age, sex and ethnicity) since the introduction of the CoP. However, there are a range of other contextual factors that might have impacted on the success of searches, including when, where and why it took place. Therefore, it is important to examine what had the biggest influence on positive detection rates. This was done using regression analysis – a technique that enables the influence of multiple factors on an outcome (in this case a positive search) to be tested simultaneously. A specific aim of this analysis was to establish whether the introduction of the CoP had an effect on the positive detection rate which was over and above that of the other factors that are known to influence detection. The results of this analysis are presented below.

7.2 Factors included in the regression model

A set of potential 'explanatory variables' that were likely to have an influence on the outcome of the search were identified. Within the stop and search database there is a limited number of variables available and it is likely that certain factors influenced detection rates that cannot be accounted for here (this is a limitation of all regression models, and is certainly the case here). Nevertheless, three sets of factors were considered, as detailed below:

1. The protected characteristics of the person who was searched:

• Sex (reference category: Female)
• Age group (reference category: 18-19 years)
• Ethnic group (reference category: White)

2. Factors relating to the nature of the search itself:

• Day of the week (reference category: Saturday)

• Time of day (reference category: 6pm-midnight)

• Reason for search (reference category: Drugs)
• Division (reference category: Greater Glasgow)

3. The time period of the search:

• Before or after the CoP was introduced (reference category: Before the CoP)

The results of a regression model are expressed in terms of odds ratios. An odds ratio greater than one suggests that there are increased odds of a positive search due to the presence of a particular variable, while a value less than one suggests that there are decreased odds of a positive search in the presence of a particular variable. A value of one would suggest that the particular variable has no effect on whether or not the search would result in a positive outcome. As all variables are tested simultaneously, the odds for any one factor are interpreted as being true when all the other variables in the model are held constant at their reference values. Confidence intervals at the 95% level are included in the models to determine significant differences between the odds ratios for all categories compared to their reference category (e.g. differences between each of the age groups with those aged 18-19). In the figures below, odds ratios are represented as a red square and their 95% confidence intervals are represented by vertical black lines. Where differences have been tested across different reference categories (e.g. using an age group other than 18-19), the results will be noted, although the additional data are not presented here.

7.3 Results of the regression model

We start this section by looking at the effect of the protected characteristics on the odds of a successful search, when holding all other factors in the model constant. The results of the full regression model are reported in the Appendix.

7.3.1 Effect of sex

In the six month review of the CoP (McVie 2018), the regression model indicated that the odds of a positive search were not influenced by sex when all other factors were taken into account. However, as noted in sections 6.2.2 and 6.3.2 of this report, the rate of search declined to a greater extent for males than it did for females in the twelve months following the introduction of the CoP, and the percentage change in the likelihood of a positive search increased more for males than females over this period. The net effect of these changes is that the odds of a positive search is now greater for encounters involving men than those involving women. This is illustrated in Figure 7.1. The effect size, although significant, is small (Odds Ratio=1.08) and means that the odds of a positive search were on average 8% higher for men than they were for women.

Figure 7.1: Regression model predicting a positive search by sex, controlling for other factors

7.3.2 Effect of age

As part of the six month review, it was reported that searches involving younger people were significantly less likely to be positive than those of older age groups. In section 6.2.1 of this report, it was shown that even though the number of searches involving young people declined significantly following the introduction of the CoP, the rates of search for people aged 16 to 19 continued to be higher than for any other age group. Furthermore, it was noted in section 6.3.1 that, while positive search rates had increased for all age groups, they were still substantially lower for young people under the age of 18.

Even when controlling for a range of other factors about the search, Figure 7.2 shows that there is a strong and persistent age effect in terms of whether or not a search is successful. Searches involving people aged under 16 had by far the lowest odds of resulting in a positive outcome than those of all other age groups. Indeed, the odds of positive search involving a person under the age of 16 were on average 42% lower than they were for that of a person aged 18-19. Searches of young people aged 16-17 also had a lower odds of being successful that most other age groups, and were on average 24% less likely to be successful than that of a search involving an 18-19 year old. Those aged 20-29 were by far the most likely to have a positive search result compared to other age groups, while searches involving people aged 30 or over were no more likely to be successful than those for 18-19 year olds.

These findings suggest that, even though the number of searches has declined markedly for younger people and success rates have increased, the decision to search younger people (especially those aged under 18) may still apply a lower threshold of reasonable suspicion than for those who are older. It is impossible from the available data, however, to say whether there were other factors about these young people that made them appear to be more 'risky' than people of older ages.

Figure 7.2: Regression model predicting a positive search by age, controlling for other factors

7.3.3 Effect of ethnicity

The six month review found that searches involving people from Other ethnic groups were less likely to result in a positive outcome than those of White people. Section 6.2.3 of this report found that searches had declined to a greater extent and positive detection rates had increased far more amongst all non-White ethnic groups compared to people who self-defined as White, although White people were still far more likely to be searched overall. Looking at the data for the twelve month review, Figure 7.3 shows that there was no significant difference in the odds of a positive detection between searches involving White people and those from Asian backgrounds or from Black, African or Caribbean backgrounds (which represented the largest number of non-White searches overall).

Searches involving people from Mixed or Other ethnic backgrounds did have lower odds of being successful, although it is worth noting that they represented only 2% of all searches. Figure 7.3 suggests that the odds of a search involving someone self-defining as belonging to a Mixed or Other ethnic background was on average 25% less likely to result in a positive outcome than one involving a White person, when taking all other factors into account. It is not possible from the available data to say whether there were other factors involved in these searches that reduced the threshold of suspicion or whether there was some other explanation for this difference.

Figure 7.3: Regression model predicting a positive search by ethnicity, controlling for other factors

7.3.4 Effect of day of the week and time of day

Turning now to factors relating to the search itself, the six month review report noted that there was some difference in the likelihood of a search being positive based on day of the week and time of the day in which it occurred. This was also true of the analysis of the twelve month data, although the findings were slightly different. Figure 7.4 shows that searches conducted on a Monday to Thursday had lower odds of resulting in a positive outcome than those that occurred on Friday to Sunday, although the differences were not extensive.

Figure 7.4: Regression model predicting a positive search by day of the week, controlling for other factors

Furthermore, searches that happened in the afternoon (12-6pm) and evening (6pm to midnight) had greater odds of resulting in a detection than those that occurred in the twelve hours between midnight and midday. This is illustrated in Figure 7.5.

Figure 7.5: Regression model predicting a positive search by time of day, controlling for other factors

Since the vast majority of searches occur between midday and midnight, and at the weekends from Friday to Sunday, these findings suggest that officers are most successful in detecting items during periods that would typically be associated with more demanding periods during which criminal activity or anti-social behaviour may be greater. Whereas at 'quieter' times of the day or on 'less busy' days of the week searches are generally less successful. It is possible that different thresholds of suspicion are applied depending on the capacity of officers to deal with situations based on levels of demand and this may have a resultant impact on productivity (as measured by positive detection).

7.3.5 Effect of reason for search

Compared to searches involving drugs (which represented the highest volume overall), searches for stolen property had higher odds of resulting in a positive detection, as shown in Figure 7.6. Searches conducted under a Warrant (which were also mainly drug searches) as just as likely to be successful as standard drugs searches. However, searches conducted on suspicion of possession of an offensive weapon or for some other reason had far lower odds of resulting in a positive outcome compared to searches for drugs, stolen property or under warrant, even when taking account of other factors that determine success. This suggests that a lower threshold of suspicion may be applied for weapon searches and for searches for other reasons, although it is not possible to know this for certain without having further contextual information about the nature of these searches.

Figure 7.6: Regression model predicting a positive search by reason for search, controlling for other factors

7.3.5 Effect of Police Division

Sections 2.3.2 and 2.3.3 of this report showed that there were considerable differences in search rates and positive detections across the thirteen Police Divisions. In the six month review, it was noted that there were also significant differences in terms of the success rates when controlling for a range of other factors. Replicating this analysis, Figure 7.7 shows the odds of a positive detection were considerably different depending on the Division in which they took place, all other factors considered.

As in the six month review, searches in Lanarkshire and Ayrshire Divisions had by far the greatest odds of a positive outcome compared to all other Divisions. Whereas, Greater Glasgow, the North East and Dumfries & Galloway had the lowest odds of success. The odds of a positive outcome were lower in Greater Glasgow than all other Divisions, except for the North East and Dumfries & Galloway. There was no clear pattern according to Command Areas, which suggests that the reasons for these continued differences in success rate are more likely to be influenced by localised factors (i.e. within Division) relating to policing practice, performance or demand, or to contextual differences in the nature of crime-related problems for which there are no variables in the data.

Figure 7.7: Regression model predicting a positive search by police Division, controlling for other factors

Finally, Figure 7.8 reveals that searches conducted in the twelve month period following the introduction of the Code of Practice had significantly greater odds of resulting in a positive detection than those conducted in the previous year. Indeed, the odds of a positive detection were 27% higher on average after the CoP came into operation. This finding was highly significant, even when controlling for all the other factors that were found to impact on detection rate. In other words, it appears that the introduction of the CoP did have an impact on improving detection rates. The reasons for this could include greater application of the rules of suspicion or more careful use of engagement with individuals prior to proceeding to search (which was a strong focus of the training introduced by Police Scotland prior to the introduction of the CoP).

Figure 7.8: Regression model predicting a positive search by time period, controlling for other factors

7.4 Summary of section 7

Regression analysis was used to examine the impact of three different aspects of searches on the odds of a positive outcome: the protected characteristics of the person who was searched; the factors relating to the search itself; and the time period of the search. A key aim of this analysis was to establish whether the introduction of the CoP had an effect on positive outcomes which was over and above that of other factors that influence detection. The results showed that positive outcomes varied significantly by age, sex and, to some extent, ethnicity even when other factors were taken into account.

The success of searches was also influenced to some extent by the time of day and day of the week when they were conducted, with evidence that searches conducted during busy periods of activity were more likely to be productive than those conducted during less demanding periods. Successful detection was found to be greater in the case of searches conducted for stolen property than for drugs, but searches conducted for offensive weapons or other reasons were far less likely to be successful. Even taking account of these other factors, there were considerable differences in the likelihood of a successful outcome based on the Division in which the search took place. Ayrshire and Lanarkshire stood out as especially successful, with the odds of a search being more than twice as high as those conducted in the least successful Divisions, which were Greater Glasgow, the North East and Dumfries & Galloway. These continued geographical differences are most likely to be due to locally specific factors which may relate to operational policing and/or the nature of the problems faced by the police in these areas.

Finally, searches that were conducted in the twelve months following the introduction of the Code of Practice had on average 27% greater odds of resulting in a positive detection than those in the previous year. This is despite the fact that a range of other factors influencing a positive detection were taken into account. Therefore, there is strong evidence of a real, measurable improvement in the likelihood of a positive search during the period following the introduction of the CoP. However, it cannot be determined from the data available within the NSSU database what specific factors may have led to this improvement.