An evaluation of Section 1 of the Offensive Behaviour at Football and Threatening Communications (Scotland) Act 2012

2. Methods

2.1 As noted in the introduction, the evaluation had three main elements: survey research with Scottish football supporters; qualitative interviews with key stakeholders; and secondary analysis of existing data sources. Each of these elements is described in more detail below.

Surveys of Scottish football supporters

2.2 Two online surveys of supporters of Scottish football clubs were conducted as part of the study. The first survey was 'live' between 20 August and 20 September 2013 and the second between 22 July and 5 August 2014.

Sampling strategy

2.3 The strategy for sampling supporters of Scottish football clubs relied on the assistance of Supporters Direct Scotland (SDS). SDS has created a network of Scottish football supporters who wish to debate issues related to Scottish football, via its 'Scottish Fans' forums and social media streams. At the outset of both surveys, every supporter in the SDS network received an emailed invitation to take part in the survey by clicking on a hyperlink.

2.4 At the time of the first survey launch, the SDS network consisted of 4,875 supporters. At the time of the second survey launch, the SDS network consisted of approximately 9,000 supporters^[41].

2.5 The use of the SDS network yielded a number of advantages:

• SDS encourages supporters of Scottish football clubs who play at all levels to sign up to its network. This meant that the survey invitation email was sent to supporters of all the 42 SPFL clubs (and responses were subsequently received from supporters of all 42 clubs).
• The SDS network included both season ticket holders and non-season ticket holders, giving access to a wider range of perspectives than would have been possible through club records (had these been available to the research team).

Limitations of the methods

2.6 Given the resources available, an online survey was considered the most appropriate method of data collection. A number of other surveys of football supporters have been conducted using similar approaches, including the Scottish National Football Survey.^{[42] [43]}

2.7 When surveying a specific section of the population, such as supporters of Scottish football clubs, it is difficult to create a sampling frame from which to select a statistically representative sample of that population. For example, there are no centrally-collected administrative datasets listing supporters of Scottish football clubs and it would be impractical to use the Post Office Address file (PAF) to screen for football supporters at randomly selected households.

2.8 The sampling and data collection methods used for both surveys were considered the best available, given resource and other constraints, and allowed for the collection of data from supporters of all 42 SPFL clubs, as described above.

2.9 However, it is important to note that these samples were not a random probability sample of Scottish football supporters in general, as only those who were signed up to the SDS database were sent the survey link. Indeed, some supporters received the link from somewhere other than SDS, meaning that not all survey respondents will have been SDS network members.

2.10 The views of SDS members (and non-members who were forwarded the link to the survey) may be considered a reasonable proxy for those of Scottish football supporters in general, although the characteristics of SDS subscribers are likely to be distinctive in a number of important respects. For example, those signed up to SDS will necessarily have access to the internet and are likely to have a reasonable degree of ICT literacy.^[44] Because of the broader character and patterning of internet access and use, it is likely that the profile of SDS subscribers may be younger and slightly more affluent than that of Scottish football supporters as a whole. It is also likely that SDS subscribers will have a greater interest in following and engaging in debate around issues affecting football clubs and football supporters (given the site's establishment of a 'fans forum' and use of social media to keep subscribers up to date with developments in the game). These characteristics are also likely to apply to non-SDS members who were forwarded the survey link by other means. Conversely, some highly-engaged supporters deliberately distance themselves from the SDS network, seeing (rightly or wrongly) its part-funding by Scottish Government as compromising its independence. In these cases the evaluation has drawn on more qualitative approaches to capture the perspectives of these groups.

2.11 When conducting analysis based on comparing survey responses from supporters of different clubs, we do not claim that the views of those surveyed represent the views of the entirety of the fan bases of those clubs. However, we feel that differences and similarities in views between the supporters of certain clubs is worth exploring, particularly where a reasonably high number of supporters of a club responded to the survey.

2.12 Despite these caveats, the sampling strategy used has resulted in survey responses being received from supporters of all 42 SPFL clubs and an almost even split of season ticket holders and non-season ticket holders, in both surveys. As such, and while acknowledging its limitations, it provides a workable basis for hypothesising about the views and experiences of Scottish football fans in general.

Data analysis techniques

2.13 Simple frequency tables have been used to show the prevalence of supporters that have witnessed particular behaviours, had particular experiences and hold certain opinions. Crosstabulation has been used to explore whether witnessing these behaviours, having these experiences and holding these opinions has an association with belonging to certain social and football related demographic groups.

2.14 The analysis in this report is based on the answers of participants who completed the survey in its entirety. There were 1,945 such responses to the first survey and 2,185 responses to the second survey. Not all participants were asked every question because participants were routed on the basis of their answers to certain questions. 'Don't know' responses to particular questions are excluded from the results referred to in this report. Although it is inappropriate to provide response rates, as it is impossible to say how many people had access to the survey, the size of the responses suggest a high level of interest and engagement.

Sample characteristics

2.15 The vast majority of those who completed both surveys were male (93% - 2014 survey), gave their ethnicity as white Scottish (89%) and were born in Scotland (91%). These figures are in line with other surveys of Scottish football supporters (for example, 95% of participants in the Scottish FA's 2012-13 National Football Survey were male)^[45]. It is doubtless the case that women, those born outwith Scotland and supporters from minority ethnic backgrounds are under-represented. There was greater diversity within the sample in terms of age, as illustrated in Table 2.1.

Table 2.1 Age of survey participants 2014

Age	%
Up to 29	17 (-2)
30-49	43 (-)
50-64	33 (+1)
65 +	7 (-)
Base	2185

N.B. Figures in brackets show percentage point change from 2013 survey
N.B. Twenty two under 16 year olds participated in the survey.
N.B. Figures do not always add up to 100% because of rounding

2.16 The Act is particularly concerned with behaviour that is 'likely' to cause public disorder by expressing hatred based on religion and as such the religious background of survey participants is of interest. The religious background of participants (Table 2.2) appears to be broadly in line with that of the Scottish population as a whole: in the Scottish Census of 2011, 37% of people described themselves as having no religion, 32% as belonging to the Church of Scotland, 16% as being Roman Catholic, six per cent as belonging to another type of Christian religion and three per cent as belonging to another religion, while seven per cent did not give a response to the census religion question^[46].

Table 2.2 Religion of survey participants 2014

Religion	%
None	49 (-)
Church of Scotland	26 (-3)
Roman Catholic	16 (+2)
Other Christian	4 (-)
Other religion	1 (-)
Prefer not to say	3 (-)
Base	2185

N.B. Figures in brackets show percentage point change from 2013

2.17 Although survey responses were received from all 42 SPFL clubs, as expected, those from Scotland's two largest cities, Glasgow and Edinburgh, comprised four of the five most-represented clubs. However, distribution across these clubs was not necessarily in line with the size of their respective fan bases. In particular, it is notable that supporters of Hearts were almost as numerous in the achieved sample as those of Rangers, while Celtic supporters were considerably more numerous than Rangers supporters. Of those clubs where more than 100 supporters responded to the survey, six out of the seven played in the Scottish Premiership during the 2013/14 season. Somewhat unexpectedly, St Mirren was the fifth best represented club in the 2014 survey, showing a substantial increase on the 2013 survey. The reason for this increase is not clear.

Table 2.3 Club supported 2014

Club supported	%
Celtic	19 (+3)
Rangers	11 (+2)
Hearts	11 (-4)
Hibernian	9 (+3)
St Mirren	9 (+5)
Aberdeen	6 (-1)
Motherwell	5 (+2)
Other teams	31 (-9)
Base	2185

N.B. Individual clubs listed where responses received from over 100 club supporters
N.B. The names of the four leagues which form the SPFL changed at the beginning of the 2013-14 season. As this report focuses on supporters' experiences of attending matches played in the 2012-13 season, the names of the divisions as they were known in the 2012-13 season are referred to in this report.
N.B. Figures in brackets show percentage point change from 2013 survey

2.18 There was a fairly even split between those who were season ticket holders for all or part of the 2013-14 season (60%) and those who were not (40%). Of those who were season ticket holders, 67% had held a season ticket for over 10 years. There was variation in the number of home games attended by supporters, over half (53%) attending 16 or more games and over one-fifths (22%) having attended five or less home games. Unsurprisingly, attendance at away games was less frequent, with 60% of supporters attending five away games or less, and 11% attending 16 or more away games.

2.19 Other questions sought to identify different 'types' of supporters, including which people attended home games with, whether they usually sat in an area of the stadium reserved for families and how that area of the stadium compared to others in terms of noise level. In 2013-14, only a small minority of respondents (11%) watched home games from a section reserved for families. Forty-five per cent of supporters watched home games from a section they felt had about the same noise level as other parts of the stadium. The remaining supporters were split almost evenly between those who sat in quieter (27%) and louder (28%) parts of the stadium

2.20 Supporters were most likely to report attending home games with adult males (70% doing so), with one-fifth (20%) attending with adult females. 18% of supporters reported attending home games with males aged under 16 years old; only six per cent reported sometimes attending games with females in the same age group.

Qualitative research

2.21 The qualitative research focused on a number of inter-related questions:

• To what extent are fans and stakeholders aware of the provisions of the new Act, and to what extent do they understand and support these provisions?
• To what extent is the new Act working, both in terms of being supported by appropriate club and criminal justice practices, and in terms of the legislative provisions adding value in terms of securing appropriate convictions and restrictions?
• To what extent is the new Act perceived to be impacting on fan behaviour?

Sample coverage

2.22 The research involved an extensive series of in-depth interviews and focus groups (1-3 hours in length) with key stakeholders involved in the policing or conduct of Scottish football, as well as with fan groups themselves, and with journalists responsible for providing coverage of football and football-related issues. Key stakeholders were - in part - identified through early consultation with the Scottish Government, Supporters Direct Scotland, and Police Scotland.

2.23 In conducting focus groups with groups of fans we sought to elicit their understanding and their perceptions of the Act, but also to gauge their perceptions of change in behaviour and atmosphere since the introduction of the Act. In addition their perceptions of how the Act has been practically enforced at different grounds and fixtures was also a concern. Finally, in selecting focus groups we looked to supplement data being collected via the survey, in particular reaching fans who were considered under-represented on the SDS mailing list (in particular younger supporters, and supporters who might be self-styled 'ultras' or members of 'singing sections'). In selecting fan groups to look at we developed a 'measurement footprint' to help guide our choices. This was partially informed by selecting key clubs and rivalries that were associated with past issues of disorder or offensiveness, while also ensuring that clubs in at least the top three flights of the Scottish leagues were represented.^[47] We also used existing research on fan typologies (in particular Crabbe et. al, 2006) to inform our recruitment of participants to help ensure that we heard from the different types of fans. Finally, in addition to formal interviews and focus groups we also held one meeting with SPFL officials.

2.24 The following interviews and focus groups were undertaken:

Police Scotland

Interviews with seven members of FoCUS

Interviews with seven football intelligence officers (including Rangers and Celtic)

Interviews with four match commanders

Other criminal justice personnel

Interviews with three procurators fiscal

Interviews with eight Sheriffs (with follow-up communications with three of these Sheriffs)

Interview with a defence solicitor

Clubs and league officials

Interviews with three club security officers

Interview with one stand safety manager

Interview with one fan liaison officer

Fans and fan groups

Meetings with representatives of various supporter's organisations, and representatives from the campaigning group, Fans Against Criminalisation (FAC)^[48]

Focus group with two groups of Rangers fans

Focus group with two groups of Celtic fans

Focus groups with four other fan groups (three single-team affiliated, one mixed)

Journalists

The research aimed to interview a number of key journalists, both sports journalists, but also those home affairs journalists who had covered the issue from the perspective of sectarianism. Eleven journalists in total were approached, six of whom replied and expressed some interest. However, ultimately, no formal interviews were conducted.

Licensed premises and match day observations

2.25 In addition to formal interviews and focus groups a more informal set of interviews with bar and pub staff was undertaken at two locations in Glasgow, covering fourteen pubs in total. The intention was to gauge staff perceptions on changes in fan behaviour in and around the premises, as well as to gauge their own understanding of the Act, and the policies they had in the premises for dealing with potentially offensive behaviour. These interviews were complemented with a limited exercise in match day observations, with two fieldworkers attending matches at these two locations, observing atmosphere and behaviour on the way to stadia, in pubs and bars either side of the fixture, and in the stadia itself during the match. This exercise was too limited to provide data on changes in match day behaviour in and of itself, but was intended to help inform and triangulate data collected from other sources.

Limitations of the qualitative research

2.26 Though ultimately the research achieved a high level of fan input and representation, there were a variety of access issues in relation to the qualitative element of the research as a whole:

• Some fan groups were suspicious of the evaluation because it had been commissioned by the Scottish Government, although over time we did make inroads with the majority of these groups.
• Recruiting fans more generally for focus groups, particularly fans who were not followers of Celtic and Rangers, proved problematic and time consuming. The principal reason for this did not appear to be any principled reluctance, but rather simply that the Act was less of a pressing concern for other fans.
• Likewise, although interviews were conducted with all the Football Liaison Prosecutors and a number of sheriffs, others declined to come forward - again, less because of any principled reluctance to be involved, and more simply because they had not dealt with any OBFTC charges.
• Though we accessed a reasonable number of fans in the 18-21 age range through focus groups, we did not attempt to systematically access individuals under the age of 18. We cannot therefore assess the extent to which fans under this age may have a distinct view, or a unique set of experiences, when compared to the slightly older fans accessed in our focus groups. This relates to a wider ambiguity through this research in relation to what age categories were being referred to when respondent's talked about 'young supporters.' In many instances it appeared that individuals in the 'late-teens' to 'early twenties' age-bracket were being alluded to, though in some instances cases were discussed which involved individuals under the age of 16.
• The most problematic group to access was journalists, none of whom ultimately participated in the research. This may in part be because after the introduction of the Act, and in the absence of any Rangers-Celtic fixtures, that the issues involved were no longer of immediate interest. Nevertheless, given the grievances that many fans and indeed officials had with journalistic representations of fans and fan behaviour (and in their view, frequent distortions and exaggerations), and given that some journalists got directly involved in submitting evidence to Parliament in the run-up to the Act, this reticence is regrettable.

Administrative data

2.27 In addition to the main survey and qualitative fieldwork we also examined secondary data sources to gain a perspective on the impact of the Act in terms of trends in recorded incidents of crime and charges for relevant offences categories. Of equal importance, the secondary data allowed us to more systematically scrutinise how the Act was being enforced in terms of who was arrested, for what, where, and on what charge, as well as subsequent court disposals and sentences. The main data published and unpublished sources for this work were as follows:

• Statistics published by the Scottish Government detailing trends and patterns in hate crime offences (notably s. 74 offences, i.e. offences aggravated by religious aggravation as defined by the Criminal Justice (Scotland) Act 2003) and in those charges made specifically under s. 1 of the Act. These publications were predominantly based on data provided the Crown Office and Procurator Fiscal Service.
• Access to, and re-analysis of, anonymized case data held by the Scottish Government, based on records by the Crown Office. This data covered both s. 1 and s. 74 charges.
• Published national statistics on recorded crime in Scotland.
• A re-analysis of charge data collected by the Football Banning Order Authority, a unit now located within FoCUS, though previously hosted by legacy Strathclyde police.
• 2.28 None of these data sources was comprehensive or indeed free from methodological weaknesses, though when combined with each other, and with other data sources collected as part of this evaluation, they provide a very useful additional source of evidence. The strengths and weaknesses of each data source are discussed during the course of the report.

The Evaluation and the Review

2.29 The Scottish Government's commitment to review the Act may include reference to material other than this evaluation. This evaluation is however intended to provide material that may contribute to that review alongside other evidence, perspectives or material that the Scottish Government may choose to draw on. With that in mind, it may be helpful to summarise briefly what the evaluation can, and cannot, tell us about the impact of s. 1 of the Act thus far:

• The timing of the evaluation, which took place relatively shortly after the introduction of the Act, has some bearing on what can be realistically concluded about the Act's reception and impact. There has been a limited amount of time for case law to fully explore and test the legal provisions contained in s. 1. The limited evaluation period also constrains the extent to which we can draw definitive conclusions about trends in football-related crime and disorder.
• The evaluation has been based on the best available data sources, but there are other potentially relevant data that we were not in a position to access. For instance, a recent decline in the s. 1 conviction rate raises questions that could not be resolved through this research. Whether this decline is due to a change in the characteristics of cases being prosecuted, or due to issues of insufficient evidence, or due to other factors associated with the legal arguments made in courts, remains unclear at this stage.
• In assessing the impact of the Act other work could also be done to understand emerging issues - for instance, the experience of people arrested under the Act (especially in relation to the precise circumstances of arrest) and how the implementation of the Act interacts with evolving strategies for policing and stewarding football.
• A more strategic question that remains to be addressed is how changes to the policing and stewarding of football following the introduction of the Act relate to broader considerations of fandom and the commercial prerogatives of the Scottish game.

2.30 In summary, this evaluation was not intended or able to arrive at definitive conclusions as to the overall success or failure of s. 1 of the Act to date. What it can hopefully offer is robust evidence on patterns of implementation, perceptions on impact, and emerging issues and questions in relation to s.1 related practices and interpretation.