Reconviction rates - offender cohort: 2017 to 2018

Annex B – Sources of information, data quality and confidentiality, and revisions

Sources of information

B1 Information presented in this bulletin is based on data held in the Scottish Offenders Index (SOI), which is derived from the data used in the Criminal Proceedings in Scotland statistical bulletin. The Criminal Proceedings data is in turn derived from information held on the Criminal History System (CHS) which is owned by Police Scotland.

B2 The SOI currently contains a record of criminal proceedings against individuals (excluding companies) in Scottish courts as well as information on non-court disposals. The court convictions and non-court disposals are held in separate datasets by the Scottish Government and so are independent of each other.

B3 The data in the SOI on court convictions currently covers all convictions where a sentence was imposed since the beginning of 1989, and the main offence involved was either a crime in Groups 1-5, or some of the offences in Group 6, in the Scottish Government’s classification of crimes (see Annex D of the Criminal Proceedings Bulletin for further information about these classification groups). Minor offences (such as drunkenness, and almost all motor vehicle offences) are not included in the SOI.

B4 The distinction between crimes and offences is made only for statistical reporting purposes. Although the violations allocated under “crimes” tend to be more serious, there are some “offences” that have more severe punishments associated with them e.g. drink driving is classified under “offences” in the “motor vehicle offences” section rather than under “crimes”. Groups 1-5 of the Scottish Government’s classification covers non-sexual crimes of violence, sexual crimes, crimes of dishonesty, fire-raising, vandalism etc. and other crimes. The offences in Group 6 which are included in the court conviction SOI are: common assault, breach of the peace, threatening or abusive behaviour, offence of stalking, offensive behaviour at football and threatening communications (under the Offensive Behaviour at Football and Threatening Communication Scotland Act 2012), racially aggravated harassment or conduct, miscellaneous firearms offences, and social security offences. See Annex Table A2 for a more detailed list of the types of crimes in the SOI.

B5 The data in the SOI on non-court disposals covers all crimes and offences, including motor offences, where a non-court disposal was given, back to 2008.

B6 Each record on the SOI database includes information on the sex and date of birth of the offender, the dates of conviction and sentence, the main crime or offence involved and details of the sentence imposed. Information is also available on any crimes which were additional to the main crime. Each offender has a unique reference number, which allows individual convictions for that offender to be linked together. The SOI does not include the name and address of an offender, except the first half of an offender’s postcode. A privacy notice is available on the Scottish Government’s website.

B7 All but the most serious offences alleged to have been committed by children under the age of 16 are generally dealt with outside of the courts in the children’s hearings system, or by Early and Effective Interventions or other non-court disposals. The SOI does not hold data taken from the children’s hearings system.

B8 The SOI currently contains data on 595,700 offenders and 2,251,300 convictions since SOI records began in 1989. Eighty-five percent of the offenders are male and 15 percent are female.

Data quality

B9 The figures in the bulletin have been derived from administrative IT systems which, as with any large scale recording system, are subject to possible errors with data entry and processing. During the production of this bulletin we have put in place processes to ensure that the data are fit for purpose for this publication, which are listed below.

B10 Data standards are adhered to by organisations inputting data to the CHS in terms of the definitions of data items and their corresponding values. These standards are agreed under the Justice Digital Strategy and ensure there is consistency across the justice organisations in the information they collect. Further information on the data standards can be found in the Integration of Scottish Criminal Justice Information Systems (ISCJIS) data sharing manual.

B11 The following protocols also ensure consistency in the data collected:

• The Scottish Courts and Tribunals Service (SCTS) protocol for the handling of errors that may occur in the transmission of data between justice partners’ databases;
• The protocol for the investigation/resolution of disputed data between Police Scotland and the Crown Office and Procurator Fiscal Service (COPFS);
• The protocol for sharing electronic information between justice partners.

B12 The Scottish Government has representation on a data quality group and is kept informed of any data quality issues relating to the CHS. This group meets around three times a year and also has representation from Police Scotland, COPFS, the Scottish Prison Service, and SCTS.

B13 The analyses in this bulletin are based on the data published in the Criminal Proceedings in Scotland statistical bulletin, and the quality assurance process is described in the section on Data quality: Data validation during production of the statistical bulletin in Annex B of the Criminal Proceedings bulletin. In summary, the validation processes include automated and manual checks on the data. Any unusual or missing values are referred back to either Police Scotland, the SCTS, or COPFS. The figures are also checked against case processing information published by COPFS and management information provided by SCTS to ensure that the court volumes are consistent. Police Scotland, SCTS, COPFS and policy experts within the Scottish Government are also consulted to give insight on an operational level and provide insight into why any significant changes may have occurred. The figures are also checked by Scottish Government statisticians, who have not been involved in the production process, and they may highlight any issues that may have gone unnoticed.

B14 During the production of this bulletin, the data undergoes processing to calculate the frequency and prevalence of reconvictions. The numbers are manually checked to determine whether there are any unusual values, and if so, then the calculations are rechecked. Like the Criminal Proceedings in Scotland publication, the figures in this bulletin have undergone a further round of checks by Scottish Government statisticians, and policy experts within the Scottish Government are again consulted to provide insight and context to any significant changes in the figures.

Postcode data quality

B15 From the 2015-16 cohort publication onwards, we have published reconviction rates for local authorities based on offenders’ home postcode (Table 13) alongside the reconviction rates for local authorities based on court location (Table 12).

B16 Previously we could only calculate reconviction rates for local authorities based on the court location of where offenders were convicted, as this was the only geographical information that we had. This approach has limitations, because some court areas cover multiple local authorities and offenders may be convicted for crimes committed in different areas to where they live. In particular, these figures are not fully suitable for local authority community justice partners, as they have a need for reconviction figures for offenders who live in their area for planning purposes, such as: schemes to reduce reoffending, or estimating the number of offenders that social workers need to supervise in their area.

B17 We started receiving data from Police Scotland on the first half of an offender’s postcode in 2014, and coverage was high enough from 2015-16 onwards to start using the data. With the first half of an offender’s postcode we can match offenders to their home local authority and calculate reconviction rates based on offenders’ home local authority.

B18 We are classifying the reconviction rates for local authorities based on the postcode data as Experimental Statistics: Data being developed, as the data are of insufficient quality to be labelled as National Statistics as the rest of the data in this publication are. We did not have postcodes for a fifth of offenders (20%) with an index conviction in 2017-18, and we are unsure of how this may affect the reconviction rates.

B19 Postcodes may be missing because offenders have no fixed abode, but it may also be a recording issue. We are also less likely to have the postcode of offenders released from a custodial sentence, as we would have received their conviction data before we started collecting the postcode. The missing category also includes a small number where postcodes were supplied, but they were not valid postcodes. We are working with Police Scotland to improve the coverage, and as the accuracy improves we will remove the experimental label.

B20 Annex Table B1 shows the percentage of offenders with missing postcodes in each local authority group based on location of the court they were convicted in. It also shows the percentage of offenders living in each local authority that make up the local authority group. This shows that there are significant percentages of offenders who are convicted in a court that covers a different local authority to where they live.

Table B1 Postcode coverage for each local authority group

Local authority group based on court location1	Offender's local authority2	Number	% of LA group
Aberdeen City and Aberdeenshire	Aberdeen City	1,708	56%
	Aberdeenshire	954	31%
	other LA	243	8%
	Missing	131	4%
Angus	Angus	476	78%
	other LA	78	13%
	Missing	55	9%
Argyll & Bute	Argyll & Bute	228	63%
	other LA	66	18%
	Missing	67	19%
Ayrshire, East, North and South	East Ayrshire	671	22%
	North Ayrshire	615	20%
	South Ayrshire	419	13%
	other LA	215	7%
	Missing	1,196	38%
Clackmannanshire	Clackmannanshire	249	70%
	other LA	58	16%
	Missing	47	13%
Dumfries & Galloway	Dumfries & Galloway	909	78%
	other LA	193	16%
	Missing	69	6%
Dunbartonshire, East and West	East Dunbartonshire	29	3%
	West Dunbartonshire	400	48%
	other LA	208	25%
	Missing	204	24%
Dundee City	Dundee City	1,002	63%
	Fife	196	12%
	other LA	188	12%
	Missing	208	13%
Edinburgh, East Lothian, and Midlothian	East Lothian	355	9%
	Edinburgh, City of	1,912	49%
	Midlothian	391	10%
	other LA	437	11%
	Missing	818	21%
Falkirk	Falkirk	756	70%
	other LA	172	16%
	Missing	153	14%
Fife	Fife	1,702	79%
	other LA	141	7%
	Missing	323	15%
Glasgow City	Glasgow City	3,628	53%
	East Dunbartonshire	207	3%
	North Lanarkshire	160	2%
	South Lanarkshire	443	6%
	other LA	726	11%
	Missing	1,709	25%
Highland	Highland	1,044	76%
	other LA	175	13%
	Missing	158	11%
Inverclyde	Inverclyde	296	47%
	Argyll & Bute	31	5%
	other LA	72	11%
	Missing	230	37%
Lanarkshire, North and South	North Lanarkshire	1,564	39%
	South Lanarkshire	1,102	28%
	other LA	425	11%
	Missing	913	23%
Moray	Moray	502	86%
	other LA	64	11%
	Missing	18	3%
Na h-Eileanan Siar	Na h-Eileanan Siar	69	78%
	other LA	10	11%
	Missing	9	10%
Orkney Islands	Orkney Islands	58	73%
	other LA	16	20%
	Missing	5	6%
Perth & Kinross	Perth & Kinross	515	68%
	other LA	168	22%
	Missing	75	10%
Renfrewshire and East Renfrewshire	East Renfrewshire	124	9%
	Renfrewshire	571	43%
	other LA	224	17%
	Missing	401	30%
Scottish Borders	Scottish Borders	366	76%
	other LA	46	10%
	Missing	67	14%
Shetland Islands	Shetland Islands	110	75%
	other LA	22	15%
	Missing	14	10%
Stirling	Stirling	300	58%
	other LA	160	31%
	Missing	61	12%
West Lothian	West Lothian	694	70%
	other LA	148	15%
	Missing	156	16%
High Court	Postcode present	397	66%
	Missing	205	34%

1. Local authority group is based on the court location of where an

offender was convicted.

2. The offender's local authority is based on their postcode. Only authorities

in the local authority group are shown. Those that live in other local

authorities are grouped under "other LA". Missing is the percentage of

offenders convicted in the local authority with a missing postcode.

Domestic abuse data quality

B21 The domestic abuse identifier used to identify domestic abuse and non-domestic abuse crimes in these statistics has been used in the Criminal Proceedings in Scotland National Statistics publication for several years. Also the process to calculate reconvictions is the same as used to calculate reconvictions elsewhere in the bulletin. This means that relevant quality assurance has been carried out on all the data used in these figures. However, as these data have only recently started to be used in combination in these statistics, there may be some minor data quality issues that we may not have identified yet.

B22 A new statutory domestic abuse aggravation was created by the Abusive Behaviour and Sexual Harm (Scotland) Act 2016 and came into effect on the 24th April 2017 for crimes that took place on or after this date. Many of the domestic abuse crimes that were convicted in court in 2017-18 are likely to have been committed prior to 24th April 2017, so there will be fewer domestic abuse index convictions and reconvictions in 2017-18 compared to the identifier. As this statutory aggravation is not currently applied to a conviction without the non-statutory identifier, this will not affect comparability before and after the introduction of the statutory aggravation.

Data confidentiality

B23 We have completed a data protection impact assessment to assess and mitigate any privacy issues with the collection, use, and publication of reconviction data.

B24 A privacy notice is available on the Scottish Government’s website covering the data used in this publication.

B25 The Scottish Government are legally able to collect the data for this publication as Section 84(1 & 2) of the Police and Fire Reform (Scotland) Act 2012 states that the “Scottish Police Authority must provide Scottish Ministers with reports, statistics or other information relating to the Police Service or functions, and the state of crime”.

B26 Court proceedings are held in public and may be reported on by the media, unless the court orders otherwise, for example where children are involved. While our aim is for the statistics in this bulletin to be sufficiently detailed to allow a high level of practical utility, care has been taken to ensure that it is not possible to identify an individual and obtain any private information relating to them.

B27 We have assessed the risk of individuals being identified in the tables in this bulletin and have established that no private information can be identified. Where demographic information is provided this is done in wider categories.

B28 Some of the additional data tables we provide alongside this publication have local authority information related to the offender. In the local authority tables, either demographic information is provided or offence-level information is provided, but not a combination of both. The local authority is based on the court of the conviction.

B29 To maintain the security and confidentiality of the data received from the data suppliers, only a small number of Scottish Government statisticians and statistical support officers have access to the datasets outlined in the various stages of processing outlined above. The only personal details received by the Scottish Government in the data extract are those which are essential for the analyses in this bulletin and do not include the names of offenders.

Revisions

B30 The CHS is not designed for statistical purposes. It is dependent on receiving timely information from the SCTS, COPFS, and the police. It should also be noted that some types of outcome are removed from the system after a prescribed length of time. A pending case on the CHS is updated in a timely manner, but there are occasions when a slight delay may happen. Recording delays of this sort generally affect high court disposals relatively more than those for other types of court. The figures provided in this bulletin reflect the details of court proceedings recorded on the CHS and supplied to the Scottish Government by the end of November 2019 to allow later reconvictions in 2018-19 to be captured on the CHS.

B31 The CHS is regularly updated, so subsequent analyses will result in revised figures (shown in Annex Table B2) as late records are added. The first revision of the reconviction rate in the following year’s bulletin was typically around 0.4 percentage points higher than the figures published initially, although this has improved recently and the revised figure for last year was 0.1 percentage points higher. The first revision of the average number of reconvictions per offender is typically 0.01 to 0.02 higher than the initially published figure.

Table B2 Revisions to reconviction rates

Cohort	Initial published figures		1st revision of published figures		2nd revision of published figures		3rd revision of published figures
	Reconv. rate	Av. no. of reconvs. per offender	Reconv. rate	Av. no. of reconvs. per offender	Reconv. rate	Av. no. of reconvs. per offender	Reconv. rate	Av. no. of reconvs. per offender
2006-07	32.2	0.59*†	32.4	0.60*†	32.4	0.60†	32.4	0.60†
2007-08	30.9	0.56*†	31.2	0.57†	31.3	0.57†	31.2	0.57†
2008-09	31.0	0.58†	31.5	0.60†	31.5	0.60†	31.5	0.60
2009-10	30.1	0.54†	30.5	0.56†	30.6	0.56	30.6	0.56
2010-11	28.4X	0.50†X	30.1X	0.55X	30.1	0.55	30.1	0.55
2011-12	29.2	0.53	29.6	0.54	29.6	0.55	29.6	0.55
2012-13	28.6	0.51	28.9	0.53	28.9	0.53	28.9	0.53
2013-14	28.3	0.51	28.5	0.52	28.5	0.52	28.5	0.52
2014-15	28.2	0.50	28.3	0.50	28.4	0.51	28.4	0.51
2015-16	27.0	0.47	27.2	0.48	27.3	0.48	-	-
2016-17	27.2	0.48	27.3	0.48	-	-	-	-
2017-18	26.3	0.46	-	-	-	-	-	-

* These figures were not initially published, but it is possible to determine their magnitude retrospectively.

† These figures have been previously reported as the reconviction frequency rate, which was the number of reconvictions per 100 offenders. Therefore these figures are the original figure divided by 100 to get the average number of reconvictions per offender.

X From the 2011-12 bulletin, some new offences that came into effect from 2010 were included in calculations for the reconviction rate and the average number of reconvictions. These offences were not included in calculations in the 2009-10 or 2010-11 cohort bulletins. The increase in the numbers after revision led to a slightly higher increase at the 1st revision for the 2010-11 cohort than it had been in previous and subsequent years.

B32 Prior to the 2016-17 cohort bulletin, High Court convictions were included in the local authority group where High Courts were located. As the appearance of offenders at a particular High Court does not necessarily have any reflection on the local authority they were from, or where the offences were committed, they have been removed from the local authority groups. Removing these slightly elevated the reconviction rates by less than 1 percentage point for the local authority groups where the High Court permanently sits, which are: Aberdeen City and Aberdeenshire, Edinburgh and Midlothian, and Glasgow City; as offenders convicted for more serious offences typically have low reconviction rates. It had little effect on rates for the local authority groups where the High Court periodically sits, which are: Dunbartonshire, East and West; Lanarkshire, North and South; Renfrewshire and East Renfrewshire; Stirling; and West Lothian. The additional tables published alongside this publication present revised reconviction rates for previous cohorts of local authority groups for cohorts prior to 2016-17.