A process evaluation of the implementation of ASSIST in Scotland

Executive Summary

This report presents findings from a process evaluation of the ASSIST (A Stop Smoking in Schools Trial) pilot in Scotland. ASSIST is a licensed peer-led, school-based smoking prevention programme that encourages the dissemination of non-smoking norms by training S1 (aged 12 to 13 years) and S2 (aged 13 to 14 years) students to work as peer supporters. Peer supporters are trained to have informal conversations with other students about the risks of smoking and the benefits of not smoking. The programme consists of seven training and feedback sessions with the peer supporters (peer nomination, peer recruitment meeting, peer supporter training and four follow-up sessions - where peer supporters meet with trainers and receive ongoing support), delivered over a 14-week cycle by trainers from outside the school who have attended a three-day course to enable them to deliver the programme.

ASSIST was previously evaluated through a large cluster randomised trial of 59 schools in South Wales and Avon, England with results published in the Lancet in 2008. Findings from this trial showed that ASSIST was effective and cost-effective at reducing smoking prevalence in young people. In 2010 ASSIST became a licensed programme and DECIPHer-IMPACT ^[1]Ltd was established to provide training, materials, support and quality assurance to maintain effective delivery.

ASSIST in Scotland

In 2013 the Scottish Government made a commitment to undertake a pilot of ASSIST in its national Tobacco Control Strategy^[2]. One of the key differences between the delivery of ASSIST in Scotland compared with England and Wales is the age difference in school years. In England and Wales ASSIST is delivered to Year 8 students which is equivalent to S2 in Scottish schools. However, the age composition of year groups varies from England. At the end of year 8 students are aged between 12-13, whereas S2 students are aged 13-14. This is important because the slightly older age range may result in students being less receptive to the programme. The agreed approach in Scotland was to pilot in both S1 and S2 but to target S1 in the third term or second half of the school year.

The ASSIST programme was delivered in three NHS Boards across Scotland: Greater Glasgow and Clyde; Lothian; and Tayside. All three areas followed the licensed DECIPHer-IMPACT programme but their delivery models, in terms of project management, staffing and number of schools they worked with, varied. Across all sites delivery staff had a mix of professional backgrounds and came from the NHS, Local Authority and Third Sector. Some were employed as youth workers but did not have a background in smoking prevention or cessation, while others had tobacco control experience but had not previously worked with young people.

Aim and Research Design

The overall aim of the study was to evaluate the process of implementing ASSIST in Scotland. A series of research questions were developed and are included in the key findings section below.

In light of existing evidence demonstrating the effectiveness of ASSIST, this study focused on the acceptability and implementation of ASSIST to inform any potential future adoption in other areas of Scotland. The research design involved mixed methods, consisting of three elements: 1) evaluating the implementation planning process; 2) evaluating delivery in schools; and 3) assessment of costs. A range of stakeholders (school staff, trainers and students, n=101) were consulted via in-depth interviews, paired interviews, mini focus groups and observation along with a before and after survey to gather data from students (n=2130, at follow-up). To maximise available resources a two-tier design was used. Tier one included consultation with school leads and a pre and post student survey in 20 schools. Tier two involved six case study schools (two in each area, selected from the 20 tier one schools) where qualitative methods were used to observe peer supporter training and follow-up sessions and consult with peer supporters and other students.

Key Findings

Findings are summarised for each research question in the study, with recommendations in italics.

1. What are the barriers and facilitators to the implementation of ASSIST in Scotland?

Barriers and facilitators in this study were categorised as operating at the macro (strategic) and micro (operational) levels. At the macro level they were: partnership working; budget; and culture. These varied between schools but partnerships in particular were key, and findings from the evaluation suggest that future delivery of ASSIST in new areas should ensure enough time is set aside to build relationships with key stakeholders (such as school leads, NHS and Local Authority staff) in advance of programme delivery, if these relationship do not already exist.

At the micro level, barriers and facilitators were: trainers; delivery mode (peer nomination, training and follow-up); and behaviour management. Due to the different delivery models across the pilot sites, some trainers had a delivery role only, while others were responsible for both coordination and delivery. The combined role was particularly challenging. Feedback from trainers on their three day training to deliver ASSIST was extremely positive. However, potential suggestions for improvement included more time in the training on how to organise the follow-up sessions.

Levels of confidence to deliver the programme were influenced by professional background. Where possible pairs of trainers with a mixture of smoking cessation and youth work expertise should be involved in programme delivery.

Peer nomination was only delivered under exam conditions in one of the six case study schools, resulting in discussion between students which could have influenced whom they nominated. This may be a difficult element of the programme for all schools to engage with, but should be attempted.

Trainers faced some challenges delivering follow-up sessions in school (e.g. obtaining access to classrooms) and many peer supporters did not complete the requested diary of their conversations with others about smoking. Peer supporter comment suggested that the content as well as the number of follow-ups could be modified. Fewer follow-ups should be considered in future and diaries, if included, may not need to be paper-based.

2. What refinements are required to implement the ASSIST programme in Scotland?

Overall, we found that very few changes were required to implement ASSIST in Scotland. In terms of trying to deliver in either S1 or S2, there were advantages and disadvantages to both. School leads who were interviewed tended to focus on the reasons why one year was chosen over the other - not how this decision may have influenced delivery of the programme. Student feedback, however, suggested that friendship groups may have been more established in S2 than S1. Therefore, in light of student feedback, if ASSIST is delivered in S1, this should ideally be in the second half of the school year when friendships are more established.

3. Were essential elements of the ASSIST model maintained during pilot implementation in Scotland?

Overall, delivery appears to have been with a high degree of fidelity to the licensed programme. For example, the manual states that the list of students nominated to become peer supporters should be 18% of the school year to ensure adequate coverage. All case study schools delivered the four follow-up sessions, but observational fieldwork highlighted that diaries were not consistently checked in two of the six case study schools.

4. How acceptable is the programme from a stakeholder perspective (strategic leads, trainers, students and school staff)?

Stakeholder feedback suggests that delivery of ASSIST in schools in Scotland was acceptable with the (unavoidable) minor disturbance to school timetable accommodated. School leads indicated support for the peer education model and programme delivery which required a small amount of school resources. Trainers demonstrated a clear understanding of the theory behind ASSIST and recognised the importance of delivering the programme according to the manual. They were particularly clear that schools should not interfere with the peer selection process or delivery. School leads demonstrated a similar commitment to the peer element of the programme but there were examples of anxiety around student selection and an initial concern that the target of 18% might not be achievable, although it was. Peer supporters were positive about the programme and recognised that peer to peer message diffusion was more appealing than having an adult speak to them about smoking. They appeared to favour the two training sessions more than the follow-ups sessions which is not surprising given the delivery context (i.e. the former was delivered out of school and the latter in school)

5. What changes in smoking-related knowledge, attitudes and behaviour are observed amongst students in the ASSIST Scotland pilot schools?

This process evaluation was designed to look at acceptability and fidelity, not intervention effectiveness. However, from the data we have, it is clear that there is uncertainty regarding the extent of message diffusion between peer supporters and peers in their school year and any impact this may have on smoking in adolescence prevalence. Findings from the student survey showed no significant change in self-reported smoking prevalence with 1.6% of pupils (n=33) reporting that they smoked one or more cigarettes per week increasing slightly to 1.8% (n=38) at follow-up. In addition, recall of any relevant conversations about smoking with a peer supporter was fewer than one in ten (9%). However, opportunities to have informal conversations about smoking with peers may now be limited due to the ongoing decline in adolescent smoking since ASSIST was first developed. It is also worth noting that some peer supporters felt apprehensive or awkward initiating conversations about smoking with their peers for fear of being judged or ridiculed, and this also may have contributed to the low recall of any conversations relevant to ASSIST.

Data collected from the original ASSIST trial is now 13 years old. Our findings on the number of relevant conversations (in particular) do raise questions about any appropriate assessment of ASSIST in the future. A relevant future step would be an implementation trial (a Phase IV study) of ASSIST, using a similar methodology to the original RCT, to assess current effectiveness in the context of lowering smoking prevalence in the target age group.

However, feedback from participants was overwhelmingly positive regarding the benefits of taking part in ASSIST for peer supporters, commenting on the personal skills they have gained and the potential for the school and wider communities to benefit. Recognition of the wider benefits of peer supporter training should be included in discussions around the future of the ASSIST programme in Scotland.

Another important factor perceived to influence message diffusion was the view that peer supporters were more likely to talk to family members than their school year peers. This could be important, particularly if social network members are smokers. The impact of ASSIST in encouraging peer supporters to discuss smoking with smoking parents, other family members and wider social networks is an important area for research.

6. What are the delivery costs of the programme?

The average cost to deliver one cycle of ASSIST in one school ranged from £8,939 £11,357. Between August 2014 - June 2017, 72 cycles of ASSIST were delivered or confirmed for delivery in 2017 and included in the cost analysis. The actual total delivery cost for all three sites was £674,360 of which 20% (£136,800) was for the license fee. This equates to a cost per cycle of £9366.

7. What lessons can be learned to assist future roll out across Scotland?

There are five considerations for any future delivery of ASSIST in Scotland, and a sixth relating to future research.

First, if a relationship between programme deliverers of ASSIST and the school is not already established, time should be included to invest in this before delivery. This will help with the programme but also add important context in terms of what smoking prevention provision the school already has in place. If delivery is in partnership with NHS Boards, Local Authorities and the third sector, time needs to be built into the delivery timetable to establish these relationships and understand the level of resource required from each partner.

Second, identifying trainers depends on available resources and they are likely to have a variety of professional backgrounds. Using this experience to pair up trainers with different backgrounds (e.g. one with youth work experience and one with smoking cessation) and developing opportunities to share practice and experience across sites may aid delivery and increase confidence.

Third, if the school timetable permits delivery of ASSIST to S1, students should be targeted during the third term, as this will maximise message diffusion via established friendship groups. Few problems were observed with delivery in S2, so this continues to be viable.

Fourth, consideration should be given to how trainers can be offered further support (e.g. further training, better use of teaching staff who chaperone students) to manage student behaviour if they feel this is required.

Fifth, in terms of the manual and intervention delivery, DECIPHer-IMPACT may want to consider: 1) whether a paper diary is still the best medium for students to record their conversations; 2) if four follow-ups are still productive; 3) how to include content on e-cigarettes in the programme, as appropriate (in particular, making clear that they are far less harmful than tobacco but are not products for teenagers that have never smoked, and informing teenagers of age of sale laws).

Sixth and finally, as outlined above there may now be a need for a Phase IV trial of ASSIST to determine effectiveness in the context of continued decline in smoking prevalence. This should, if possible, take into account that smoking still remains an issue particularly in more deprived areas.

8. Is there the scope to expand the model to look at other risk taking behaviours in Scottish schools in the future, e.g. drugs, alcohol?

There was general agreement (from school leads and students) that the ASSIST model (ie peer to peer message diffusion) could and should be applied to other risk taking behaviours such as alcohol or drugs. Currently and previously there have been other versions of ASSIST (focusing on sexual health, physical activity, healthy eating, binge drinking and drug prevention). An NIHR funded feasibility trial of the ASSIST model as applied to drug prevention has just concluded in Wales. Plans are now underway to seek funding for one component of this for a larger trial. In addition, a second NIHR feasibility trial of the ASSIST model applied to sexual health behaviour is currently underway in Scotland. Early results from both these studies look promising. Both studies should provide valuable data on the key elements of the peer to peer model that are particularly relevant for behaviours beyond smoking. School staff and stakeholders were also interested in how the ASSIST model could address multiple behaviours in one intervention, but this will be far more challenging to deliver. Investigating how/if the ASSIST model could be developed to address more than one risk behaviour is an important area for further research.