A process evaluation of the implementation of ASSIST in Scotland

Chapter 3: Preparing for Delivery

The focus of this chapter is to examine the key considerations required in preparation of ASSIST delivery in schools. Drawing first on interviews from site leads and key stakeholders it will reflect on the preparatory 'behind the scenes' work required before actual delivery in school. Then, for additional context it will describe where schools were starting from in terms of current provision for smoking prevention. Finally it will outline the experience of trainers, specifically their feedback on the training and their perceptions of their readiness to deliver ASSIST in schools.

3.1 Preparatory Work and Delivery Models in Pilot Sites

Delivery of ASSIST required time and careful planning, working in partnership with school staff and other key partners (appended in Table A1). Site co-ordinators and key stakeholders were asked to reflect on this process and highlight three areas for discussion: identifying and engaging with key partners; delivery model; and identifying schools.

3.1.1 Identifying and Engaging with Key Partners

Identifying and engaging with partners was an important first step in delivery, with stakeholders commenting that a key learning point was to allow enough time at the planning stage to create these partnerships and obtain buy-in. This was especially true for Site 1 and 2 where engagement was necessary with seven Local Authorities. Reduced funding meant job roles and the remit of Local Authority staff had changed, resulting in additional time to identify the correct person to approve the resources needed, or secure any in-kind contribution. Key to facilitating these discussions was the existing evidence-base regarding ASSIST's effectiveness which helped secure buy-in from partners.

"I think the evidence, the initial evidence; we had been running tobacco programmes in [Site 3] for young people for a very long time….The problem had always been how to evidence that those programmes were effective…..So it [ASSIST] was something that was clearly evidence based, we wouldn't have to do the research to work that out." (Stakeholder 11)

Having existing relationships and previous positive experience of working with partners was clearly advantageous to engagement with ASSIST. All three pilot sites had this, some stronger than others, but this foundation was clearly beneficial and more productive than tendering to a sub-contractor (as is the more common model in England and Wales).

Concerns around sustainability of a pilot programme, like ASSIST, are common and justified. This is why 2 sites made the decision to deliver ASSIST more than once to schools who signed up. A particular aim of Site 2's model of a partnership approach between NHS, Local Authority and the third sector was to embed the tobacco prevention agendas and build capacity in other organisations (i.e. not just the NHS).

Finally, partners helped deliver ASSIST by offering in-kind resources - e.g. staff to deliver ASSIST, liaise with schools and encourage their participation, school staff time to help plan delivery and chaperone peer supporters, classrooms for follow-up.

3.1.2 Delivery Models in each NHS Board

The ASSIST pilot programme was delivered in three NHS Boards across Scotland: Greater Glasgow and Clyde; Lothian; and Tayside. All three areas followed the licensed DECIPHer-IMPACT programme but their delivery models in terms of project management, staffing and number of schools they worked with varied across pilot area. A summary of the delivery model in each area is presented in Figures 2, 3 and 4. Further detail on: funding source; key partners; number of schools who participated' pilot start and end dates; and number of trainers can be found in Table A1, appended.

Figure 2: Site 1 delivery model

An 'in-house' model operated in Site 1 with a pool of NHS and seconded Community Learning and Development (CLD) staff delivering ASSIST as their sole remit (except the NHS lead). The delivery model was one of devolved responsibility with the NHS lead appointing three lead trainers who were responsible for securing schools, planning delivery, and liaising with trainers in a specific geographic area. They also worked as trainers and cross-covered other areas to offer support when necessary.

Figure 3: Site 2 delivery model

A partnership model was used in Site 2, with delivery subcontracted to a third sector organisation (TSO). The NHS lead took responsibility for securing school participation and sourcing trainers, with the TSO then following up to arrange delivery dates, co-ordinate trainers and manage all aspects of delivery. All staff involved in delivery of ASSIST had other responsibilities in addition to their ASSIST role.

Figure 4: Site 3 delivery model

Site 3 operated a flat structure where the site lead along with another member of staff took responsibility for all set-up and delivery tasks (securing participation from schools, organising the delivery timetable and associated tasks - booking in trainer and the materials required for delivery). Trainers were NHS staff who worked across the three Community Health Partnership sectors, which meant that site co-ordinators had to liaise with each sector to secure participation from staff - this made planning delivery more time consuming (especially if schools had to change delivery dates with short notice) than the other two delivery models. Like Site 2, Site 3 staff had other responsibilities in addition to their role in ASSIST.

Model 1 is similar to how ASSIST was previously delivered in England and Wales (the current approach is to outsource to a third party). Stakeholders commented that the Site 1 model worked particularly well. However, this approach was only possible because this NHS Board purposely moved resources from cessation to prevention programmes. The more complex approach to securing trainers in Sites 2 and 3 meant that the site co-ordinations and lead trainers had to manage staff that may not have worked with young people before and who had other responsibilities in addition to ASSIST. This required extra time to build relationships, contact staff to confirm training sessions, and accommodate timetable changes when they arose. Time that was not required in Site 1.

3.1.3 School Selection

School selection varied across sites (as illustrated in Table 3). In Site 3, to address sustainability concerns, the decision was made to target 10 schools who were all eligible to receive three cycles of ASSIST. In contrast, Site 1 had the resource to offer ASSIST to all the schools in their NHS Board, and used SIMD data to identify the schools most in need (i.e. those in deprived areas, with higher smoking rates) of ASSIST and tried to target them first. Site 2 targeted schools with higher smoking rates, but also took into account other factors that may impact delivery i.e. change in the senior management team. That said, all sites indicated a degree of pragmatism and were keen to work with schools who showed an interest and enthusiasm, especially when they first started to deliver the programme.

3.2 Schools' Starting Point

In order to assess delivery of a smoking prevention programme such as ASSIST, it is important to understand where the school was starting from in terms of smoking prevention coverage in the existing curriculum. Answers to these questions may influence why the school decided to take part and add important context to understanding delivery.

Across the 2491 respondents at baseline, findings from the pre-intervention student survey suggested a reasonable awareness of their schools' smoking policy and any lessons on smoking harm/prevention, with two thirds (64%, n=1532) recalling being taught lessons on smoking prevention^[7]. However, half (55%, n= 1306) did not know if their school had any rules on smoking, and 57% (n=1365) did not know what action would be taken if they were caught smoking.

Schools participating in the evaluation appeared to either adhere to the standard Local Authority policy or did not have their own school specific smoking policy - with some commenting that they did not know if their school had a policy or not. The exception was school 12 who had included their own smoking policy in the school handbook (i.e. 'no smoking on school grounds, any student found with cigarettes will have them confiscated'). The general point was also made that the number of students smoking these days is very low, with a perception in some schools that smoking is not much of an issue anymore - "I could probably tell you children that smoke." (School lead 20, follow-up interview)

Existing education on smoking was generally delivered in social education or personal, social and health education (PSHE), but the school year varied. For example, in some schools it was delivered across all years whereas in others it was targeted at younger students (S1/S2). National 'no smoking day' was an important feature of the school calendar at the time of the study, with a number of schools involved in different activities, often in partnership with local organisations.

3.2.1 School Perceptions of ASSIST

School leads demonstrated a good understanding of the ASSIST model, grasping the key components e.g. a focus on smoking prevention using a peer education approach with a 'non-preachy' approach to message diffusion. One school lead interviewee raised a salient point when they commented that they were not sure if ASSIST would include anything about vaping (electronic cigarettes).

There was also a great deal of support for the programme. Delivery by external trainers (i.e. not school staff) and the two training days delivered away from school, were viewed as particularly important because it showed the young people how valued they were. Even in schools who commented that smoking was not 'a huge issue' they still felt the programme was worthwhile because 'young people will experiment with cigarettes.' Comment was also made regarding the existing evidence base which showed the effectiveness of ASSIST. Again this emphasised the importance of setting aside time to engage with schools to build up a relationship. However, there was also speculation regarding how effective message diffusion would actually be (i.e. will peer supporters have conversations?).

There were a number of reasons school leads gave when asked why their school decided to take part in ASSIST. First, was the opportunity to receive an additional (evidence-based) resource which required minimal school input to protect young people from smoking harm. Second, signing up to ASSIST seemed to have developed as a result of long-standing relationships with NHS health improvement teams and a specific or general willingness to help them or take part in the pilot to help future delivery. Third, was a specific interest in the peer education element of ASSIST either because they had seen the benefit of this approach previously or wished to pursue it more fully in the future. One school lead said that in addition to the possible impact of the programme, ASSIST would help them meet requirements of national policy - partnership working set out in the National Curriculum for Excellence, for example. Finally, one interviewee was not aware of why their school had decided to participate but had simply 'been asked (told)' to lead on this by a senior member of staff.

3.3 Trainer Perspectives on Preparing for Delivery in Schools

One essential element in preparation for delivery was training of trainers. Baseline interviews with trainers explored this in detail, as well as their perceived role, confidence to administer the role, and sources of support that would be available to them.

3.3.1 Feedback on 'Train the Trainer' Training

ASSIST trainers took part in three days of "train the trainer" training delivered by DECIPHer-IMPACT (funded as part of the licence fee). Overall, trainers were very positive about the training and particularly liked the interactive and energising nature of delivery, leading many to describe it as 'fun', 'enjoyable' and the 'best training they had done.' A stakeholder comment was similar - one participant commented that their colleagues (who had attended the training) were struck by the quality of the training programme, and about how well thought out it is and delivered.

Training content gave participants multiple opportunities to obtain first-hand experience of delivering training activities and observe the different approaches of fellow participants, which was viewed positively.

"I liked the fact that it was more interactive than sitting being talked to. And it was giving us the chance to practice and to go through which was, certainly for me it was more helpful and more memorable because I can think back to oh that activity was very, so and so did that or I did that and it's making it more memorable for me." (Trainer, Site 1)

Additionally, trainers mentioned benefiting from being asked to act as prospective audience members (i.e. young people) and therefore gaining differing perspectives on delivery.

"Because you are the peer supporters and basically somebody is delivering this to you and you start to get the feelings of what sounds good, what feels good for me, what doesn't feel good, and that is important to get that feeling because the audience is all important and if you are not getting it right for them then you know you miss the target." (Trainer, Site 2)

These aspects were perceived as very useful in preparing to deliver ASSIST as were the opportunities training provided for trainers to get to know one another, particularly those with whom individuals would be working closest.

"Probably one of the key things - as well as learning the whole thing - but one of the key things that I think comes out of that is the fact that you get to know everybody else so well, whereas, I don't know, maybe if it was structured differently and it wasn't as long a time, you wouldn't feel as comfortable." (Trainer, Site 3)

The three day duration of the training, though daunting for participants beforehand, was thought to have been necessary and appropriate. Venues and catering for training events were also positively received with one participant remarking on how such detail had made them 'feel special' and was thought to be beneficial for creating the right environment for young people.

"… also we were made to feel special, but I think that was psychological out of what they were meant to do, so that you were feeling, so you got a lovely lunch each day you know and we were all talking about the lunch, so we need to actually incorporate that into what we are supplying to young people so that they feel special and they want to engage with the programme." (Trainer, Site 3)

One possible improvement mentioned, however, was for more attention and/or time to be given to the follow-up sessions within schools. One participant commented that follow-ups were discussed at the very end of the course when they were more likely to be tired and lacking in energy.

"I remember at the end of Day 3, sitting in the group to do the follow-up…..I remember sitting there and I know I wasn't the only one an' I was, like, 'I'm really sorry but I am knackered. I am not taking this in. I'm reading it but I'm reading the same sentence twice. I'm just not taking this in.'…..So, I don't know whether - I mean, another half day's probably not practical - but maybe there needs to be, maybe the follow-up needs to come earlier on Day 3 or more time given to it, or something different done around it." (Trainer, Site 3)

Additionally an important suggestion came from participants not working in tobacco control/health promotion, that the training course could involve more information on smoking and tobacco. However, they also acknowledged that the presence of others with such knowledge positively off-set this challenge.

"For me I mean I have no knowledge or background knowledge of smoking and prevention stuff like that so I was a bit kind of flapped about kind of not having that. But I think what we quickly picked up on, day one is more about the information that they require whereas day two is more about the skills that they require. So I felt more confident about the delivery of day two, that I could you know get involved with that. Whilst also being involved in the day one smoking part but that's not my strong point. Whereas that is [other trainer's] strong point, is the smoking knowledge." (Trainer, Site 2)

3.3.2 Understanding of ASSIST

In general trainers demonstrated a good understanding of the ASSIST model and were able to pick out key aspects which they felt were most important for explaining the programme to others. The characteristic of ASSIST most commonly mentioned was the peer nomination element, which was viewed positively.

"I think the good thing is that you are going to get a really good mix of young people no matter what school you go to because teachers haven't picked them, so they might not be the most academic, they might not be the best behaved but it's still something for them to take ownership of and work with." (Trainer, Site 3)

Trainers were also clear that the ASSIST programme is aimed at training young people to positively influence their peers in an informal and non-prescriptive way. Some also emphasised that the programme was about empowering young people or giving them the skills to communicate with their peers about smoking.

"I mean if I was to encapsulate it in one word it would be to empower these children, to actually give them that power to be able to just confidently go forward, so we give them knowledge, we give them skills, you know, we are teaching them context and things, so in some ways we are empowering them..… They are walking in and saying 'hey I am confident about this, I can talk about this. I am going to be able to slip these suggestions into conversation, constantly and without batting an eye because I know this, I am trained in this and I can do this'." (Trainer, Site 1)

Trainers' understanding of the ASSIST programme generally took into account the importance of fidelity to the model. Trainers understood that to ensure the effectiveness of ASSIST they had to adhere closely to what they had learnt in their training though some felt there were opportunities for flexibility.

"I think yes the prescriptive thing is great but sometimes there may be opportunities for us just to adapt slightly. But it's about making sure that the main part is as it should be." (Trainer, Site 1)

For some, the importance of adhering to the established model of delivery seemed to reinforce their appreciation for the effectiveness of the approach and represented an exciting prospect.

"For me I was quite excited actually because I've worked in tobacco for quite a while and I think especially in schools we have not really came across anything that is evidence based and that actually works and that you actually have a thing, like a script, not a script, but it's all written out for you, we've never came across that before with tobacco it's always been a wee bit woolly and maybe trying things, you know different areas try different things in schools but actually to have something that is evidence based to take it into the school I think that is quite exciting to be involved in it and because it's new and it's been piloted I think it's a good opportunity." (Trainer, Site 2)

3.3.3 Trainer Perspectives of their Role within the ASSIST Programme

Delivery models varied across sites with some trainers having a delivery role only, whereas others had a remit for both coordination and delivery (see Figures 2, 3, 4). As might be expected, given these differences, there was also variation in expectations of how much time the implementation of ASSIST would involve, but there was agreement that delivery would become quicker as they worked with more schools.

"I think, initially, it'll take up more time than probably I would, you know, envisage because, just to you get familiar, and I think it's like anything - once you start delivering it, you become more comfortable and, you know, your knowledge of the pack becomes better." (Trainer, Site 3)

There were also trainers who delivered ASSIST alongside other responsibilities completely separate to ASSIST. This caused some anxiety for the trainers with regards to managing these different commitments.

"I'm hoping it's not too much because nobody will do my substantive role while I'm not there - not that I think it will deflect me from the programme - but I don't know how I'm going to juggle it yet, to be honest with you. I haven't had a look at the form, for the timings etc, so that is a bit of a concern for me, to be honest." (Trainer, Site 3)

3.3.4 Levels of Confidence Prior to Delivery

Trainers' levels of confidence prior to delivery of ASSIST in schools were generally high, mainly reflecting satisfaction with the training they had received on the programme. For those who expressed low confidence levels in the baseline interviews this appeared to simply be a reflection of the fact that they had not yet delivered the programme.

"… basically there is a lot of things that no matter how much preparation that we do as a team there is always going to be that little unknown and in some ways it's that little unknown. And it's not that I am not confident to deal with it, but there is just a little unknown…" (Trainer, Site 2)

Trainers felt that once they had had this experience and were familiar with what they were doing their confidence would increase. However, there were two other concerns that also influenced confidence to deliver ASSIST. First, was the lapse in time (often involving several months) between receipt of training and expected delivery and the possible impact on their training capability. Though all felt this issue would be resolved by re-familiarising themselves with programme materials prior to delivery. Second, feelings of frustration and being under-prepared were reported by some trainers who were unclear of delivery details (e.g. training delivery dates).

Experience of working with children or young people or having a background in/knowledge of tobacco control/health promotion were important factors that affected confidence. Those that had little or no experience of working with children or young people worried about how they would respond in challenging situations. These are important issues related to behaviour management discussed more fully in Section 4.9.

"…. we know that we are going to go along and we are going to participate, but how do you know that your young people will, so it does, because after my other week when I had a group, like I did the sort of world café thing and some of the groups were lovely and then this, you get a wee group that just doesn't want to even communicate with you, so how do you know that that is not going to happen?" (Trainer, Site 3)

Similarly those with limited knowledge of tobacco were concerned that they may find themselves in a situation where they could not adequately respond to a question.

"Maybe where I do require support is because my background isn't in I suppose health promotion and relating to smoking…. For me I would find it quite challenging if a young person were to say oh right so you said there is these amount of chemicals in cigarettes what are they called, you know, if there was a specific question that I maybe didn't have the knowledge of…" (Trainer, Site 3)

However, despite these concerns, participants felt that support from trainer colleagues with knowledge and skills in the areas where they felt weakest would make up for shortcomings.

3.3.5 Sources of Support and Working with Other Trainers

Trainers identified a number of sources of support for ASSIST delivery. This included calling on the site co-ordinator or their own line manager (if not the same person). Others contacted the DECIPHer-IMPACT team directly and reflected on this support very positively. Another important source of support were the schools themselves, especially where good links were already established (i.e. where trainers had worked with them in the past).

"….a lot of support from the schools as well because….it is about relationship building with the schools…..because we've always had support from the schools in all the bits of work that we've done…..without them we wouldn't have any work because they don't have to let us in basically and you know if the schools say 'no we don't want you' there is nothing we can do. So they are our most valuable source of support, depending on who you get." (Trainer, Site 1)

Most commonly trainers perceived that, if necessary, they would call on the support of other trainers working alongside them in the same area. As highlighted in the previous section, this fellowship between trainers was considered important in responding to specific issues where individual trainers may lack confidence but it also seemed to be important more generally perhaps due to a sense of a shared experience.

"I think the core support is coming from each other, because they are all, it is all very new and we will all experience it at the same kind of time that we are able to turn to each other and utilise certain skills that certain people have got, and do it that way" (Trainer, Site 1)

In one particular site, co-ordinators were highly aware of the potential for support amongst trainers as indicated by the set-up of an online forum and meetings held in preparation for the 'train the trainers' course and prior to first delivery.

In contrast, some participants indicated reservations about working with other trainers, primarily because they had not worked together before. Trainers who held this view indicated a preference to work with someone they already knew (who had also been trained in and would deliver ASSIST) in order to benefit from knowledge of each other's way of working.

"… and we won't ever work together… It is a shame in the sense that we kind of know how each other works to a certain extent, even though our jobs are different we still, it's still nice to know how somebody works, but that won't happen and that's fine." (Trainer Site 3).