A process evaluation of the implementation of ASSIST in Scotland

Chapter 4: Delivery of ASSIST

The aim of this chapter is to discuss the different elements of ASSIST delivery (peer nomination, recruitment meeting, peer supporter training and follow-up sessions) with a specific focus on fidelity. This section will also discuss six elements related to delivery - initiating conversation, school contribution to delivery, scheduling ASSIST, school year of delivery, behaviour management, and will finish with a review of costs associated with delivery.

4.1 Peer Nomination

Feedback on peer nomination has been summarised into four main themes: reflection on the theory behind the peer nomination process; practical application of peer nomination; nominated students; and delivery fidelity.

4.1.1 Peer Nomination Theoretical Approach

In general, school leads appeared supportive of the peer nomination approach. On one hand, there were schools who put their faith in the existing evidence base and supported all students nominated to be peer supporters. These school leads believed that the best outcome would be for students with challenging behaviour to be nominated because some of them can be important influencers of their peers. However, there were other school leads who had concerns about the suitability of some students to take part, mainly due to immature behaviour and mixing of students who do not usually spend time together. For example, in one school the school lead had a particular concern over mixing 'street kids' and 'quieter kids', but was pleasantly surprised to see that they did engage and there was an improvement in their behaviour.

There were also some schools who wanted to maintain a degree of control over who took part in the peer supporter training. For example, if nominated students were perceived to be disruptive they would not be allowed to go, or were told they could go but were warned that they would be pulled from the programme if they caused any trouble or their behaviour reflected poorly on the school

"Then I saw the list an' I just thought, 'Oh God, no, not on your life.'….There was about three - two or three - we had to withdraw cos there was absolutely no way on earth." (School lead 9, baseline interview)

Another school lead reported that they had made the decision to exclude certain students from peer nomination. While acknowledging that this was at odds with the peer nomination ethos they had concerns that peer nomination was being used as a form of implicit bullying. They held this view because they felt some students from their first wave of ASSIST had been nominated by other students to make a fool of them. Another school also commented that they thought some of the students may have been nominated for 'a joke.' Due to the seriousness of this perception it was followed up in interviews with trainers and site co-ordinators. Trainers did not feel this was a widespread problem and commented that students did not know what they were nominating their peers for so bullying was unlikely:

"Nope I think that's rubbish…. because that young person will have no knowledge what they are nominating that person for. So that's more personal in my opinion what they are saying." (Trainer, Site 2)

However, there was also the view (from trainers) that some students had nominated other students for 'a joke.'

Finally, there was a belief that it would be the most popular students who would be selected or the ones who usually put themselves forward for things like the school show. This caused some concern that peer nomination may exclude more disadvantaged pupils.

"I don't see a lot of young people that are, that look to me that they are from a deprived background, so who is influencing them because I don't think, my feeling is that these ones we are training wouldn't have smoked anyway, maybe one or two of them…a lot of them that come along to me are high achieving young people, they are very good at drama, well you'll have seen them yourself, a lot of them are very confident." (Trainer, site 3)

However, as this school lead comment shows there were examples of student diversity, which is the aim of peer nomination.

"Q: And what was your initial impression when you saw the students that had been nominated?"

"Some of them were, some of them were who I would have expected but some of them were quite different so I think it was a real mix… I expected it to be the same folk all the time, the normal kids that would do it, and some of them were obviously, but some of them were quite different from that, so that was really good." (School lead 8, follow-up)

Trainers were clear that school staff should not interfere in the delivery of ASSIST. There was a recognition that schools may attempt to intervene in the nomination of students they may deem unsuitable for participation, yet trainers at baseline seemed largely unconcerned about responding to and resolving such challenges in line with the ASSIST model.

"We've got to be quite forceful about that……We can't add extra people in so, again, but … just be really clear that they can't - and it is, it's interfering in the process. They can't interfere in the process." (Trainer, Site 3)

At trainer follow-up it became clear that in instances of school intervention in peer nomination, trainers were willing to challenge schools though not necessarily to reject attempts to remove certain students from the programme.

"…we normally give them all a wee bit of leeway if there was somebody they'd felt couldn't handle the training at all or would be extremely disruptive to the point where it would destroy it for everybody else. We've gone with that because the teachers know the pupils." (Trainer, Site 1)

4.1.2 Peer Nomination Practical Application

A range of comments were received from school leads regarding the practical application of peer nomination (i.e. delivery in a school setting). For example, concern was expressed at baseline interview that there may not be enough students nominated or there would be gaming i.e. 'I'll nominate you if you nominate me.' The former was not an issue with the school lead noting their surprise (during follow-up interview) at how easy peer nomination was and that the critical mass of 18% (discussed in Table 1) was achieved. As most peer nominations were not conducted under exam conditions, researcher observations recorded several instances of students conferring with each other. Thus there is a possibility that gaming may have occurred. For example, students who did not understand what they were being asked to do tended to nominate their friends in the hope that they would be nominated by them:

"I thought it was for classes." "That's why I put all my friends down so I would be in their classes." (Peer Supporter, school 1)

"It was good but, like, you didn't know what this was for so I just wrote all my friends down an' I should of wrote down other people." (Student, school 17)

Other views centred on the mode of delivery i.e. a special assembly, with students sitting next to their friends, which may have influenced who they nominated. Trainers noted that during the scoring process following peer nomination tended to highlight friendship groups.

"…when we are doing the cleaning [scoring] process, you can tell whose group of friends have been sitting with each other when they're doing it. You get so many with the same names and then suddenly it's the next lot and its different names but then they appear. You can tell who's been sitting together cos they are sitting nominating each other." (Trainer, Site 2)

Linked to this was discussion around the school year being so large that students often became disengaged and did not pay attention during a special assembly. In light of this it is perhaps not surprising that one school lead questioned whether peer nomination would have been different had they approached it on a class by class basis. Stakeholders commented that delivery of peer nomination via special assembly was 'not ideal' and trainers noted that exam conditions were preferable but that this could be overly burdensome for schools to arrange.

"Very rarely have we managed to have them at individual desks and that's just practicalities within schools, they don't have the time and the feasibility to set-up the assembly halls like that. There's been one or two but that's quite, like near exam time because it's been like that anyway." (Trainer, Site 1)

It was suggested (by a school lead at follow-up) that peer nomination could be done on a class by class basis using survey monkey (or similar online survey package) which would allow school staff could greater control over levels of conferring and students would have more anonymity. This may also make score tallying to identify the list of peer supporters easier for the trainers.

School leads also commented that some of the students found it difficult to answer some of the questions - who they admired, for example. Student feedback supported this view, with reports of students finding some of the questions "really hard" to answer. Follow-up interviews with trainers also highlighted that some students struggled to understand the language used in questionnaires.

"And I think some of the questions in the peer nomination are maybe a bit difficult for them to understand. Is there one to do with respect? There's two of them that seemed quite similar. And so I think the young people get a bit confused by that." (Trainer, Site 1)

4.1.3 Nominated Students

Just over one fifth (22.5% n=477) of students said they were asked to be a peer supporter, which is slightly higher than the fidelity measurement of 18% recommended in the original effectiveness trial of ASSIST. This is most likely a reflection of tied scores where one or more students had the same score - rather than exclude some, they were all put forward.

Both school leads and students expressed an element of surprise regarding those nominated to be peer supporters. Student opinion from mini group discussion was split in terms of how representative peer supporters were of their year group. On the one hand, was a view that those who were nominated did represent their year group because of the various friendship groups. However, there were also those who felt that some groups of students had been left out, resulting in an under-represented section of the year group. Results from the student survey revealed that 16% (n=317) felt that students chosen to be peer supporters were not the best ones to speak about smoking. This view became more apparent once the purpose of the peer nomination was explained during the focus group discussion. There was also a belief that some students did not 'deserve' to be nominated because their motivation to take part was perceived to be getting out of classes, with other more deserving students being left out who would have worked harder.

Additionally, drawing on observation notes from a peer recruitment meeting, a school lead noted their surprise to trainers that none of the students selected to be peer supporters came from an ethnic minority background despite students from this demographic making up approximately ten per cent of the year group.

In general schools did not believe that students who were not selected to be peer supporters would be too concerned. However, there was a view that there may be a degree of jealousy at being away from school for two days, with the perception that some were 'a bit grumpy' at not being selected. School leads also reported that a few students questioned why they had not be chosen. On one occasion, students who were not selected as peer supporters attended a session which would suggest they were interested in what was taking place. Survey results presented a similar view - of those who answered, 66% (n=833) said 'I don't mind' when asked how they felt about not being nominated and just over a quarter (27% n=343) said they were either 'quite/very unhappy' or they 'did not know' how they felt.

In contrast however, there was a view from peer supporters that students who were not nominated may have felt left out.

"I think some of them felt a bit left out." (Peer Supporter, school 21)

"Yes they were upset because they felt left out almost." (Peer Supporter, school 21)

However, in agreement with school leads, they also acknowledged that this may have had more to do with them missing out on two days out of school.

"But I don't feel they felt left out because they weren't getting to learn information, it was more because they weren't getting to miss out on school." (Peer Supporter, school 21)

4.1.4 Fidelity of Peer Nomination

In terms of fidelity, feedback from school leads, site leads and observation of ASSIST delivery highlighted student confusion regarding the peer nomination questionnaire which asked them to write the "tutor/year group" for the student they nominated (Case Study 1). In Scotland, the term "registration class" is used instead so students were unfamiliar with the different term. This was fed back to the DECIPHer-ASSIST team who amended the paperwork to "registration class." However, despite this correction the requirement to write both name and class for the person nominated still created a degree of conferring (shouting) as students asked one another for the class information. This is clearly at odds with the theory of peer nomination which states that the process should be done independently. It is, therefore, not surprising that observation highlighted occasions where students were not asked to write the registration class of the students they nominated. In addition, follow-up interviews with trainers and site leads noted that class detail was often abandoned opting for name only.

A high ratio of adults to students was present in all of the case study schools for peer nomination, with the number of trainers present varying between three and six depending on availability and the size of the year group. Trainers were often supported by school staff, which was important for administering the peer nomination questionnaire and behaviour management.

The ASSIST manual specifies that if the peer nomination process is undertaken in a large hall then it should be set up for examinations with each student having an individual desk. Observation highlighted that this was only possible in one school (Case Study 6). Conditions in the other schools varied between having students sitting on chairs in rows with nothing to lean on (Case studies 1 and 5), students sitting at dining tables facing each other (Case studies 3 and 4) and a large gym hall with neither chairs nor tables resulting in students sitting in groups on the floor (Case Study 2). Where exam conditions were observed, it was noted that there was minimal opportunity for pupils to discuss the nomination process with each other. Other conditions resulted in a high level of disruption with many students sitting closely together and discussing answers with each other.

A microphone was used on three occasions to help introduce the questionnaire (Case studies 3, 4 and 6) and a power point presentation was used on one occasion (Case Study 3).

4.2 Peer Recruitment

The peer recruitment meeting is when nominated students find out what they are being asked to do and decide if they want to participate or not and is, therefore, an important part of ASSIST delivery. This section will outline perceptions of how students felt to be nominated, and their understanding of their role. It will also consider fidelity of delivery.

School leads generally felt that students were excited and flattered to have been nominated, with one commenting that:

"… some of the kids who were on the ASSIST thing said that that was one of their greatest, you know, best achievements - was being selected." (School lead 4, Follow-up)

Observation of delivery in the case study schools also suggested that students were happy to take part in ASSIST. As illustrated in Table A4 (appended) student drop off between the peer recruitment meeting and the two training sessions was very low, with no drop off in schools 1 to 3. However, there was also a view, from school leads, that some students may have been reluctant to take part, mostly because they were apprehensive of trying something new. Students also reported feeling unsure about what they were being asked to do and what it would mean to be a peer supporter. For example, there was an impression that being a peer supporter would mean giving a talk to the whole school about the dangers of smoking, which was a daunting prospect:

"I thought we were gonna have to, like, speak to, like, big groups o' people. Yeah, that's, I thought, 'We need to speak at, like, an Assembly or something.'" (Peer Supporter, school 5)

Others thought that they had been singled out because trainers were under the impression that they smoked and they were going to get into trouble. However, once it was explained that their fellow students had nominated them and what the peer supporter role entailed, many students mentioned that it made them feel "proud", "important", "special" and "respected" to be "looked up to" and nominated by their peers. Many students believed that being nominated as a peer supporter had a lot to do with their level of popularity or because they were good at sports (another common indicator of popularity). Some students were surprised that they had been nominated as they had never previously perceived themselves as popular.

"Like I didn't see myself as like popular…In a way, I know it sounds really weird, but I always thought people knew them but didn't really know me." (Peer Supporter, school 21)

At the peer recruitment meeting, trainers explained that taking part in ASSIST would mean attending a two day training course and four follow-up sessions which they would be taken out of class for. This was a source of great excitement and one of the main factors in their decision to take part in the programme. School leads also commented that students would find the prospect of being away from school for two days very appealing. This was also evident in observation of the peer recruitment meeting where it was noted:

"One child puts the date in his diary as 'smoking road trip - no homework!" (School 1, observation notes)

Overall trainers regarded the operation of peer recruitment meetings as largely straightforward and noted few issues or concerns. In agreement with school leads, trainers generally perceived excitement and enthusiasm in the response of peer supporters to the prospect of attending training, particularly the fact that they will be out of school for two days.

"…the recruitment [meeting], that's great, they are just really excited at the meeting because they are hyper because they get out of school for two days, they are delighted." (Trainer, Site 1)

That said, trainers (reflecting on the peer recruitment meetings) were not always sure that peer supporters fully grasped what the role of a peer supporter would entail.

"I suppose it's exciting isn't it [being nominated]? And they are still really quite young, it's not, so they maybe don't understand the commitment they're making." (Trainer, Site 1)

Trainers also noted that some students chose not to participate in the programme and perceived that these were perhaps quieter or less engaged pupils some of whom maybe smokers themselves.

"…maybe they are smokers, maybe their parents were smokers, I've always wanted to know that, who are those ones that don't come, and sometimes you get that opportunity to speak to the teacher about it." (Trainer, Site 1)

4.2.1 Fidelity of Peer Recruitment

Observation of recruitment meetings showed a high degree of fidelity, covering each objective specified in the manual. All sites gave information about the role and responsibilities of being a peer supporter and reassurance that students were only expected to speak to students that they knew in their year.

Peer recruitment meetings tended to take place in classrooms which could become quite cramped depending on the size of the group. In two cases (Case studies 3 and 6), the student group was very large. Although these groups were split into two for the training sessions, they were grouped together for the peer recruitment meeting resulting in large and disruptive groups which were difficult for the trainers to manage. Trainers also reported feeling restricted at times in terms of the time available to deliver the peer recruitment meeting.

"Sometimes in terms of timing, you've not always got time to do loads of games, because by the time you get people out, cos they come from different classes to a set location and then back again, you've not always got the time to do that." (Trainer, Site 3)

As specified in the manual, the tables were often laid out in order to allow students to have informal discussion in groups. The manual also specifies that students should be seated in small groups. However, there were occasions where tables were set up in such a way that a larger group were seated together (Case Study 1 and 2). This caused disruption, as these groups began to dominate the session. In contrast, those in smaller groups appeared to get on with the task and engage in group discussions with minimal disruption.

Where facilities were available, information on the role and responsibilities of a peer supporter was communicated via a standard ASSIST power point presentation.

As specified in the manual, informal discussions were used to establish what the students already knew about smoking. One group (Case Study 1) were observed doing an exercise to look at the reasons why someone would start smoking and the reasons they should stop. Discussion with the trainers revealed that they decided to add this exercise so they could gauge student knowledge and awareness.

Trainers' style of delivery promoted a friendly atmosphere, keeping interactions informal and emphasising the positives of being nominated as a peer supporter. In one school, trainers encouraged students not to view them as teachers by creating and adopting nicknames, sitting amongst the students and talking about things they found interesting (music, tv, etc). Students were given paperwork for their parents/carers to complete. Great importance was placed on returning the forms and students were told they would not be able to attend the training without returning the forms. This often resulted in the students being given the forms with only a few days to return them, sometimes over the weekend.

A large part of the session was taken up by outlining the training days and what the peer supporters could expect from them. However, minimal detail was given on the follow-up sessions at this stage and in some cases, these were not mentioned at all.

4.3 Peer Supporter Training

Feedback on peer supporter training was generally positive. School leads spoke highly of content, delivery, organisation and logistics to deliver peer supporter training.

"I thought the training was phenomenal, the best I've ever seen." (School lead 8, follow-up)

On the odd occasion where something did not go to plan (bus not arriving for example), trainers were praised for their ability to deal with the situation "robustly"' and keep the training schedule on track. As this was a pilot it is not surprising that school leads, trainers and site co-ordinators observed that some venues were more suitable than others. For example, one school lead commented that for wave 1 of ASSIST, training was delivered in the Science Centre which they thought took 'centre stage' suggesting that the students were less focused because of the exciting venue. For the second wave, training was delivered in a community centre and the students were perceived to be 'more focused.'

Points for consideration from school leads focused on rewards and 'fiddle toys.' One school lead commented that trainers had given students sweets as a reward but their view was that this was not the best approach for a school that was trying to promote healthy eating. Fiddle toys were viewed as a distraction which meant students were not concentrating on the training course.

Trainers exhibited a high degree of enthusiasm and support for the training.

"I think the kids are getting a wonderful experience, a day out of school, somewhere really nice, with people who want to be there to teach them and invest time in them. So it's all win, win…" (Trainer, Site 1)

4.3.1 Style of Training

Peer supporters felt that the style of the training was an effective way of learning facts about smoking and was described as 'a fun way to get points across'." The training's emphasis on activities and games, as opposed to classroom learning, meant students were more enthused about the material and more likely to remember what they had learned.

"Like, when you're just writing stuff down, you don't really remember it but when you do it as an activity, it makes you remember it." (Peer Supporter, school 5)

Trainers also felt that the programme benefited from the interactive training style, creating an appropriate mix of activities, listening and reflection.

"…all the activities were really well done, you know, they were interactive, and they were constantly changing from areas in the room to listening to talking to, so it was all really well put together." (Trainer, Site 2)

Students felt that dropping smoking facts into conversations with their peers was a more natural way of talking about smoking and could reach students who would not normally respond to being taught about the dangers of smoking by their teachers.

"Yeah, because when a teacher says something to you, you don't really pay attention to it, like, when they say something an' you're like that, 'Alright then,' but then when your mate says something, like, you pay more attention, like, you're, 'Oh, right,' an' you actually have listened to him whereas a teacher, you just sort o' blank them an'." (Peer Supporter, school 5)

The style of the training also meant that students were encouraged to mix with those they previously may not have interacted with. Dividing the students into groups was also done in a fun, interactive way which the students responded positively to:

"Yeah, like, when you have to, like, go up in a line, like, you could work with different people... cos, like, normally all the girls would go together an' all the boys would go together. Like, we wouldn't mix together." (Peer Supporter, school 1)

"Like, it was good because you made new friends out of them." (Peer Supporter, school 1)

Peer supporters felt that the trainers worked hard to keep the activities interesting and because each activity did not last too long, it helped maintain students' enthusiasm and energy.

4.3.2 Content of Training

Some peer supporters commented that the first training day, which focused on teaching the students facts about smoking, contained a lot of information, which was difficult to take in. However, the second training day, which focused more on the practical application of the knowledge gained during the first day, was received more positively.

When asked to recall their favourite activities, two games stood out from the peer supporter training. One was the 'Ready, Steady Cook' task from day one, which students felt was a good way of communicating the chemicals in cigarettes and provided an opportunity to discuss exactly what went into making a cigarette.

"I didn't know acetone was in it." (Peer Supporter, school 17)

"I thought we were making a cake. [Laughter] Then she put rat poison in it. [Laughter]." (Peer Supporter, school 17)

The other activity was the 'Listening Train' task from day two which helped peer supporters improve their confidence and communication skills. Peer supporters also stressed the importance of interspersing the activities with games in order to keep levels of enthusiasm high.

"I think the training, because you did an activity an' then you done a game so it was, like, you done two activity an' then a game, so, instead o' messing about on loads o' activities, we got to do stuff in between so it was kinda, it made some people want to do it more." (Peer Supporter, school 17)

Trainers also noted that 'Ready Steady Cook' and the 'Listening Train' worked particularly well in terms of engaging peer supporters. One trainer commented on the 'Listening train':

"I was thinking wow, okay, because kids that age don't usually want to listen or speak to each other like that, and they absolutely adored it. You could feel the buzz in the room." (Trainer, Site 2)

However, there were some activities that they perceived to be repetitive such as "The Wind Blows", which some felt 'fell flat.'

"…it's a bit of a hit or miss because by the time that you do that, you've gone over all the stuff so you are saying to them you know, 'smoking wind blows if you think that….' but you've already dispelled that myth or you've already spoke about it." (Trainer, Site 1)

4.3.3 Relationship with the Trainers

Peer supporters felt that it was important that they could feel relaxed and build up a good relationship with the trainers. The fact that some trainers had previous experience working with children was also viewed as beneficial because they were able to speak to the students on their level. Trainers' informal approach and delivery style helped to further differentiate the training from a normal day at school which in turn made peer supporters feel more comfortable and added to the impression that they were being treated like mature, young adults as opposed to children.

Trainers also felt the benefit of building rapport with peer supporters. There was a sense that good relationships with peer supporters could facilitate the delivery of the training but also make the experience enjoyable for trainers too.

"It's quite important obviously to establish some kind of working relationship with them, so I quite like the first part of the first day just getting a bit of a laugh, getting a bit of rapport with them. Hoping that then they will sit and listen and take part without much issues with behaviour or anything like that. That's really important. I quite enjoy going on the bus for that reason as well. Because it gives me the chance to have that extra fifteen, twenty minutes with them on the bus." (Trainer, Site 1)

4.3.4 Confidence Building and Communication Skills

Peer supporters felt that taking part in the training gave them the confidence, knowledge and skills to effectively communicate the dangers of smoking to their peers; something they would not have been able to do before taking part in the training.

"Yeah, I kinda felt a little bit more confident about speaking about it. Like, it's still obviously quite a big thing to speak about and it's kinda, like, it's affecting someone's life obviously, it's affecting their health an' how they do things so it's like, you don't really want to try an' interfere as much but it helps, like, it does, you get a bit more confident about speaking to it. The training really helps." (Peer Supporter, school 4)

Peer supporters also indicated that their confidence grew as a result of getting to know their classmates better.

"You learned more and you got to know everyone better and they were really helpful." (Peer Supporter, school 17)

Trainers also observed that peer supporters' levels of confidence grew over the two training days. One trainer, however, felt that the training could do more to develop peer supporters' communication skills to help them initiate a conversation.

"The way the manual goes, I think there is an expectation that young people know a little bit about communication skills. But I think communication is getting worse with young people. I do think they communicate very differently from how we did. It's a very comprehensive programme, but I don't think the manual maybe had enough in it. How do I have a conversation?" (Trainers, Site 3)

4.3.5 Choice of School Chaperone and Behaviour Management

Peer supporters felt it was important for the school chaperone to be someone that they had a positive relationship with rather than someone who was in a more senior, authoritative position as this may affect their ability to be open and honest with the trainers.

"So it was quite awkward. And XXXX was there an'."

"Yeah. So we had to kind of behave ourselves."

"I was more nervous when she was there when, say, she wasn't. Cos one day she wasn't there for a wee bit an' we had more fun. So, like, maybe she should have just not gone next time." (Peer Supporter, school 1)

Some peer supporters found the venue for the training to be very formal which affected the way they behaved. This group associated the venue with business meetings and conferences which meant they were worried about getting into trouble for misbehaving. They also mentioned that other meetings were going on which affected their level of comfort.

Choice of training venue was also important to trainers who realised the benefits of having access to additional space (particularly outside space) for peer supporters and themselves to spend time away from training rooms.

"At lunch time they got taken out and they were allowed to use the basketball courts and football courts. The weather was nice and it made a difference - they got out." (Trainer, Site 2)

4.3.6 Fidelity of Peer Supporter Training

Observation of peer supporter training noted that students began the training with a great deal of enthusiasm and excitement, but levels of engagement varied across the two days. Students appeared very interested and attentive while listening to the trainers' introductions and contributed well to the first half of day 1. However, on both days the enthusiasm of the students dropped after lunch. The trainers noted that this was very common. Some students showed signs of disinterest (looking bored) at the end of day 1.

The contribution of students varied across the sites with a noticeable change in their confidence levels during the two days. For example, on the morning of day 1 students were reluctant to give full answers or put their hand up to offer suggestions but as the days went on their confidence levels grew and many more hands were raised at the end of day 2 with students offering much longer answers.

Venue suitability varied across the six case study schools. Three training sessions were held at local hotels (Case studies 1, 3 and 6). None of these three venues offered a self-contained outside space. The other three schools travelled to local venues which offered function rooms (Case studies 2, 4 and 5). Only one of these three venues contained an outside space (Case Study 2).

The ASSIST manual specifies that peer supporters should be given healthy foods but that it also needs to appeal to young people. Two schools (Cases 1 and 2) opted to serve foods such as hot dogs, pizza, chips and chicken nuggets in an effort to ensure that the students would eat the food as they had provided healthier options in the past which often went uneaten. The remaining schools offered options such as soup, sandwiches, wraps, fruit, yoghurt and fruit juice and the students appeared to eat this. Two schools (Case studies 1 and 2) offered prizes for those who answered questions and participated in the group discussions. These often consisted of sweets which were eaten right away.

The ASSIST manual specifies that toast and hot or cold drinks should be served at the morning break, the reason being that many students will not have had breakfast before attending the training. Two schools served breakfast at the morning break as suggested in the manual (Cases 1 and 2), however, several schools opted to serve breakfast earlier in the day as the trainers believed that since many students would not have had breakfast it was better that they had something as early as possible (Case studies 3,4,5 and 6).

Observation suggests that the overall objectives of the training were met. Trainers demonstrated an extensive knowledge of tobacco and communicated this to the students who appeared to easily digest and store this information. However, there were disruptive students and it was not clear whether these students understood what their role as a peer supporter entailed, as many were not able to recall skills or practice conversations.

4.4 Follow-up Meetings and Diaries

As detailed in Table 1 there are four follow-up sessions where trainers come back into school to meet with peer supporters to discuss progress and any issues. School lead's opinion on the number of follow-ups that should be undertaken was mixed. Some felt that four sessions were too long and became repetitive, which meant that peer supporters began to lose interest and got 'a bit fed up.' Others felt that four sessions worked well, maintained momentum and allowed peer supporters to raise questions about their role. One school lead commented that their students would have been happy to attend more follow-ups.

Feedback from students suggests that they did not feel the follow-up sessions were as useful as the training days. Several mentioned that they did not enjoy the follow-up sessions as they found them to be boring and repetitive. The availability of classroom space meant that some groups did not have the room to play games which left students feeling bored and unmotivated. However, some students mentioned that they found the follow-up sessions helpful as they provided an opportunity to refresh their knowledge and skills although these sessions tended to take place during one school period which many students felt was not long enough.

"On [the training days], we had the whole day but in that we had, like, one period to do it which was kinda taken up by coming up, getting your names an' all that. Sitting down an' getting settled an' that." (Peer Supporter, school 1)

Peers supporters suggested that the follow-up sessions could be combined into two longer sessions which more closely resembled the training days. To address the time pressures, some trainers also suggested that it may be better to combine follow-up sessions so that they may be conducted over two school periods or an afternoon. In one pilot site this approach was trialled out of necessity (the school had forgotten the follow-up had been arranged and a double session was all that could be accommodated) with mixed results. Some felt it provided more time to reflect on peer supporter experiences but others found it more difficult to maintain peer supporters' engagement over a longer session.

On a positive note peer supporters noted that the follow-up sessions offered them the opportunity to discuss any worries or concerns around message diffusion. For example, before taking part in the two day training, some mentioned that they were worried about the response they would get if they approached their peers and started talking about the dangers of smoking. They anticipated disinterest and hostility from those who smoked because they might not want to be told that what they are doing is bad for them. The follow-up sessions were believed to have been helpful in dealing with these types of issues and the trainers were on hand to reassure peer supporters that even though they might be greeted with resistance or indifference, they may still be having an effect. The follow-ups also gave the peer supporters an opportunity to go over any conversations they may have had and identify any areas they could improve upon.

"But, the one thing I liked about the follow-ups was, when you'd done your conversations and they went over them and then they tried to make them better. Like, they tried to make you more confident than you were." (Peer Supporter, school 1)

Trainer experience of follow-up sessions highlighted some challenges. One issue related to the logistics of hosting follow-up sessions in school with issues around the allocation of suitable rooms, the timely release of peer supporters from class and directing them to the correct room. Trainers reported that on arrival at some schools, staff were not aware of their visit. Problems with logistics could add further time pressures to the already short period of time allocated to follow-up sessions with trainers often feeling that they were rushing through the required tasks.

"Just getting into the school on time and getting into a decent room and getting set up and having the time to deliver it is really difficult… The attention of the peer supporters tends to be really good, but you're meant to have an hour and only get twenty minutes by the time they round up all the pupils, you get a room and you're able to get started. The follow-ups are the hardest bit." (Trainer, Site 3)

Maintaining peer supporter engagement within and between the four follow-up sessions was perceived as problematic. Trainers notice a difference in peer supporter engagement with follow-up sessions compared to that on the training days. For some this was due to follow-ups being conducted in a school environment and peer supporters' attitude and behaviour reflecting this more formal and familiar environment. Other trainers commented on the less interactive style and repetitive content of the follow-up sessions which discouraged peer supporters.

"The pupils are different. They behave differently. When you take them out of the school setting, you would think that they would behave worse, but actually most of the time, they actually behave a bit better and they are more respectful. When you are back in school with them, they kind of revert to that pupil mode and they start behaving like young people in school and try to have a laugh and trying to be difficult - I don't have a pen and I don't have this…" (Trainer, Site 3)

"I think the follow-ups, I don't like to say they fall a little bit flat, but they've had the two days training, they have had the hype of going out. And then you come into the school. I mean all the activities in the follow-ups are fine you know, but the kids' evaluation as I've said, there was too much writing, and there was too much sitting about." (Trainer, Site 1)

In contrast to the feedback from peer supporters above, some trainers felt follow -ups did not provide enough time to address peer supporters' concerns or difficulties around conversations. Trainers reported that there was no specific activity included in the follow-ups that considered conversations and that the specified activities reduced the time available to discuss peer supporters' issues.

"…we did say if anyone had any issues and some would say something, I think we could have spent more time on it, discussing what the issue was, so that everybody was hearing how you could do that…I think the follow-ups would be better being more about them and them coming to us…" (Trainer, Site 2)

4.4.1 Diaries

Peer supporters are encouraged to complete a diary to record conversations and serve as a resource for facts they could use in conversations. Diaries were intended to be used as a tool to help peer supporters remember facts and record conversations to share at follow-up sessions. Robust monitoring of the number of conversations recorded is not required (but the manual states that they should be checked at each follow-up). Diaries, therefore, were not used as a measure of message diffusion because this was never their objective. With this in mind, feedback on diaries focused on two key areas: their utility and fidelity with regard to checking at follow-up sessions.

The perception from school leads was that diaries were not being completed (students forgetting to bring them in or losing them). The suggestion was made that perhaps a paper diary was not the most conducive way for peer supporters to record conversations, with an electronic version perhaps more appealing.

Peer supporters made similar comment noting that they received their diaries at the end of the second training day and were shown how to use them. Several stated that although they found the facts listed inside to be useful conversation starters, they did not really use their diaries, even if they were having conversations with their peers. The main reason for this was simply that they did not remember to record each conversation because they would not have the diary close at hand. When they did have their diaries, many had forgotten the detail of their conversation or believed them to be too insignificant to warrant recording. This meant that it was common for peer supporters to only remember to fill in their diaries before the follow-up sessions as they knew they would be checked by the trainers. One group mentioned a girl who felt awkward when she found out she was the only one using her diary on a regular basis which led her to no longer use her diary.

"So then she used it a lot but then she was like, 'Why is no one else using them?' and we were, like, 'We all forgot,' and she was like."

"Yeah, she felt it, like, 'Why am I the only one doing it?'"

"Like there wasn't a point in doing it if no one else is gonna."

"And then last week she didn't bring it in." (Peer Supporters, school 5)

More emphasis was placed on diary keeping amongst trainers in one pilot site but all reported issues with peer supporters losing diaries or frequently forgetting or not bringing them to follow-up meetings. Corroborating the feedback of peer supporters, trainers did not perceive that poor diary keeping reflected a lack of conversations but that simply these were not being recorded in diaries.

"They are having the conversations - a lot of them don't even fill the diaries out and they lose them after day 1." (Trainers, Site 3)

4.4.2 Fidelity of Follow-up Meetings and Diaries

For some case study schools, diaries were not always checked by trainers at follow-up, which was a fidelity issue. This was also reflected in feedback from peer supporters with some under the impression that the main focus of the follow-up sessions was to check their diaries to see if they had been recording their conversations. Whereas, others felt that the diaries were hardly mentioned during the follow-ups. Trainers mentioned that they found the diaries problematic (for reasons mentioned above in (4.4.1) and in Sites 1 and 3 they decided not to place a great deal of emphasis on them. In contrast, Site 2 examined diaries at each follow-up to count conversations and allocated time for diary completion at the start of each session. This variability and confusion is perhaps a reflection of the point made earlier (3.3.1) where trainers commented that there was not enough time allocated to delivery of follow-up sessions in their training. Peer supporters appeared to retain facts from the training days, however observation of follow-up sessions suggested that it was not clear whether they understood their role in terms of preventing their peers from starting to smoke or who they should speak to. For example, follow-up observations highlighted examples of peer supporters talking to smokers and random people they would pass on the street (Case Studies 1 and 3).

Due to timetabling issues, the room allocated to the follow-up sessions was often small and cramped. Some rooms were large enough to have the tables rearranged but this was not always possible. It was common for rooms to be laid out with desks in rows facing the front or several tables grouped together. Rooms with large tables, were generally unsuitable because it often resulted in large all male groups with disruptive behaviour (Case studies 2 and 5). Students were allowed to sit where they wanted which meant boys and girls were not interacting during the follow-up sessions. In some schools (Case Studies 5 and 6), the follow-ups took place in various classrooms which left the trainers with very little time to set the room up and much of the time allocated was taken up with trying to get the students to the correct room.

Where peer supporter groups had been split into two groups for the peer supporter training, they were sometimes combined into one large group for certain follow-up sessions. For example, the first follow-up session in one school (Case Study 6) combined the groups into one very large group which was too large to manage. This resulted in many students disengaging from the material and a lot of disruptive behaviour. Likewise, for the fourth follow-up session at one school (Case Study 5) the groups were brought together, resulting in a very cramped and hot room in which the students found it difficult to concentrate.

Timetabling issues within the school could also result in difficulties communicating to students when and where they should be. For example, during one observation (Case Study 5), students were sent out of the room to find other students and let them know where they were supposed to be.

Due to the restricted time allocated to the follow-up sessions, activities often had to be cut short or be left out altogether. This was particularly true for the first follow-up which contains three activities. Trainer feedback suggested that some preferred to prioritise time to discuss examples of conversations and how the students should deal with certain situations rather than completing activities.

Table A5 (appended) indicates that school staff presence at each stage of delivery varied. For example in one school the same senior member of school staff attended all stages except one, whereas in the second school no representative from the school was present for any of the follow-up meetings. Finally in the third school the same member of staff was present throughout delivery and was a member of the pupil support team. The ASSIST manual recommends that teaching staff attend all stages. There were occasions where this caused challenges for trainers who had to manage behaviour (discussed further in Section 4.9) but also retain the informal friendliness required to be an ASSIST trainer.

The fourth and final session varied greatly in terms of thanking the students for their participation and presenting them with their certificates. For example, in one school (Case Study 1), the entire year group was gathered together for an assembly and the peer supporters were presented with their certificates in front of their peers. In other case study sites (Case Studies 2, 3 and 5), the peer supporters were presented with their certificates in front of the other peer supporters and thanked for their participation. In one case study site, (Case Study 6), the final follow-up session ended very abruptly. Several students had already been permitted to leave the session and return to their English class and the students were informed that they would receive their certificates at a later date.

4.5 Initiating Conversations Following Training

Peer supporters reported feeling apprehensive and nervous approaching their peers following receipt of training. This indicated that they may have found it difficult to initiate conversations, not necessarily about smoking, but conversations in general.

"Yeah, I don't really like starting conversations." (Peer Supporter, school 21)

Trainer feedback on peer supporters' initiating conversations was more mixed. Some peer supporters were perceived to be very confident in starting conversations but trainers were also highly aware that many other peer supporters lacked the necessary confidence for fear of being laughed at or risking offence.

"…in the main I would say no they weren't confident. They still questioned themselves just in terms of you know whether my friends are going to laugh at me. And that seemed to be something that was quite prevalent, they weren't sure if they were going to appear a geek if they suddenly started spouting all this stuff." (Trainer, Site 2)

Peer supporters who found it difficult to begin conversations felt nervous about asking for help during the follow-up sessions in case they were singled out or made to feel less able than the other students.

"Because everybody else knows what to do and you don't want to be the one that doesn't know what they are doing or anything." (Peer Supporter, school 21)

When peer supporters did manage to have conversations, these tended to be based on the facts and information gained during 'Ready, Steady, Cook' or other activities from the training days.

On smoking specifically, some peer supporters expressed concerns that by approaching their peers and telling them facts about this issue, they would come across as patronising. This was echoed by non-peer supporters who felt it was inappropriate for their peers to be giving them this type of information.

"Cos they're acting like they know better than me an' I know I'm more or less the same." (Student, school 17)

However, there was also the view that peer-to-peer conversation was more effective than having a teacher deliver a lesson on smoking.

"But, see, if ma pal says something to me, I'd probably take it more than if Miss xxxxx said something to me." (Peer Supporter, school 17)

Although peer supporters were aware that they were supposed to be having conversations with other students even if they did not smoke, many stressed that they were not having conversations with their peers because they did not smoke or because their friends had been nominated to be peer supporters.

"All my friends were pretty much at the ASSIST meetings." (Peer Supporters, school 21)

Trainers also reported that peer supporters, initially at least, perceived that they should be speaking to smokers and that this was a barrier to conversations trainers attempted to address in follow-up sessions.

"And then they go, 'oh but my friends don't smoke'. Then it was about telling them it's not just about people who are smoking, it is about stopping them smoking. They found that hard to grasp sometimes." (Trainer, Site 2)

Many students recalled having conversations with family members who smoked in an attempt to get them to stop. These conversations also gave some the confidence to try and have further conversations with their friends.

R: "Yeah, my Gran was, she only has, like, a few now so she's cut down."

Q: "Because of what you said to her?

R: "Yeah, because, she already was trying to stop but because she didn't get anything like this in school, she didn't know as much about, like, the facts so, yeah, she was, like, 'Oh, I never knew that. That's interesting.'" [Laughter] (Peer Supporter, school 5)

Trainers were also aware of peer supporters having conversations with family members and mentioned encouraging this in order to increase confidence in starting conversations in the short term.

4.6 School Contribution to ASSIST

Schools participated in the delivery of ASSIST in three ways. First, was logistics which included: being the point of contact for the trainers; organising class time and rooms for recruitment and follow-up meetings; emailing colleagues to inform them students would not be in class because of ASSIST training or follow-up meetings; administering and collecting consent forms for students to attend the two training sessions and helping trainers identify the best time of day to conduct the follow-up meetings.

Generally schools appeared to accommodate the required input to deliver ASSIST which was helped by the offer of back fill costs (but not all sites could offer this and most schools did not request it). However, it was noted that at times it was 'a bit of a challenge' to organise cover.

School lead perception of their role at baseline interview was generally the same at follow-up, which was manageable alongside their existing workload. On the rare occasion where unanticipated time was required this was due to internal school procedures e.g. one school used specific computer software to manage school trips so the school lead had to upload information to the system, even though the school was not organising it.

Second, was organising staff cover to chaperone students for the two day training. This worked better in some schools than others. For example, it was notable that the school chaperones for one case study school did not contribute to the training but instead spent the first day chatting at the back of the room (Case Study 2). As per the training manual, schools were given a briefing sheet outlining what the expectations were from chaperones. However, it was clear that interpretation of this varied amongst schools.

Trainers provided mixed feedback on the role of the chaperone and some of this was dependent upon whether or not they had experience of working with young people. Trainers with a background in youth work expressed concern that peer supporters may feel inhibited by their presence.

"They [peer supporters] think teachers are there to spy on them and grass them up for stuff…..Some of them, they maybe just didn't get on with that teacher. They just wind them up." (Trainer, Site 2)

Other trainers and particularly those with little or no experience of working with young people were happy for school staff to be present particularly with regard to supporting behaviour management but were frustrated when chaperones did not fulfil their expected role and/or showed no engagement with the programme.

"Obviously a teacher or support teacher comes with the pupils. I've had two different experiences of that. The first one was not a good experience. She wasn't supportive at all of us managing the behaviour of this really challenging group and sat outside the room, despite being asked to come in." (Trainer, Site 3)

Finally, school staff encouraged and supported students in their peer supporter role. This ranged from encouraging them to accept the role, to checking in on them once they attended training to see how they were getting on.

4.7 Scheduling of ASSIST

There were two approaches to the scheduling of ASSIST. One was to avoid having the follow-up meetings at the same time (i.e. same class) so that students were not missing too much of one subject, the other was to schedule during PSHE or PE in order to minimise the number of teaching staff that had to be notified. A learning point was raised by one school lead who explained that when they completed their first wave of ASSIST, peer nomination was on Friday afternoon which meant the students were 'excitable.' It was subsequently moved to a morning slot where the students were described as being 'calmer.' Trainers also noted that follow-up sessions scheduled after lunch-time could be affected by peer supporters experiencing a lull in energy.

It is, therefore, not surprising that school leads cautioned against timetabling ASSIST sessions over lunch breaks or after school and recommended taking advice from the school as to when was best to deliver the sessions. Trainers, also stressed the importance of working with the schools to schedule meetings in advance particularly given the range of competing activities going on in schools.

It should be emphasised that site co-ordinators and trainers were praised for the flexibility they displayed to schedule ASSIST, with one school lead commenting that they appreciated having an input to the timetabling of ASSIST.

"It definitely helped that we had a say in the time of year that we wanted to run the project. Once we agreed that we would be part of the project, it wasn't that we were then told, 'your six weeks will be those six weeks'. We had a say when we would start the project so it would fit in with our school calendar, so that helped." (School Lead 5, follow-up)

4.8 School Year of Delivery

As noted in Section 1.3 in order to reflect the age range used in the original RCT a pragmatic decision was made to deliver ASSIST in either the last term of S1 or the first term in S2. In reality this was not always possible to due school timetables. Assessing what this meant for delivery of ASSIST was a key area of interest for this process evaluation. However, as one stakeholder observed, because this was a process, not an effectiveness evaluation we do not have any evidence to show whether the varied delivery slots made a difference or not.

"I think, yeah the truth of the matter is we don't know." (Stakeholder 3)

That said, feedback from school leads did not seem to suggest there was any particular difference in terms of delivery targeting S1 or S2, with most commenting that they were given the option to choose S1 or S2 (the exception to this being site 3 schools where ASSIST was delivered to S2 year groups only). The focus of discussion was on the reasons why one year was picked over the other (i.e. S1 chosen because S2 was busy with course selection for future years, or because the school thought it would be particularly beneficial "to bond the year group"). One school lead reflected on whether S1 would have been better because they would have been more 'eager' than the S2s but overall there was a sense that the decision was based on pragmatism.

S1 students commented that because they were still getting to know people in their year group they felt it was too early to identify the most influential people in the year. The concept of nominating other students they trusted was also difficult for some which resulted in many students nominating their friends or people that they knew:

"I thought it was really hard because like you have to put people like who you trust, it's really hard for me to trust people because it depends like if I know them and that and the same probably goes with…that you've got to trust them before you tell them stuff." (Student, school 5)

This view may be a reflection of the delivery date which was January, meaning S1 students had just four months to establish friendship groups.

4.9 Behaviour Management

School lead discussions suggested that trainer management of student behaviour could be challenging at times. In the main the trainers were praised for striking a balance between maintaining discipline and establishing a rapport with students, with one school lead describing them as 'sensational' and another commenting that the trainers had 'a good approach with young people.' However there were examples where the school lead felt the behaviour of the students was not acceptable. In one school, for example, the school lead described the peer nomination process as a bit of a 'rammy' and noted that he had received negative feedback from colleagues regarding student behaviour in the follow-up sessions. He wondered if trainers should think more about the kind of atmosphere they want to create and the balance between a youth club and school atmosphere. A similar view came from another school who noted that whilst trainers should not take on the role of a teacher, they do need to have some authority.

"The trainers are really, they were really really nice an' the kids warmed to them so well, and I think they need that as well. They can't be too, they can't take the role of a teacher cos that's not the point. They have to take, they have to be that friendly person who the kids'll go to if they have a problem with smoking, cos they're saying to them, 'If you come to us' - they have to be approachable but they also have to have the authority." (School lead 9, follow-up)

In light of this it was not surprising that some school leads felt that it should be class teachers not support staff who attend the training sessions but this very much depended on the resources available within the school.

Trainer's perspectives on behaviour management was different to school leads. Despite valuing the support provided by the presence of teachers, trainers were keen to distinguish between the formality of the school environment and the informality of the training course. They were clear that teachers should not interfere in the running of the peer supporter training course specifically with regard to discipline.

"… they are just there really not to really, just as chaperones and if somebody is misbehaving or whatever, but again that is something we are capable of dealing with that you know, and I mean if it got to the stage if somebody had to be taken out." (Trainer, Site 3)

However, case study observations revealed a degree of confusion surrounding the role of the school chaperones, particularly in relation to behaviour management. In situations where the the whole group was poorly behaved it became obvious that the trainers would have liked the school chaperones to step in and speak to particular students. For example, during one observation (Case Study 2), the behaviour during day 1 of the peer supporter training was very disruptive, however, the school chaperones did not get involved. Instead, they (i.e. the school chaperone) reported the behaviour when they returned to school to the Head Teacher and some parents were called to inform them of their child's poor behaviour. Trainers would have preferred the school chaperones to have got involved on the day, rather than punishing the students retrospectively.

In addition, observation data highlighted that trainers with a background in youth working appeared more equipped to deal with disruptive behaviour from the peer supporters. Follow-up interviews with trainers also supported this view with many of those with experience of working with young people appearing largely unfazed by poor behaviour in peer supporters.

"I mean I've been working as a youth worker for twenty to twenty five years, so if somebody was getting, and I think it was excitement the behaviour, just over excited it's just a quiet chat outside the room just to say that you know it's not appropriate and you are spoiling it for others." (Trainer, Site 2)

4.10 Delivery Costs

Funding to deliver the ASSIST pilot came from the Scottish Government and NHS Boards tobacco prevention budget, with Local Authority partners providing in-kind resources only. At the time of writing this report the Scottish Government had made no commitment to fund delivery of ASSIST beyond the pilot. Setting the budget for each site varied depending on their delivery model and the number of schools they planned to work with. All three pilot sites praised staff at DECIPHer-ASSIST for their help to cost delivery of the programme, with the only reports of unforeseen expenditure for things like post-it notes, pens, coloured paper. As would be expected with any pilot, cuts and saving were made as the pilot progressed - preferred venues, coach hire, catering were used and staff time to prepare for delivery became streamlined the more experienced they became.

Stakeholders commented that ASSIST was not 'the cheapest programme to deliver' but also added that not only was it an investment to potentially protect young people from smoking harm it also equipped them with a skill set that will serve them well in the future.

Between August 2014 - June 2017, 72 cycles of ASSIST were delivered or confirmed for delivery in 2017 and included in the cost analysis. The average cost to deliver one cycle of ASSIST ranged from £8,939 to £11,357. The actual total delivery cost for all three sites was £674,360 of which 20% (£136,800) was for the license fee. This equates to a cost per cycle of £9366. Costs include the license fee and VAT. However, this cost can only be regarded as a ball-park figure, as circumstances differed between the sites in terms of the size of the school and location (e.g. costs for training in rural areas will likely be higher than for urban schools). Therefore it is not possible to rely heavily on the average experience by these three sites when planning future implementation of ASSIST.