Evaluation of the 'You First' Programme for Young Parents - Appendices

APPENDIX 4: METHODOLOGY

This appendix provides further details of the methods used in the evaluation.

Research with parents who attended You First

At the outset of each programme, facilitators gave all parents an information sheet informing them that the evaluation was taking place and requesting their participation. They were informed that they would be asked to complete a questionnaire at the end of the programme and that they may also be contacted to take part in the qualitative research. All parents who were willing to take part completed a consent form which was returned to Ipsos MORI. Table 2.1 in the main body of the report summaries the research undertaken with parents.

Quantitative research

All parents who finished a You First programme were asked to complete a paper questionnaire covering their experience of the programme (Appendix 6). The questionnaire gathered views on what they liked/didn't like about various aspects of the programme and sought suggestions for improvements. They were given the questionnaire at the final group session of You First and were asked to complete it at home and return it to the programme facilitators, in a sealed envelope, at the time of the final home visit. The questionnaire was designed with the intention that it could be used by You First facilitators in the future self-evaluation of the programme. Forty-nine parents, across the nine programmes, completed and returned questionnaires. In order that Barnardo's could measure parents' progress in relation to the programme outcomes, parents also completed a self-assessment questionnaire at the beginning, middle and end of the programme. This did not form part of the evaluation (see paragraph 1.11 for further details).

Qualitative research

Nineteen parents who had completed a You First programme were also selected to take part in the qualitative element of the evaluation. The qualitative research sought to establish the benefits to parents of attending You First and to gather their views on the content and delivery of the programme, including any suggestions for improvements. Table 1 shows the number selected to take part during each phase and the specific method used.

The depth interviews and the focus group were conducted soon after the group sessions had finished and the final home visits had been conducted. The You first facilitators provided us with details of all parents who had consented to take part in the research. Participants were then selected by the evaluation team with the aim of covering a wide range of circumstances. This included: a range of programmes (interviews or focus groups were conducted with parents who attended 8 of the 9 programmes including the two programmes that ran without a financial incentive); a range of demographic characteristics (age of parent, age of baby, living arrangements and work status); and a spread of parents in terms of level of attendance at the programme. The balance of interviews was skewed towards Phase 1 of the pilot in order to provide more early feedback. The impact of recommended changes to the programme was explored in the interviews conducted in the subsequent phases of the pilot.

During Phase 1, six depth interviews and one focus group (containing five participants who had attended the same programme) were conducted. This mixed method approach was used in order to explore which method would be most effective in addressing the research objectives. As anticipated, both methods provided useful, and complementary, data. However, it was felt that it would be more effective to use individual depth interviews in Phases 2 and 3. In part, this was to allow us to interview parents across a greater number of programmes.

Three parents, who had participated in the qualitative research conducted at the end of Phase 1, also took part in a face-to-face follow-up depth interview. These interviews were conducted around five months after the programmes had finished and after the programme reunions had taken place (reunions happened around three months after the programme had finished and involved the facilitators and the parents and their babies meeting up for lunch and chatting about how things had gone since they finished You First). The purpose of these follow-up interviews was to get some indication of the medium term outcomes of the programme by establishing, for example, whether parents were accessing other sources of support and whether they were they still in touch with others in the group. The interviews also allowed parents to talk about You First, having had some time to reflect on it. Participants for these interviews were carefully selected, on the basis of what they said in the initial interviews, with the aim of covering a range of experiences. For example, one parent envisaged she would stay in touch with others in the group and seemed interested in attending other groups, but lacking in confidence to do so, and one parent had already attended other groups but had not formed close friendships with others in her You First group.

To understand the reasons why some parents had stopped attending You First before the programme ended, three additional face-to-face depth interviews were undertaken with such parents. The facilitators provided the research team with details of parents who had not attended You First for several weeks and who had not informed the facilitators that they did not intend to return. The three parents selected all attended different You First programmes.

All qualitative research with parents was conducted using discussion guides designed by the research team and reviewed and approved by the Scottish Government project manager (Appendix 7). All depth interviews were conducted in participants' homes and lasted around 45-60 minutes. The focus group was conducted in the same venue as the You First sessions had been held and lasted 90 minutes. To thank them for giving up their time, all parents who took part in the qualitative research received £20 in cash.

Attendance and incentive payment data

The research team were also provided with data on programme attendance and, where applicable, whether or not parents chose to defer their weekly £20 financial incentive until the end of the programme (they were encouraged to defer the full amount but they could choose to receive it every week if they wished). Analysis of this data is provided in Sections 8.1 and 3.38 respectively.

Self assessment

In order to measure parents' progress in relation to the programme outcomes, the You First management staff designed a self-assessment questionnaire for parents to complete at the beginning, middle and end of the programme. In advance of Phase 2, the Ipsos MORI research team worked with the programme facilitators to revise the questionnaire following their acknowledgement that there was scope for it to be improved (Appendix 8). As well as providing a measure of parents' progress, facilitators found that the self-assessment exercise was a useful way of stimulating one-to-one discussions with parents about their progress and their plans for the future. As the self-assessment questionnaires were not completed independently by parents, and because they were primarily a tool designed by You First management staff to meet their monitoring requirements, as opposed to a tool designed to inform the evaluation, analysis of these is not included in this report.

Research with professionals

Table 2.2, in the main body of the report, details the research undertaken with professionals involved with You First.

Qualitative research

Across the three phases of the pilot, qualitative research was undertaken with local stakeholders who had been involved in various aspects of the programme, including the set up of the programme in their local area, the referral process and the delivery of sessions at You First. The aim of this element of the research was to understand views and experiences of the programme from the range of professional perspectives and to explore how the programme fitted in with, and linked to, existing services in each area. The You First facilitators provided the research team with details of the key local stakeholders. Participants were then selected with the aim of covering a range in relation to local authority area and type of involvement with You First. A combination of face-to-face and telephone interviews was used. With the exception of one which took place in a café, all face-to-face interviews took place in participants' offices. Interviews lasted, on average, between 45 and 60 minutes and, as with all interviews conducted with professionals, were conducted using discussion guides designed by the research team and reviewed and approved by the Scottish Government project manager (Appendix 9).

It was also important to understand how the programme was working from the perspective of Barnardo's staff involved in its design, management and delivery. Initial interviews with programme managers and facilitators were undertaken during Phase 1. These interviews served to explore how they felt the pilot was working so far, as well as to provide factual information about the management and delivery of the programme. The interview conducted with the You First Senior Manager during Phase 3 was used to obtain her perspective on the success of the pilot and to discuss her thoughts on the future of You First. These interviews took place in the Ipsos MORI or Barnardo's offices and lasted around 60 minutes, with the exception of the paired depth interview conducted with facilitators during Phase 1, which lasted around three hours (split across two days).

In addition to depth interviews conducted with You First staff, sessions were held with the You First facilitators after each phase of the evaluation. These lasted around two hours and were used to feed back the research findings to the facilitators and to discuss recommendations for the delivery of the programme in subsequent phases of the pilot and beyond.

Quantitative research

At the end of Phase 1, all professionals who had presented at a You First session were sent a paper questionnaire (Appendix 10). The purpose of this exercise was to understand their experience of presenting, including whether they felt they had been provided with enough information in advance, whether they felt that the session was useful for participants, whether any You First participants had since engaged with their service and whether they would be willing to be involved with You First again. Questionnaires were sent to 13 contributors and 11 were completed and returned. This questionnaire was primarily developed to provide You First facilitators with a tool they could use in the future self-evaluation of the programme. For this reason, the exercise was not repeated in subsequent phases of the pilot.

During Phase 3 of the pilot, the research team felt that it would be useful to have more information on the number of parents who were eligible for the programme but did not attend. In order to capture this information, all health visitors who had been involved in referring parents to You First were sent a paper questionnaire (Appendix 11). The questionnaire covered: whether the programme was offered to all eligible parents, the number of parents who were offered the programme but declined, and the reasons for this, and health visitors' views on the eligibility criteria for the programme. Nine of the 12 questionnaires sent out were completed and returned.