Early Adopter Communities: Process Evaluation

Appendix E: Discussion guides

Discussion guide for families

Initial notes

Interviewers to review project information and screening questionnaire, and pre-populate guide with any relevant information ahead of the interview.

Section 1: Introduction and consent (5 min)

Aim: to make sure we gain informed consent from participant before taking part including consent for recording.

Thank participants for taking part & introduce yourself and Ipsos.

Check that they received and if they have read the information sheet (if they haven't, explain that you will go through the key points now and share a hard copy with them for them to have for reference).

• Explain purpose of this interview: The Scottish Government has asked us at Ipsos to get feedback from families about their experiences of using (for Glasgow this may be partially funded) funded out of school childcare. We want to understand how well it's running, including what works well and what could be better. We will not be asking about how the project has made a difference for your family in this conversation, although you are welcome to share this. The main thing we want to find out is how well the project is running day-to-day, for example application processes or how projects communicate with families etc.
• Provide reassurances of anonymity and confidentiality: It will not be possible for the Scottish Government or anyone else to know who took part in the research. No identifying information about individuals or families (e.g. names or contact details) will be passed on to anyone outside the research team (me and my colleagues at Ipsos).
• We will be writing a report for the Scottish Government that will be published and we will include some quotes to illustrate what people have told us, but these would be anonymous.
• Emphasise that taking part is completely voluntary: they don't have to answer any questions they don't want to answer, can take a break at any time and can decide to stop the interview at any point.
• Remind that they are free to change their mind and decide not to take part at any time before or during the interview, or after the interview until the findings have been written up.

Remind participant of the following:

• The interview will last around an hour. (If joint interview, explain that the child will only need to stay for the first part – around 10/15 minutes).
• They will receive a thank you voucher/payment which you will sort out at the end of the discussion.
• There are no right or wrong answers. It's really useful to hear what could have been done better as well as what went well.
• Check if participant has any questions.

Note: Collecting child consent is important here as we will so far have only communicated with the parent/carer. Spend time making sure child understands what the interview is about and what we/the Scottish Government will do with the information.

• Request permission to record interview. Explain that this is for transcription and analysis purposes and that recordings will not be shared outside the research team at Ipsos. If participant does not give permission, interviewer to take detailed notes.

That's recording us now. Could I quickly ask you to confirm for the recording that you are happy to take part based on the information in the information sheet, and that we just discussed?

Section 2 - Background (3-5 min)

Aim: to build rapport and gain an initial understanding of how they use childcare services

To start with, could you tell me a bit about yourself/yourselves?

• Where do you live at the moment?
• Who do you live with?
• What do you do on a typical day?
• (For children and young people) What kinds of things do you like doing?

It would be great to hear a bit about the out of school childcare services that you are using. You go to [project] is that right? Can you tell me a bit about what that involves?

• Where is it?
• When and how often/how long attending for? Is that the same every week?
  • Before school (breakfast)? After school? Holidays?
• How long have they been using it?
• How do they get there?
• Is there food provided?

Section 3 – Child experiences (8 minutes)

Aim: to understand what is working well or less well from the perspective of children attending the service

I have some questions for [child] about what it's like at [project].

• Do you like going to [project]?(Explore why/why not).
• If time, possible probes around children's charter principles: fairness, happiness, fun
• What do you like about going to [project]? (What is good about it? What is your favourite thing about going?)
• What kind of activities do you do there?
• How many different activities are there?
• Do you get to choose what you want to do?
• Do you like the activities there?
• What are the people who work there like?
• What kind of snacks or food do they have?
• Do you like/eat the food? (If no: what would make them want to?)
• What time do you get food? Is that a good time?
• If you could change anything about [project], what would you change? (How could it be improved?)
• Have the people who work there asked what you think about [it] before? (Did you write anything down about what you like to do at [project]?)
• If yes: When was that? (was it before they started attending?)
  How did that work?

Thank child and explain that they can now leave. The rest of the interview will take place with the parent/carer.

Section 4 – Engaging with EAC services (10 mins)

Aim: To gather information on the activities in the theory of change which relate to initial engagement with EAC services: promotion / referrals / application & registration

I'd like to ask about when you first heard about [the childcare service/the opportunity for funded childcare].

AC1: Promotion & engagement with families and/or stakeholders

First of all, when and how did you hear about the [service/funding]?

• What were their first thoughts when they first heard about it?
• Did they think it was the sort of thing that would they/their children would feel able to attend/engage with? Why/why not?
• What was it that appealed to them?
• Can they think of a better way for families to hear about things like this?
• If participant mentions they were specifically targeted for support: We know that sometimes parents feel uncomfortable with using funded childcare services. Is that something you've experienced?

AC2: Establish referral pathways

If referred by somebody: I wanted to ask a little more about the fact that you were referred to [project/funding] by [organisation/person].

• By whom? How did this work?
• What did they say about it?
• Was it a smooth process?
• What worked well?
• How, if at all, could the referral process have been better?
• How important, if at all, do you think the referral process was in your decision to use [project[?

AC3: Application process

When you decided you would like to use [project], how did that work? Did you fill in an application form for the service/funding?

• How did you feel about having to go through an application process?
• How easy was it to apply for a space/for the funding?
• o If filled in a form: Was the form easy to complete? (Potential prompt – any questions that were difficult to answer or you didn't want to answer? Anything off-putting about it?)
• How did you submit your application?
• o Who did you submit it to?
• o How long did it take for you to hear back?
• Overall – what worked well about the application process?
  (What would you want to stay the same?)
• How, if at all, could the application process have been better?
  (What would you want to change?)

AC4: Registration process

Once you knew you had been given a funded place, were there any more steps you needed to take in order to register your child for the service?

• If yes, explore fully and how this worked from their perspective.

Section 5 – Views on service delivery (10 minutes)

Aim: To gather information on the activities in the theory of change which relate to initial service delivery: day-to-day practicalities/accessibility/further support

AC5: Delivery of a range of activities (inc. active play & skills-based activities) & food as part of a childcare offer that meets the needs of target families – free for those on low incomes

Overall, how easy or difficult has the service been for your family to attend [project]?

• Timings:
  • Are the timings convenient? (do they fit with other commitments?)
  • Are they flexible? Is this important?
• Travel to activities:
  • How do they get there – walk/public transport/car?
  • How long does it take?
  • How easy it to get there?
  • Is there support provided to help your family travel to/from the project?
  • Is any transport provided?
  • Is the cost of transport covered?
  • If no: is this something you feel would help you?
  • Is there anything that would make it easier for your family to get to the project?
• Cost:
  • Does it cost them anything to attend? (directly or indirectly)
  • Does this impact on how much they use the service?
• General barriers:
  • Is there anything else that makes it difficult/can make it difficult to attend?
  • Is there anything that would help to make that easier?
  • Is there anything that normally makes it difficult to access after school childcare/that you expected might be an issue but turned out not to be? E.g. additional support needs, not speaking English as first language etc.

If child has spoken to us already: I know I've already spoken to [child], but I wanted to ask you…

Overall, how well do you think the service works for your child?

• Are the activities suited to their needs?
• Is the space suited to their needs?
• What in particular works well for them?
• What, if anything, works less well?

AC6: Delivery of family support (where appropriate)

As well as childcare, some services offer wider family support such as advice on a range of things like financial advice or help with employability. Is this something you have had access to as part of [service]?

If yes:

• What kind of family support have you received?
• When/how often?
• Who delivered this?
• What worked well about this?
• What, if anything, could be improved about the family support offer?
• Is there anything else that it would be helpful to have support around?

If no:

• Do you feel you/your family need any support like this at the moment?
• How do you think it would be to be able to access it via [project]?

Some services are able to refer you to other organisations that will provide this type of support. Is this something you have experienced at [project]?

If yes:

• How did this work? Was it a smooth process?
• Was the support useful to you?
• Could anything about the experience have been improved?

Section 4 – Service design (10 min)

Aim: To understand if parents/carers took part in any co-design or evaluation activities and their views on this.

AC8: Ongoing codesign of provision with families/partners

Thinking back now to before you attended the service, were you asked for your views on how it should be set up?

If yes:

• When was this?
• In what format? How did this work?
• Was your child asked for their views when the service was being designed?
• Do you see your views reflected in the design of the service now?
• What worked well about this process?
• What could have been improved?
• How important do you think it is for parents and children to be involved in designing services/projects like this one?

If no:

• Is this something you would have liked to take part in?
• Do you think that if you had been part of something like this, where parents are involved in the design of childcare services, that [service] would look different?
• What would you have said was most important to you?
• How important do you think it is for parents and children to be involved in designing services/projects like this one?

AC17: Collect operational data / AC18: Collect & analyse monitoring data

Have the staff at [project] asked for any additional information from you after you registered for the service, for example through filling in a form?

• If yes: We know not all parents want to share information or don't have time. When projects have asked you to do this, is this something you have shared with them? (Probe on why/why not)
• If yes: what kind of information have they been asking for?
  • Probe on whether this has been information about them/their family, or sharing their views about the project
• When did this happen and how often: (Is it ongoing?)
• What format was this – survey/feedback form, verbal discussions etc?
• How did you feel about being asked to share this information?
• How would you ideally prefer to be asked about your views on the project?
• Did you notice any changes made to the service as a result of your feedback?

AC9: Align with other local initiatives

We know there are other childcare services or family support services in [local authority]. Have you used any of these apart from [project]?

• If yes: Which ones & when (ongoing?)
• How does [project] compare to that service?
  • Is it helpful for services to take a similar approach to their processes, like application forms or how they communicate?
• Do you know if [that service] and [EAC project] worked together to support you?
  • For example, if you gave permission for this, were they able to share information about your family to avoid you having to repeatedly answer questions about this?
• Do you think that [this project] fits in well alongside these services?
• Do you feel fully supported? Is there anything else you would like support with? (i.e. are there any gaps)

Section 4 – Summary (3 min)

Aim: To wrap up the interview and understand what stands out to participants.

Overall, how would you summarise your / your family's experience of the project so far?

• Do you think that there is anything about your personal experiences, such as communities you're part of, or particular experiences you've had in your life, or your background, which makes a difference to your experience of using childcare?
• Of all the things we've talked about, what is it that [project] does best?
• And if there was just one thing that you could change about [service], what would that be?
• How could people organising similar projects in future encourage more children/families like yours to come along?
• Any final thoughts/feedback they think it would be useful for us to know?

Section 7: Wrap up (2 min) (1 hour total)

That's everything I wanted to ask you today, thank you very much for taking the time to talk to me today. Is there anything we haven't covered that you would like to say something about?

Thanks for speaking with me today, I really appreciate it. I'll stop the recording now[stop recording].

Just to remind you, we'll be writing a report on our findings. It'll be published on the Scottish Government website, so you'll be able to search for it and read it if you're interested. If you would like us to email you a link to the report when it's published we would be happy to do that. Is that something you would be like us to do?

If yes:In that case we may keep your contact details a little longer than usual, until the report is published. We will only use them to share the link to the report with you, and then we will securely delete them from our systems.

That's everything from me. Do you have any questions before we finish?

Offer to share a printed or soft copy of sources of support leaflet with participant.

Thank and close.

Discussion guide for project leads

Initial notes

Interviewers to pre-populate with information (can be brief bullets) on the EAC provision

Section 1: Introduction and consent (5 mins)

Aim: to make sure we gain informed consent from participant before taking part including consent for recording.

Thank participant for taking part and for their involvement in the evaluation so far.

Check that they received and if they have read the information sheet (if they haven't, explain that you will go through the key points now and share a hard copy with them for them to have for reference).

Explain purpose of this interview – to inform the process evaluation. We will be discussing project processes and looking to understand how implementation has gone to date, including what has been working well and less well. We will not be discussing the impact of the project at this stage.

We will use the standard local Theory of Change that I sent you in advance of this interview to guide this interview. However, there will still be scope to cover the specifics of your project.

As you know, we are also speaking to children and families who have attended the projects as well as project stakeholders. Once we have spoken to everyone, we will write a process evaluation report and submit it to the Scottish Government.

• Explain anonymity and confidentiality: we will use the interviews to inform the findings of the process evaluation. They will be used alongside the findings of the interviews with project stakeholders, interviews with families and the analysis of monitoring data to write a report that will be published by the Scottish Government. In the main, quotes included in the report will be anonymised. However, where areas have unique experiences, it may be helpful to identify areas and attribute quotes accordingly. Therefore, it is possible that some parts of the report may identify something you've said. There is also a chance that some parts of the report may be identifying (even if it you are not quoted directly) due to the small number of people in relevant roles.
• After the interview, we will check if there is anything that you would not like to be quoted in the report. We will also share with you any elements of the draft report we feel may be identifying, before sharing it with the Scottish Government. We can remove identifying information/quotes upon request. However, it will not be possible to change the overall findings of the report.
• Remind participant that the interview will last around an hour.
• Remind participant that there are no right or wrong answers. The purpose of the process evaluation is learning, to inform ongoing design and delivery, and not to find out if any particular projects are more or less successful.
• Emphasise that taking part is completely voluntary – they don't have to answer any questions they don't want to answer and can decide to stop the interview at any point.
• Remind that they are free to change their mind and decide not to take part at any time before or during the interview, or after the interview until the findings have been written up.
• Check if participant has any questions (offer to go through information sheet if necessary).
• Request permission to record interview. Explain that this is for transcription and analysis purposes and that recordings will not be shared outside the research team at Ipsos. If participant does not consent, researcher to take detailed notes.

That's recording us now. Could I quickly ask you to confirm for the recording that you are happy to take part based on the information we just discussed?

Section 2 – Recap on operation and general overview of implementation (5 mins) (10 mins total)

Aim: to get an overall sense of how implementation has gone

Through the previous discussions we've had, and the Theory of Change workshop, I feel I have a good understanding of how the project is delivered. Before we go any further, I wanted to check if there have been any recent changes to this or if things are still happening in broadly the same way?

If no: continue

If yes: ask them to explain changes and rationale behind these.

We'll talk in detail about various aspects of the project delivery as we go through the interview. However, to begin, it would be useful to get a sense of how you feel delivery is going overall?

• What are you most pleased with?
• And is there anything that hasn't gone so well or has been particularly challenging?

How close would you say the current delivery model is to what you proposed at the outset?

• What are the reasons for any differences?

Section 3 – Design and set up of provision (10 mins) (20 mins total)

Aim: To gather information on the activities in the theory of change which are in the 'System level delivery' section and relate to the initial design and set up of provision

I'm now going to talk through some of the activities in the Theory of Change that relate to system level delivery. We'll start with those which cover the design of the provision. These are all activities listed in the standard local Theory of Change so, if any don't apply to your project, we can just skip past them. We'll go through each in turn now.

AC6: Scoping/mapping exercises

First of all, can you talk through any of the scoping or mapping exercises you conducted in the design stages of the project?

• How did you decide how to approach this?
• Who was involved?
• [If not already covered] How did you decide on which specific area/s within the LA to focus on?
• Did any scoping work focus on understanding the needs of groups that are typically harder to engage with? (E.g. minority ethnic families or young mothers)
• What worked well?
• How did the process serve to inform the design of the provision?
• What were the challenges?
• Did anything help with these challenges/Could anything have helped?
• Is any work still ongoing?
• What would you do differently if you were to do it again?

AC8: Align with other local initiatives

And did any of the initial scoping work or any ongoing partnership work have a focus on ensuring the provision aligns with other local initiatives? What did that involve?

• How did you decide how to approach this?
• Who was involved?
• What worked well?
• Did it inform the design of the provision?
• What were the challenges?
• Did anything help with these challenges/Could anything have helped?
• What would you do differently if you were to do it again?

AC7: Ongoing codesign of provision with families/partners

And similarly, in relation to codesign of provision with families and/or partners, what work have you done to date?

• How did you decide how to approach this?
• Who was involved?
• How easy was it to engage families in this process? What helped?
• What worked well?
• How did it serve to inform the design of the provision?
• What were the challenges?
• Did anything help with these challenges/Could anything have helped?
• Is any of this work still ongoing?
• Do you plan to do anything different in future?

Are you aware of the Children's Charter?

If yes: To what extent have you used the Children's Charter when designing the provision? Probe fully

AC9: Procurement/distribute funding to childcare services

And, finally in relation to the initial design, how did you approach the procurement of childcare services and the distribution of funding?

• How did you decide how to approach this?
• Who was involved?
• What worked well?
• What were the challenges?
• Did anything help with these challenges/Could anything have helped?
• Do you plan to do anything different in future?

Section 4 – Direct delivery for families (15 mins) (35 mins total)

Aim: To gather information on the activities in the theory of change which are in the 'Direct delivery for families' section

I'm now going to talk through the activities in the Theory of Change that relate to direct delivery for families. Again, we'll just focus on those that apply to your project. We'll go through each in turn now.

AC2: Establish referral pathways

First, can you remind me of all the ways in which families are referred to the provision?Probe on all referral pathways

• Do these differ by provider or by different family types?
• Has this changed at all in the past 6 months or so? If so, how?
• How did you decide on your approach to referrals?

In your view, to what extent are the referral pathways supporting a smooth and easy experience for families?

• If more than one pathway: does this differ across the different pathways?
• What has worked well?
• What have the challenges been?
• Have you any plans to change the processes?

AC1a: Promotion & engagement with families

• And, aside from being referred, are there other ways in which families hear about the project?
• Have these changed at all recently?
• How did you decide how to approach this?
• What's currently working well? Which ones are most effective?
• What challenges have there been?
• Did anything help with these challenges/Could anything have helped?

Overall, how well do you think the referral and engagement processes are working to reach target families?

• What are the key barriers to reaching and engaging families?
• Are there any groups that are harder to reach or engage with?
  • Is there anything that has helped reach these groups in particular?
  • Are there any groups you are still missing?
• Are there any other general barriers to engaging families?
  • Have you been able to mitigate these barriers?

Do you think there are issues of embarrassment or stigma for children and young people or families attending targeted services like these?

• Is there anything you have found makes a difference/works well in relation to this?

And how well attended would you say the provision has been, both in terms of the initial engagement of families and sustaining attendance?

• How does this compare with any targets / expectations you had?
• Is the project better attended by some target groups than others?
• How does it vary by different service/providers?
• What do you think helps to facilitate sustained attendance?
• What have challenges been in terms of initial and sustained attendance?
• Did anything help with these challenges/Could anything have helped?

To what extent do you think the design of the provision has impacted on attendance?

• What elements in particular?
• Does it vary by different providers?

AC3: Application/registration processes

Moving on now to the application or registration processes. Once engaged or referred, what do families have to do to apply or register for a place?

• If more than one provider: does this differ across the different providers?
• How did you decide how to design the application/registration processes?
• What has worked well? (for families/providers/leads)
• What have the challenges been? (for families/providers/leads)
  [The key thing here is to understand whether they think there are many families who are referred/engaged but then end up not taking up places - and why?]
• Have you any plans to change the processes?

AC1b: Promotion & engagement with stakeholders

We'll talk about partnership working in more detail later but, for now, could you tell me a bit about how the project was promoted to and how you engaged with key stakeholders at the outset?

• How did you decide how to approach this?
• Who was responsible for this?
• How well did it work during the initial set up?
• Is there still ongoing work in engaging more partners?
• What's currently working well?
• What are the key barriers to engaging with stakeholders?
• How important is it to engage with stakeholders? What are the (potential) benefits?

AC4: Delivery of a range of activities (inc. active play & skills-based activities) & food as part of a childcare offer that meets the needs of target families – free for those on low incomes

And thinking now about the actual delivery of the childcare element to families. For each of the following aspects, what has worked well in terms of delivery and what have the challenges been? We can skip any that are not so relevant to your project:

• Before school provision
• After school provision
• ASN specific provision
• Holiday provision
• Food provision
• Transport

Are you also offering wider family support to your families at the moment?

• What types of support? And what types are accessed most by families?
• Who is responsible for this?
• Has it changed at all recently?
• How important do you feel it is to offer this alongside the childcare provision?
• What's currently working well?
• What are the challenges of delivering family support as part of the provision?

Section 5 – Partnership working, governance and knowledge sharing (15 mins) (50 mins total)

Aim: To gather information on the activities in the theory of change which are in the 'System level delivery' section and relate to the initial design and set up of provision.

Note: SG particularly interested in the role of schools and headteachers.

AC11: Close partnership working

Now moving on to think about the key providers and other partners you've been working with. We talked through these at the theory of change workshop so I think I have a good awareness of who they are and their roles. I wanted to check, however, whether these partnerships existed before the EAC was developed or whether any of them are new?

• If any new: how did you go about forming these relationships?

What are the main ways in which you communicate with partners?Probe:

• Inter-agency meetings
• Face-to-face, email, phone
• How frequent does contact tend to be?
• How did you decide on this approach?

What is working well in terms of partnership working?

• What have been the benefits of any new partnerships?
• Have any existing partner relationships been strengthened? What, if any benefits has this brought?

And what challenges have there been?

• What are the barriers to engaging with partners?
  • Any partners they would like to work with but not been able to?
• Have you been able to mitigate these?
• What has helped?

To what extent do partners share the same vision for the EAC?

• What has helped to get them on board?
• Have there been any barriers or challenges?

If more than one childcare provider, ask: What are the benefits and challenges of working with more than one provider?Probe:

• Cultural
• Regulatory
• Operational

AC12: Stakeholder engagement - make the case' for the programme

And thinking a bit more widely than the partners you work directly with, have you done any other stakeholder engagement which advocates for the EAC project, or in other words 'make the case' for the project?

• If yes: Which stakeholders? What has the reaction been?
• How did you decide how to approach this?
• What has worked well?
• What have the challenges been?
• How important is it that you do this as part of your role?
  • What is the potential impact of this type of engagement?
• If no: is this something you would consider doing in future?
  • With which stakeholders?
  • What is the potential impact of this type of engagement?

AC10: Project governance (internal meetings & multi-agency meetings, regulatory process etc)

Changing the subject slightly, we've talked a bit about this in relation to partnership working but, thinking more generally, how well has the governance of the project been working (for example, things like the way in which the project is managed and staffed)?

• How did you decide on the governance arrangements?
• What has worked well?
• What have the challenges been?

AC14: Collaborate with other EACs e.g. sharing challenges, learning, best practice/ AC19: Share learning with SG on an ongoing basis as part of feedback loop

And moving on now to think about collaboration and the sharing of learning. Thinking first about other EACs, what has happened so far in terms of any sharing of challenges and learning?

• What has worked well?
• What has facilitated this
• What have the challenges been?
• What has the impact on your project been of any learnings from other EACs?

And could you tell me a bit about how you and the Scottish Government have been working together to facilitate the ongoing sharing of learning and best practice?

• How would you describe that relationship generally?
• What has worked well?
• What has facilitated this?
• What have the challenges been?
• Is there any further guidance or support the Scottish Government could provide that would be helpful to you? (e.g. relating to governance, development, co-design etc as well as provision.)

Section 7 – Training and workforce development (5 mins) (55 mins total)

AC15: Ongoing training for providers/other local actors/ AC16: Work with partners to help develop childcare workforce

We're coming towards the end now and the final part of the theory of change we need to cover is around activities on training and workforce development. So, first of all, what training has there been for providers other local stakeholders?

• How were training needs identified?
• Who has been involved in the delivery of training?
• What has worked well?
• What have the challenges been?
• What has the impact of the training been?
• Are there any unmet training needs among staff? Skills gaps?

And have you been involved in any other work to develop or broaden the local childcare workforce?Probe for details

If time: in your experience, are many parents interested in working in childcare? What might make this appealing/not appealing to parents?

Potential scalability across processes:

When considering increasing the scale of the EAC in [the local authority], what would be the key barriers and enablers to this?

What key inputs would be required in order to increase the scale of the EAC work?

Section 8 – Summary & thoughts on improvement (4 mins) (59 mins total)

Aim: summarise thoughts and explore suggestions for improvement

We're coming to the end of the interview now. I'd like to spend the last few minutes summarising your thoughts and discussing ways to improve the programmes.

So to summarise:

• What aspects of the programme do you feel are working well?
• And what aspects do you feel are not working so well?
• What are the priorities in terms of making any improvements?
• If you were speaking to a new EAC what about to set up, what are the main pieces of advice you'd give them?

Section 8: Wrap up (1 min) (1 hour total)

That's everything I wanted to ask you today, thank you very much for taking the time to talk to me today. Is there anything we haven't covered that you would like to say something about?

Thanks for speaking with me today, I really appreciate it. I'll stop the recording now[stop recording].

Now the interview has finished are there any comments you have made that you would like to not be included as direct quotes in the report? You can also get back in touch with us later if you think of anything you'd like not to be included. We'll offer you the chance to review any sections of the draft report we feel may be identifying before submitting it to Scottish Government.

Nb: make note of comments not wanting to be included below or if appropriate mark in guide above.

A reminder of the next steps for this evaluation work:

• Once interviews completed, Ipsos to write up report over the next couple of months.
• Evaluability assessment to be completed by date.
• This early process evaluation will be published later this year. If you would like us to send you a link to the published report, we would be happy to do that. Is that something you would like us to do?
• If yes, make a note in rec spreadsheet.

If time:

The Scottish Government are exploring the best ways to communicate findings from this evaluation work back to EACs. Do you have any views on what would be a good way to do this?(What would be useful for you to know?)

Thank and close.

Discussion guide for stakeholders

Initial notes

Interviewers to pre-populate with information (can be brief bullets) on the EAC provision

Section 1: Introduction and consent (5 mins)
Aim: to make sure we gain informed consent from participant before taking part including consent for recording.

Thank participants for taking part & introduce yourself and Ipsos.

Check that they received and if they have read the information sheet (if they haven't, explain that you will go through the key points now and share a hard copy with them for them to have for reference).

Explain purpose of this interview – to inform the process evaluation. This means we will be discussing projects' processes, i.e. how well the project is working, and *not* the impact it's having on families (this may be the subject of a later evaluation).

Explain that this interview is based on a theory of change for the EACs (if unfamiliar: this is essentially a diagram summarising what EACs are doing and their expected outcomes). We have also designed a monitoring framework which outlines how to measure progress in setting up the project processes and how to measure outcomes (although we won't be discussing outcomes today).

As you know, we are also speaking to the project leads, as well as children and families who have attended the projects. Once we have spoken to everyone, we will write a process evaluation report and submit it to the Scottish Government.

• Explain anonymity and confidentiality: we will use the interviews to inform the findings of the process evaluation. They will be used alongside the findings of the interviews with project leads, interviews with families and the analysis of monitoring data to write a report that will be published by the Scottish Government. In the main, quotes included in the report will be anonymised. However, where areas have unique experiences, it may be helpful to identify areas and attribute quotes accordingly. Therefore, it is possible that some parts of the report may identify something you've said. There is also a chance that some parts of the report may be identifying (even if it you are not quoted directly) due to the small number of people in relevant roles.
• After the interview, we will check if there is anything that you would not like to be quoted in the report. We will also share with you any elements of the draft report we feel may be identifying, before sharing it with the Scottish Government. We can remove identifying information/quotes upon request. However, it will not be possible to change the overall findings of the report.
• Remind participant that the interview will last around an hour.
• Remind participant that there are no right or wrong answers.
• Emphasise that taking part is completely voluntary – they don't have to answer any questions they don't want to answer and can decide to stop the interview at any point.
• Remind that they are free to change their mind and decide not to take part at any time before or during the interview, or after the interview until the findings have been written up.
• Check if participant has any questions (offer to go through information sheet if necessary).
• Request permission to record interview. Explain that this is for transcription and analysis purposes and that recordings will not be shared outside the research team at Ipsos. If participant does not consent, researcher to take detailed notes.

That's recording us now. Could I quickly ask you to confirm for the recording that you are happy to take part based on the information in the information sheet that we just discussed?

Section 2 – Background (2 min)

Aim: to get an overall sense of how implementation has gone

To begin with, could you give me a quick overview of your role, your organisation, and how you are involved in [area]'s EAC?

• Probe on involvement with design and set up (e.g. scoping/codesign etc) or working with families (referrals, delivery of provision etc)
• How long have they been involved with the EAC?
• If partner provider: fully explore services provided
• How often do they interact with EAC project staff/project lead?

Thank you. For the rest of the interview, I'm going to be asking about activities that most EAC areas are doing (based on the standard, local-leveltheory of change that I mentioned earlier). Not all of them will necessarily be relevant to you, in which case we will just skip past.

Section 3 – Design and set up of provision (10 mins) (20 mins total)

Aim: To gather information on the activities in the theory of change which are in the 'System level delivery' section and relate to the initial design and set up of provision

To start with, I have questions around your experiences with the design of the EAC project in [area].

AC6: Scoping/mapping exercises

First of all, were you involved in any scoping or mapping exercises that were conducted in the design stages of the project?

• Could you explain how this worked and what your role is/was?
  • At what times (ongoing?)
  • Who did you work with?
• What worked well?
• What were the challenges?(Did anything help?)

AC7: Co-design

Were you involved in any co-design activities?(If needed: Activities which facilitated parents/carers and children feeding into the design of the provision)

• Could you explain how this worked and what your role is/was?
  • At what times (ongoing?)
  • Who did you work with?
• What worked well?
• What were the challenges?(Did anything help?)
  • [If not already covered] Was it easy to engage families in the process?
  • Are there certain families that you find harder to engage with? Was there anything in particular you did to try and engage these groups? (Probe around whether they reached typically harder to reach young mums and minority ethnic families)
  • Similarly, are certain types of family more likely to be assertive about what they want?
  • Any issues with suggestions that turned out to be unrealistic/difficult to implement?
  • Disagreements among parents?
• If you were to do this again, what would you do differently?
• How did the process inform the design of the provision?
• And how well would you say the suggestions from co-design have been working in in practice?
• And are there any key insights that have since emerged, that you're surprised didn't come out in the co-design process?

AC8: Align with other local initiatives

In your view, how well has the project been designed to align with other local initiatives supporting families?

(Probe on what (if any) changes were made to the EAC service to do this?

Are any further changes planned to improve alignment with local initiatives further?

• What has worked well to support this?
• Have there been any challenges around this? (Did anything help?)
• Could anything be improved?
• If not already discussed: Did this process have an impact on relationships with partners?

AC9: Procurement/distribute funding to childcare services

Did you receive funding directly from the EAC in [area]?

If yes:

• Could you explain how this worked?
  • How was this decided?
  • Was there an application process?
  • How long did the process take?
  • Did they receive the amount of funding they wanted/needed?
• What worked well about the funding process for you?
• What were the challenges?
• Could anything have been improved?
• Would you like to change anything about it?

Section 4 – Direct delivery for families (15 mins) (35 mins total)

Aim: To gather information on the activities in the theory of change which are in the 'Direct delivery for families' section

I'm now going to talk through activities related to working with and delivering for families. Again, we will skip any that aren't relevant for you.

AC2: Establish referral pathways

Firstly, are you involved in the process of referring families to EAC services at all?

If yes:

• Could you explain how this works and what your role is/was?
  • At what times (ongoing?)
  • Who did they work with?
• What worked well?
  • Are the are the referral pathways supporting a smooth and easy experience for families?
• What were the challenges?(Did anything help?)
• Do you plan to do anything differently in the future?

AC1a: Promotion & engagement with families

Other than direct referrals, are you involved in any other ways of promoting the project to families?(E.g. sharing information via social media/speaking to families/distributing leaflets or posters)

If yes:

• Could you explain how this works and what your role is/was?
  • at what times (ongoing?)
  • Who did they work with?
  • Has the approach been tailored to different types of families?
  • If yes: How well has that worked?
• What worked well?
• What were the challenges?(Did anything help?)
  • Are there any families that have been more challenging to reach?
  • Do you think there are issues of embarrassment or stigma for children and young people or families attending targeted services like these?
• Do you plan to do anything differently in the future?

And how well attended would you say the provision has been, both in terms of the initial engagement of families and sustaining attendance?

• How does this compare with any targets / expectations you had?
• Is the project better attended by some target groups than others?
• How does it vary by different services?
• What do you think helps to facilitate sustained attendance?
• What have challenges been in terms of initial and sustained attendance?
• Did anything help with these challenges/Could anything have helped?

AC3: Application/registration processes

Moving on now to the application or registration processes – are you involved in this at all?

If yes:

• Could you explain how this works and what your role is/was?
  • at what times (ongoing?)
  • Who did they work with?
  • How do families apply/register?
• What worked well?
• What were the challenges?(Did anything help?)
  • Does the application process present any barriers to families from registering for EAC provision?
  • Are there any tensions between what is easiest for families and what services require?
• Do you plan to do anything differently in the future?

AC4: Delivery of a range of activities (inc. active play & skills-based activities) & food as part of a childcare offer that meets the needs of target families - free for those on low incomes

And thinking now about the actual delivery of the childcare element to families. To what extent are you involved in this?

If involved:

For each of the following aspects, what has worked well in terms of delivery and what have the challenges been?

• Before school provision
• After school provision
• ASN specific provision
• Holiday provision
• Food provision
• Transport

Do you plan to do anything differently in the future?

Are you also involved in providing wider family support?

If yes:

• What types of support? And what types are accessed most by families?
• Who is responsible for this?
• Has it changed at all recently?
• How important do you feel it is to offer this alongside the childcare provision?
• What's currently working well?
• What are the challenges of delivering family support as part of the provision?

Do you plan to do anything differently in the future?

Section 5 – Partnership working, governance and knowledge sharing (15 mins) (50 mins total)

Aim: To gather information on the activities in the theory of change which are in the 'System level delivery' section and relate to the initial design and set up of provision

AC11: Close partnership working

Now moving on to think about partnership working with the EAC.

In what ways and how often are you in contact with staff at the EAC?

• What information do you share/do EAC staff share with you?
• And thinking back to when you first heard about the EAC project – how did you find out about it?
  • What information did they share? (Was there anything that was unclear? E.g. purpose of the funding/practicalities)
• Overall, what is working well in terms of partnership working?
• What challenges have there been?
• Do you plan to do anything differently in the future?Has your involvement with the EAC had broader implications for your partnership working?

AC12: Stakeholder engagement - make the case' for the programme

And thinking about other partners you've been working with, have you done anything to advocate for the EAC project, or in other words 'make the case' for the project?

• If yes: Which stakeholders? What has the reaction been?
• What has worked well?
• What have the challenges been?
• How important is it that you do this as part of your role?
• Do you plan to do anything differently in the future?
• If no: is this something you would consider doing in future?

AC10: Project governance (internal meetings & multi-agency meetings, regulatory process etc)

Changing the subject slightly and thinking more generally- from your perspective, how well has the governance of the project been going? (for example, things like the way in which the project is managed and staffed)?

• What are the strengths of the way the EAC services are currently governed?
• What, if anything, could be improved?

AC17: Collect operational data (inc. user needs/ data on families not engaged with services)

To what extent are you involved with collecting operational data for EAC services? (e.g. collecting information on families using or not using EAC services, data on attendance, asking for feedback from families etc.)

If relevant, probe on:

• What information exactly they collect and at what times
• Why do they collect this?
• What are they not collecting and why?
• What is the process for sharing data with EAC staff?
• What has been working well?
• What have the challenges been?(Did anything help?)
• Probe on issues with data sharing / burden for families or staff
• Do you plan to do anything differently in the future?

Section 7 – Training and workforce development (5 mins) (55 mins total)

AC15: Ongoing training for providers/other local actors/ AC16: Work with partners to help develop childcare workforce

This is the final section before we wrap up now.

Have staff attended any training as part of working with the EAC?

If yes: What did this involve? (When/how often?)

• How were training needs identified?
• Who arranged and delivered this?
• What has the impact of the training been?
• What worked well?
• What have the challenges been?
  • Are there any unmet training needs among staff? Skills gaps?
• Do you plan to do anything differently in the future?

Are you aware of any other activities as part of EAC services that help to develop or broaden the local childcare workforce?Probe for details – refer to specific local activities if relevant

If yes: How well has this been working? Could anything be improved?

Potential scalability of EAC processes:

When considering increasing the scale of your work with the EAC, what would be the key barriers and enablers to this?

What key inputs would be required in order to increase the scale of your work with the EAC?

Section 8 - Summary & thoughts on improvement (4 mins) (59 mins total)

Aim: summarise thoughts and explore suggestions for improvement

We're coming to the end of the interview now. I'd like to spend the last few minutes summarising your thoughts and discussing ways to improve the programmes.

So to summarise:

• What aspects of the programme do you feel are working well?
• And what aspects do you feel are not working so well?
• What are the priorities in terms of making any improvements?
• If you were speaking to a new EAC, what are the main pieces of advice you'd give them?
• What opportunities has the creation of the EACs presented for you?

Section 8: Wrap up (1 min) (1 hour total)

That's everything I wanted to ask you today, thank you very much for taking the time to talk to me today. Is there anything we haven't covered that you would like to say something about?

Thanks for speaking with me today, I really appreciate it. I'll stop the recording now [stop recording].

Now the interview has finished are there any comments you have made that you would like to not be included as direct quotes in the report? You can also get back in touch with us later if you think of anything you'd like not to be included. We'll offer you the chance to review any sections of the draft report we feel may be identifying before submitting it to Scottish Government.

Nb: make note of comments not wanting to be included below or if appropriate mark in guide above.

A reminder of the next steps for this evaluation work:

• Once interviews completed, Ipsos to write up report over the next couple of months.
• This will be published later this year. If you would like us to send you a link to the published report, we would be happy to do that. Is that something you would like us to do?. In this case, we would keep your contact details potentially a bit longer than usual (3 months after the project ends) to make sure we can do this. Then they will be securely deleted.

If yes, make a note in rec spreadsheet.

If time:

The Scottish Government are exploring the best ways will be to communicate findings from this evaluation work with EAC leads and partners such as yourself. Do you have any views on what would be a good way to do this?(What would be useful for you to know?)

Thank and close.