Regeneration Capital Grant Fund: evaluation

Appendix 2: Method and research tools

Research support and limitations, participant information sheet, consent form and discussion guide

Research support

This research was supported by a Research Advisory Group (RAG), comprising:

• Scottish Government officers from the Regeneration Unit
• Scottish Government officers from Communities Analysis Division
• a COSLA (Convention of Scottish Local Authorities) representative
• a SOLACE (Society of Local Authority Chief Executives) representative

Study limitations

The research used qualitative methods to gather an in-depth understanding of the experiences and outcomes of 14 focus projects which had received an offer of grant from the RCGF. This approach allowed for an exploration of the range of experiences of involvement in RCGF projects, from the perspectives of project leads, partners, funders, community members, service users and others. However, it does not mean that the experiences within these 14 projects are transferrable to the experiences of other projects. RCGF projects are all different and develop in different ways, in different contexts.

The research method involved contacting the project lead initially, and from there building a network of partners, community members and others to contact. The project lead sought initial permission for the researchers to contact potential participants, to avoid cold contact – in line with good research ethics and data protection legislation. However, this approach had the potential to allow project leads to act as 'gatekeepers', signposting researchers to those who had a particular experience of the project. The ability of one individual to act as a gatekeeper was reduced, however, through researchers also asking other participants whether there were key individuals or groups that they felt should be involved in the research.

Finally, while discussions were led by experienced senior researchers, each interview was a one-off contact with an individual. While participants were encouraged to be open and honest in their reflections, and reassured about their anonymity, participants may not have felt comfortable talking about all aspects of their experience with someone they had only recently met.

Information sheet

Evaluation of the Regeneration Capital Grant Fund

Scottish Government, 2019

Invitation to take part

We would like to invite you to take part in our evaluation of the Regeneration Capital Grant Fund. Taking part is entirely up to you. Before you decide whether to take part or not, we would like to explain why the study is being done and what it would involve. If you have any questions, please contact Christina or Katy at Research Scotland. Our contact details are at the end of the document.

What is the study about?

We are evaluating the Regeneration Capital Grant Fund, exploring the outcomes it has brought about and the factors which have contributed to this. This is to help the Scottish Government and COSLA to better understand the social outcomes the Fund has achieved, and what has helped and hindered with this.

The main focus of the research is on the social outcomes and community involvement aspect of the Regeneration Capital Grant Fund. The physical and economic outcomes will be explored separately by the Scottish Government.

The evaluation will take place between February and June 2019. It will involve fieldwork in 14 projects funded through the Regeneration Capital Grant Fund across Scotland. Views will be anonymised within the report, but the participating 14 projects may be listed as an Annex to the report.

It is important to note that these 14 projects have been selected to give an overview of the outcomes of the Fund, and enable in-depth exploration with project leads, partners and communities. The focus is on evaluating the Fund, not evaluating the performance of the 14 projects.

Who is organising and funding this study?

This study is being carried out by Research Scotland on behalf of the Scottish Government and COSLA. The Research Advisory Group for the evaluation includes the Scottish Government, COSLA and SOLACE.

Why have I been invited to take part?

You've been invited to take part as you are a project lead for one of the 14 projects we have selected as a focus for the evaluation. These projects were selected through discussions with the Scottish Government, COSLA and SOLACE. We worked to try to involve a good mix of projects which were ongoing, completed or withdrawn; from different parts of the country; and which had different aspects of the project funded by the RCGF.

What does taking part involve?

Taking part as a focus project involves the project lead, project partners, community members and service users taking part in discussions with researchers, about their experiences of the project.

Your role as a project lead would involve:

• a 90 minute individual interview - this could be face to face or over the phone. The discussion would explore your views on community involvement within the project, the social outcomes you have achieved, and the key success factors and challenges you have experienced
• support engaging with partners, community members and service users - we would work with you to discuss and agree the best way to involve key stakeholders, in a way which suits them

Do I have to take part?

No, it is entirely up to you. Participation is voluntary. You do not have to participate if you do not want to. If you decide to take part, you will be asked to clearly give consent to confirm that you are happy and willing to take part.

Even if you tell us you want to take part or sign the consent form, you're still free to withdraw from the study at any time, without giving a reason.

Will my taking part in this study be kept confidential?

Yes, all information collected from and or about you will be kept confidential. We will store notes of our discussion in our IT system in an anonymised way, without your name or any other details about you.

You will not be identifiable in any study outputs, such as reports or presentations. We may use some direct quotes from what you say in study reports and presentations but where we do this, we will make sure we do not include information that may identify you.

Although all information and views will be completely anonymised within the report, the 14 projects which take part may be identified as an Annex within the report. The staff involved on the Research Advisory Group (Scottish Government, COSLA and SOLACE) also know which 14 projects were involved.

What will happen to the results of this study?

Your anonymous data will be combined with that of other participants and this will be used to produce a research reports and presentations to be shared with the Scottish Government. At the end of the project, the research report will be available on the Scottish Government website. The report is expected to be available in autumn 2019.

Details on data protection

Information collected from you as part of the study will be processed by Research Scotland. The information collected will only be used for the purposes of this specific study. Your data will be processed only so long as is required for this study. One month after the research is completed, Research Scotland will delete all the personal information from its systems.

In order to collect and use your personal information as part of this research, we must have a basis in law to do so. The basis that we are using is that the research is 'a task in the public interest'.

During the study, your data will be stored in secure, locked cabinets or secure password protected servers for electronic data with access limited to the research team at Research Scotland. To safeguard your rights, we will try to minimise the processing of personal data wherever possible. If we are able to anonymise or pseudonymise the personal data you provide, we will do this at the earliest opportunity. This means that personal details such as your name and contact details will be removed from the data, and a number will instead be assigned to it. That number will then be used whenever referring to it.

Withdrawing from the research

If you decide you don't want to take part before or after the interview or discussion group, please contact the research team (details below). We'd appreciate it if you could give us as much notice as is possible. If you decide you don't want to take part during the interview or discussion group, simply let the researcher know you'd like to stop.

Please note that we will not be able to exclude the information you have provided after it has been combined with that of other people taking part and we will need to keep the information you've already provided. Your rights to access, change or move your information will be limited as we need to manage your information in specific ways for the research to be reliable and accurate. To safeguard your rights, be assured we will use the minimum amount of personally-identifiable information possible.

Contact details

If you have any concerns or questions at all about taking party in the study, please contact Christina Bruce or Katy MacMillan at Research Scotland on 0141 428 3972 christina.bruce@researchscotland.org / katy.macmillan@researchscotland.org

If you are still concerned or are unhappy about any aspect of the study, please contact the Scottish Government study lead, Nadine Andrews, Senior Social Researcher on 0131 244 2949 or at Nadine.Andrews@gov.scot

Thank you for taking the time to read this and considering taking part. Participant Consent Form

Evaluation of the Regeneration Capital Grant Fund

Research for the Scottish Government, by Research Scotland

Participant Identification Number for this study:

Name of researcher:

This consent form is to ensure that you understand the nature of this research and have given your consent to participate. Your participation is entirely voluntary and you are free to change your mind about taking part at any time. Just tell a member of the research team at Research Scotland if you wish to do this.

Please read each of the statements below. If you have any questions please ask a member of the research team at Research Scotland. Please only sign the form when you are happy with ALL statements.

By signing this form, you agree to take part in an interview or discussion group about your experiences of involvement in (x project – to be adapted dependent on nature of discussion).

1.	I confirm that I have read and understand the participant information sheet. I have had the opportunity to consider the information, ask questions and have had these answered to my satisfaction.
2.	I understand that my participation is voluntary and that I am free to withdraw from the study at any time, either during the discussion group or afterwards (up to the point my information has been combined with that from other people) without giving a reason and without there being any negative consequences.
3.	I understand that direct quotations from my discussion may be used in an anonymous way in the research report.
4.	I give permission for members of the research team to have access to my anonymised responses.
6.	I understand that I do not need to answer any questions that I do not wish to.
7.	I understand that my views will be completely anonymised and reported in a collated way, but that the 14 projects involved in the evaluation may be identified.
8.	I agree to take part in the above study.

Name of participant

Date Signature

Name of person taking consent

Date Signature

Discussion Guide: Project Leads 90 minutes

Regeneration Capital Grant Fund Evaluation

Set up and consent

Introduction

We have been appointed to evaluate the social outcomes of the Regeneration Capital Grant Fund. Economic and physical outcomes will be assessed separately - internally by the Scottish Government.

The evaluation is of the RCGF programme as a whole and is not an evaluation of specific projects. To inform the evaluation, we would like to focus on the experience of 14 anonymised projects. In each project, we would like to speak with:

• a project lead
• project partners
• community organisations involved in planning, delivering or managing the project
• service users or individuals supported by the project

This will allow in-depth exploration of social outcomes, community involvement and success factors. We can be flexible in our fieldwork, depending on the nature of your project and who you think it is most appropriate to involve.

About the interview with you

We would like to interview you as the project lead in order to:

• explore your views and experiences
• gain your views on the other key stakeholders to involve from your project

Taking part is voluntary and up to you. You can choose not to answer any questions and you don't need to give a reason. You can choose to withdraw your participation up to 28 June 2019. Just get in touch with Research Scotland at katy.macmillan@researchscotland.org After 28 June 2019 your views will have been amalgamated with the views of others, and so can't be removed.

Views will be reported anonymously. Our report won't state the names of the people who took part. It will not be possible for readers of the report to identify which individuals took part in the research. Any quotes or comments that potentially identify participating projects or individuals will not be used. However, the 14 projects involved in the research project may be identified as an Annex to the report.

This interview explores your views. Please tell us what you think in an open and honest way. Your views will only be seen by the Research Scotland team involved in this work and won't be passed on to anyone else.

Consent

Do you have any questions about the evaluation or this interview?

Do you understand that:

• your participation is voluntary and you are free to withdraw from the study at any time
• direct quotations from your discussion may be used in the research report, but your identity will not be revealed
• you do not need to answer any questions that you don't wish to

Do you agree to take part in this interview?

Background

Your role

1. What was your role in the RCGF funded aspect of the project? Did this change along the way? Are you still involved?

Aims

2. How would you describe the main aims of your project?

3. Did the project aims change along the way? Why? In what way?

Project delivery

4. What stage is the project at just now? What is your current focus?

5. What has helped you get to this stage? What factors contributed to delivering your project?

6. What key challenges have you faced? How have you worked to overcome these? What learning have you gained?

Community involvement

7. How would you describe the community or communities that this project aimed to involve?

Did this change along the way?

8. How did you assess need for the project? How were communities involved?

9. How were community members involved in:

• planning and designing the facility?
• designing and delivering services in the new facility?
• using the facility?

10. What worked well? What didn't? How could this be improved?

11. Did your approach to community involvement change along the way? Why? In what way?

12. Did you conduct a charette (or similar design event) as part of the community involvement work? When did this happen? How useful was it?

13. How do you think community involvement influenced how your project developed?

• what positive impact (if any) did it have?
• what negative impact (if any) did it have?

14. In what way, if any, have you:

• focused delivery on people's needs?
• empowered communities to improve their area and assets?
• ensured everyone had a voice and could participate?
• worked in partnership with the community?

Social outcomes

15. How would you describe the social outcomes that your project was aiming to bring about? Did these change along the way?

16. How would you describe the balance of priority in your project across social, physical and economic outcomes? Which of these was the main driver for the project?

17. To what extent do you believe you:

• enabled people to feel safe and protected?
• supported people to have good physical and mental health?
• supported strong and effective community networks?
• provided access to appropriate community facilities and services – including health, education, early years support and places to meet?
• helped communities build a positive identity and future aspirations?
• developed places as a focal point for social and economic interactions?
• supported the development of fair and inclusive communities?
• supported sustainable employment to tackle worklessness?

(Tailor based on social outcomes mentioned within application and monitoring forms)

18. How do you know this? What evidence do you have?

19. Do you believe you have contributed to any other social outcomes, beyond those already explored? Probe.

20. What helped you to achieve social outcomes? What hindered?

21. Did your project bring about any unintended social outcomes – positive or negative? In what way?

Partnership working

22. Who were the key partners in delivering your project? How much did you work with them?

23. What encouraged you to work in partnership?

24. Did working in partnership help you to deliver your project effectively? In what ways?

25. In your view, did all partners have the capacity to support the project effectively?

26. What challenges did working in partnership bring? How were these addressed?

Monitoring and evaluation

27. How did you monitor and evaluate your project? Did you use any good practice methods or approaches?

28. What skills and resources do you think were needed to effectively monitor and evaluate your project? Do you feel your project had access to these skills and resources?

29. Were communities involved in evaluation? How?

30. How was the information you gathered used to support future project development?

31. What worked well about your monitoring and evaluation approaches? Why? What doesn't work so well? Why?

Impact of RCGF

32. Have you any comments on the process of applying for RCGF?

33. Overall, what difference do you think the RCGF has made to your project?

34. Do you think the project would have gone ahead without RCGF? Would it have developed differently?

35. Did receiving RCGF make any difference to:

• how you involved communities?
• how you worked in partnership?
• how you monitored and evaluated your project?
• the outcomes that your project brought about?
• the match funding that you were able to generate?

36. What advice would you give to organisations planning RCGF funded activity?

Discussion Guide: Partners 60 minutes

Regeneration Capital Grant Fund Evaluation

Set up and consent

Introduction

We have been appointed to evaluate the social outcomes of the Regeneration Capital Grant Fund. Economic and physical outcomes will be assessed separately - internally by the Scottish Government.

The evaluation is of the RCGF programme as a whole and is not an evaluation of specific projects. To inform the evaluation, we would like to focus on the experience of 14 anonymised projects. In each project, we are speaking with:

• a project lead
• project partners
• community organisations involved in planning, delivering or managing the project
• service users or individuals supported by the project.

About the interview with you

We would like to interview you as a project partner in order to explore your views on social outcomes, community involvement, success factors and lessons learned.

Taking part is voluntary and up to you. You can choose not to answer any questions and you don't need to give a reason. You can choose to withdraw your participation up to 28 June 2019. Just get in touch with Research Scotland at katy.macmillan@researchscotland.org After 28 June 2019 your views will have been amalgamated with the views of others, and so can't be removed.

Views will be reported anonymously. Our report won't state the names of the people who took part. It will not be possible for readers of the report to identify which individuals took part in the research. Any quotes or comments that potentially identify participating projects or individuals will not be used. However, the 14 projects involved in the research project may be identified as an Annex to the report.

This interview explores your views. Please tell us what you think in an open and honest way. Your views will only be seen by the Research Scotland team involved in this work and won't be passed on to anyone else.

Consent

Do you have any questions about the evaluation or this interview?

Do you understand that:

• your participation is voluntary and you are free to withdraw from the study at any time
• direct quotations from your discussion may be used in the research report, but your identity will not be revealed
• you do not need to answer any questions that you don't wish to

Do you agree to take part in this interview?

Background

Your role

1. What was your role in the RCGF funded aspect of the project? Did this change along the way? Are you still involved?

2. What stage is the project at just now? What is your current focus?

3. What has helped you get to this stage? What factors contributed to delivering your project?

4. What key challenges have you faced? How have you worked to overcome these? What learning have you gained?

Community involvement

5. How would you describe the community or communities that this project aimed to involve?

Did this change along the way?

6. How did you assess need for the project? How were communities involved?

7. How were community members involved in:

• planning and designing the facility?
• designing and delivering services in the new facility?
• using the facility?

8. What worked well? What didn't? How could this be improved?

9. Did your approach to community involvement change along the way? Why? In what way?

10. How do you think community involvement influenced how your project developed?

• what positive impact (if any) did it have?
• what negative impact (if any) did it have?

11. In what way, if any, have you:

• focused delivery on people's needs?
• empowered communities to improve their area and assets?
• ensured everyone had a voice and could participate?
• worked in partnership with the community?

Social outcomes

12. How would you describe the social outcomes that your project was aiming to bring about? Did these change along the way?

13. How would you describe the balance of priority in your project across social, physical and economic outcomes? Which of these was the main driver for the project?

14. To what extent do you believe you:

• enabled people to feel safe and protected?
• supported people to have good physical and mental health?
• supported strong and effective community networks?
• provided access to appropriate community facilities and services – including health, education, early years support and places to meet?
• helped communities build a positive identity and future aspirations?
• developed places as a focal point for social and economic interactions?
• supported the development of fair and inclusive communities ?
• supported sustainable employment to tackle worklessness?

(Tailor based on social outcomes mentioned within application and monitoring forms)

15. How do you know this? What evidence do you have?

16. What helped you to achieve social outcomes? What hindered?

17. Did your project bring about any unintended social outcomes – positive or negative? In what way?

Partnership working

18. What encouraged you to work in partnership?

19. Did working in partnership help you to deliver your project effectively? In what ways?

20. What challenges did working in partnership bring? How were these addressed?

Impact of RCGF

21. Overall, what difference do you think the RCGF has made to your project?

22. Do you think the project would have gone ahead without RCGF? Would it have developed differently?

23. Did receiving RCGF make any difference to:

• how you involved communities?
• how you worked in partnership?
• how you monitored and evaluated your project?
• the outcomes that your project brought about?
• the match funding that you were able to generate?

24. What advice would you give to organisations planning RCGF funded activity?

Discussion Guide: Communities 60 minutes

Regeneration Capital Grant Fund Evaluation

• consent process to be gone through separately with consent form
• choose relevant sections/ adapt/ add new sections dependent on focus of project

Background

1. How have you been involved in (the project)?

2. Have you been involved in:

• planning and designing the facility?
• designing and delivering services in the new facility?
• using the facility?

3. Why did you get involved?

Planning and designing the facility

4. Were you involved in thinking about whether this project was needed and what it might look like? How?

5. What worked well? What didn't? How could this be improved?

6. How do you think community involvement influenced how the project developed?

7. Did you feel that:

• the community was empowered to get involved in a meaningful way?
• the community was listened to?
• community views were respected?

8. How did being involved influence you personally?

• health and wellbeing?
• sense of community or sense of belonging to the area?
• how you felt about the local area?
• learning new things/ skills?
• your aspirations and aims for the future?

9. How did being involved impact on your community?

• connecting the community or strengthening relationships?
• building a positive identity for communities or for the local area?
• influencing aims or aspirations for the future as a community?
• supporting fair and inclusive communities?

10. Were there any negative aspects of being involved? What were these? How could they be avoided?

Using the facility

11. Do you use the facility/ service? How? When?

12. What difference does the facility / service make to your life?

13. What works particularly well?

14. What could be improved?

15. Has the new facility impacted on your ability to:

• access the services you need – health, education, early years, skills development, etc?
• have good places to meet people and socialise?
• make connections in the community?
• use local businesses and services?
• work, volunteer or learn locally?

In what ways? Why do you think this?

16. Has the new facility impacted on your feelings of:

• safety in your community?
• physical and mental health?
• your community being fair and inclusive?

In what ways? Why do you think this?

(Tailor based on social outcomes mentioned within application and monitoring forms – and run as an interactive exercise using e.g. dot voting, statements on cards dependent on needs of the group)