The impact of children and young people's participation on policy making

Appendix A: Methodology

It was not within the scope of this study to achieve a comprehensive understanding of the entire extent of children and young people's participation and engagement work within Scotland, or to be able to fully identify and quantify the impact this activity has had on policy making. Rather the intention was to describe and illustrate different ways in which participation work has developed in Scotland and to identify some key success factors and limitations to support effective child and young people orientated policy-making in the future.

A case study approach therefore was selected for this study, as it offers a good means to understand complex situations and explore relationships between actions and outcomes ^[11] .

There were several steps to this study: a survey to identify potential cases; a prioritisation exercise to select the cases; qualitative research with multiple stakeholders to gather data to inform the cases; and analysis and synthesis of qualitative data to develop 6 cases studies.

Initial survey

Organisations across Scotland were invited to submit examples of work where they had developed and delivered participation and engagement work with children and young people to influence policy development or implementation. In order to capture this information, we developed a brief online survey, which focused on the following information:

• Name and brief outline of the participation activity
• Demographic information about participants
• Geography
• Policy area and stage of policy making (development, implementation)
• Whether children and young people involved in the activity are contactable
• Contact information

The survey had a mixture of open and closed questions. Closed questions allowed us to identify particular demographics and timeframes for each project. Open questions allowed us to capture the stories of our participants in more detail.

The survey was launched on 6 March 2017 and closed on 16 March 2017. Information about the survey was distributed via the Children in Scotland membership list (6300 contacts), via social media, and directly to organisations we know have undertaken participation and engagement work with children and young people to inform policy making in Scotland.

The survey received 37 responses to be considered for case study from a range of voluntary and statutory organisations across Scotland, influencing diverse policy areas.

Case studies selection

Exclusion criteria

In order to select the most relevant, up to date cases, we identified the following exclusion criteria, which were agreed with the commissioners:

• Engagement work that had been reported on more than three years ago
• Participation and engagement work undertaken for reasons other than policy making / implementation
• Organisational policy development

Three responses were excluded as a result of this criteria, resulting in 34 responses that were discussed for case study selection.

Selection process

All 34 included survey responses were plotted according to the following criteria:

• Stage of policy making / implementation as outlined in the ROAMEF policy cycle ^[12]
• The policy area
• Geography - national and local activity, rural and urban engagement
• Demographics
• Dates of work - we asked for recent examples, reported on within the past 3 years
• Whether participants were contactable

Children in Scotland, together with the commissioners, then discussed the responses collectively and prioritised 6 responses (and 2 reserves) to focus on as cases. These were selected to give a good range of examples according to our criteria outlined above. It is important to state that many additional examples could have been selected in addition to these eight and the selection is no reflection on either the quality or impact of the responses received.

The responses selected to form case studies were as follows:

1. Equally Safe
2. Perth and Kinross SNAP Innovation
3. Police Powers to Stop and Search Children and Young People for Alcohol
4. Renfrewshire Champions' Board
5. UNCRC Reporting Cycle
6. Young Edinburgh Action

Case study data collection

The case studies allowed for collection of qualitative data; this was gathered from a minimum of two perspectives:

• The policy maker
• The organisation developing and delivering the participation work

The initial remit for the project had been to interview one organisation but certain case studies utilised interviews with more than one organisation to give a more rounded view of the engagement work. We also conducted a focus group for one project with some of the young people who were involved in the engagement work to gather their views on the process. Ideally we would have included interviews with children and young people for all the case studies. This was, however, not possible within the scope and timescale of the study.

Interviews with professionals

Six policy makers and six representatives from organisations were interviewed for the project. Because of the short timescale and limited budget for this work, interviews with all professionals were undertaken by telephone. It was anticipated that interviews would take roughly half an hour, however this was varied and depended on the project and respondent.

Semi-structured interview schedules were developed to guide the interviews, focusing on the following topic areas:

• About the policy/legislation area
• Views on the purpose of the engagement - why they did it
• Methods - what they did, who with, when and how
• How evidence was used
• Perspectives on what went well and what didn't go so well. Anything they would have done differently in retrospect and why?
• Perspectives on impact of participation - what did it change/affect?

The questions for interviews and focus groups was informed by the Council of Europe Child Participation Assessment Tool ^[13] . The tool has been designed as a method for evaluating national governments' performances in implementing children's rights. This allowed some assessment about whether certain key factors that promote meaningful participation are being met and provided the basis for recommendations.

Focus groups with young people

Recruitment of children and young people took place through the organisations responsible for their participation. All participants were given an (age appropriate) information sheet about the research, and were asked to complete and return a consent form (with parental consent requested for those aged under 16).

A focus group was utilised to gather information from the young people involved on the Young Edinburgh Action project.

All interviews and focus groups with professionals and young people were recorded and transcribed.

Data Analysis

Transcriptions were coded thematically and the data from the policy maker, organisation and children and young people's perspectives was synthesised and written into short, accessible case studies. It was agreed with the commissioners that the structure of the case studies would follow the interview topic guide and include a short discussion section.

Limitations of methodology

As discussed above, the six case studies form a small sample of all the participation and engagement work with children and young people in Scotland to inform policy making, and as a consequence they should not be viewed as representative of the whole. The short timescales involved may have restricted the number of responses received, for example. Furthermore the qualitative research methods did not allow for interviewing all those involved in the cases and perspectives and insights may have been missed. This is particularly the case for those cases where we were unable to include the perspectives of children and young people involved.

Nevertheless, we believe that that the evidence produced in these case studies provides valuable evidence of the types of participation and engagement work that has taken place in Scotland in recent years, and has allowed us to draw observations about the success factors and limitations of current approaches, thus adding to the existing evidence base in a meaningful way.