Scottish Asylum Right to Work Proposal

4. Pilot Evaluation

It is proposed that a pre/post evaluation of the pilot is conducted to assess whether the pilot delivers its intended objectives and benefits, as well as to explore any process-related issues. This evaluation design would establish a baseline before launch of the pilot, against which impact and changes can be assessed once the pilot has been operational for twelve months. It is proposed that Scottish Government analysts will support the development of monitoring and evaluation strategies, which will be tailored to each pilot delivery area and respective delivery partners.

The evaluation would capture how many people are granted the right to work and how many people are able to take up employment, and to what extent the pilot delivers its intended outcomes, including having a positive impact on people seeking asylum, their families, and the communities they live in. The data requirements, evaluation questions, and the scale of the evaluation would need to be further explored once the details of the pilot are agreed. However, a pre/post mixed methods design could involve some or all of the following elements:

Pre-implementation

• Agreeing outcomes to monitor and measure success through a theory of change;
• Baseline survey with people seeking asylum and employers before the pilot is launched to establish current experiences in relation to the pilot’s objectives and outcomes.

Post-implementation (post-twelve months)

• Follow-up survey with people seeking asylum and employers to compare any changes to experiences and outcomes;
• Interviews or focus groups with employers to discuss implementation of the pilot, identify any key issues, and explore delivered benefits;
• Interviews or focus groups with people seeking asylum, to explore their experiences of the pilot and its potential impact in-depth;
• A consultation-based survey with professional stakeholders and interested organisations;
• Analysis of secondary data sources, including to explore the pilot’s objective regarding reducing cost and demand for UK/Scottish public services.

Due to the inclusive approach to eligibility, it is anticipated that the evaluation will be representative of the broader asylum-seeking population in Scotland, to ensure the evaluation is able to explore the impact and needs of different groups. To establish a counterfactual, we may consider including a comparable sample of people seeking asylum who are not participating in the pilot. Depending on the roll-out and scale of the pilot, this could either be people seeking asylum that are not participating in the scheme, or we could consider people seeking asylum residing in the north of England as a comparator. However, including a counterfactual sample of people seeking asylum may not be feasible due to a number of practical and ethical issues which would need to be carefully considered.

As set out above, we would also seek to involve employers in the evaluation, and, depending on what wrap-around support is offered as part of the pilot, local authorities and other organisations involved in support delivery could be invited to participate in the evaluation.

The outputs of the evaluation would provide a robust evidence base, and lessons learned could support decision-making around whether and how the right to work could be rolled out more broadly following the pilot.