Developing young workforce strategy - impact on employer engagement: evaluation

Appendix 1: Detailed methodology

To answer the research questions and aims of this evaluation as set out in the Invitation to Tender, Rocket Science developed and implemented a six stage methodology which is summarised below:

Stage 1: Inception and planning

• Understanding the context for the work, expectations and requirements.
• Agreeing on the methodology and project management terms, including regular meetings with the contract manager and the Research Advisory Group.

Stage 2: Desk Research and sampling framework

• Desk review of research to date to identify gaps and priority areas for consultation with employers and practitioners.
• Agree a sampling framework for both employer and practitioner participants.

Stage 3: Evaluation framework and research design

• Identify employers, practitioners and stakeholders to speak to in line with the sampling framework agreed in Stage 2.
• Design research materials which will include:
  • Introductory emails and content to send to employers.
  • An agreed contact plan for employers and practitioners.
• Design topic guides for all engagements.

Stage 4: Fieldwork

• Telephone and video calls with employers, practitioners and stakeholders.

Stage 5: Analysis and reporting

• Analysis of employers and practitioners using a charting approach to develop a full audit trail of the points and issues raised under each topic area so that common threads and variance can be identified clearly.
• Outputs:
  • A Draft Report, submitted to the contract manager for review and comment by the Scottish Government and RAG.
  • A Final Report which takes account of our discussions on the Draft Report with the client and the RAG.

Stage 6: Stage 6: Dissemination:

• A PowerPoint presentation for use with a range of stakeholder audiences. This will be designed both for our own presentation and for use by others to further audiences.

Table Notes:

1. Those who have been involved in virtual activities

2. Chamber of Commerce members which will include some who have not engaged

Sampling framework

A sampling framework was used to ensure that the research was able to articulate common themes of activity and impact, and importantly variations across DYW Regional Groups, sectors, and population differences. While this research did not require a systematic sample of employers, the framework below considers the scale, sector, and geography of employers.

Overall, the selection of employers and practitioners includes:

• All the main urban centres – Glasgow, Edinburgh, Dundee, Aberdeen.
• Rural areas in the north, east and south of Scotland.
• An island grouping.
• Former manufacturing/coal/steel areas now with range of new local businesses and inward investors.

Table A.2: Proposed selection of 8 DYW Regional Groups and 4 nested LAs, with rationale

DYW Regional Groups / Local Authorities

Inverness and Central Highlands:

Large rural area with a fast growing city

• Highlands:
  Inverness plus range of dispersed rural communities

Outer Hebrides

Remote island grouping

North East

Aberdeen – in transition as significance of oil and gas declines - plus large rural hinterland with range of small rural and commuting towns

Glasgow:

Largest city with wide range of employment and significant areas of deprivation wide range of schools and employers

Ayrshire:

Highly varied area including Glasgow commuting, old mining communities, heavy engineering and textiles, and coastal tourist towns as well as market towns and extensive agricultural areas. 
Shift to services sector (e.g. call centres and light manufacturing. Some high levels of unemployment in areas such as Irvine and Kilmarnock

• North Ayrshire:
  Long standing focus on regeneration, skills and employability with wide employer base

Edinburgh, Midlothian and East Lothian:

Fast growing region based on the capital city with range of FINCAP, digital businesses and professions, strong university research base and associated employers, and large public sector employment

• Edinburgh:
  Rapidly growing city with wide range of growing high skill employers

Dumfries and Galloway:

Rural area with dispersed communities

Lanarkshire and East Dunbartonshire:

Area formerly dominated by steel and coal and now a range of new industries across manufacturing, service and logistics

Within these 8 DYW Regional Groups and 4 nested Local Authorities, Rocket Science along with the Scottish Government identified the main engagement approaches which would be used to identify and make contact with the required number of employers.

It should be noted that this sampling framework was used as a guide to ensure that we were able to gather views and opinions from a range of employers and practitioners, whilst remaining responsive to changing priorities and findings emerging from the interviews conducted. The final number of employers and practitioners interviewed is set out in the main report in Table 1.

In the table below we set out the employer and practitioner target framework, which was regularly updated, iteratively, in line with the changing priorities, and emerging findings from the desk research and the interviews with employers.

Where possible, analysis of employer views was conducted according to the size of businesses. These were identified using standard definitions based on number of employees, consistent with Scottish Government statistics Small businesses have from 0 to 49 employees; medium-sized businesses have 50 to 249 employees; and large business have 250 or more employees.

Limitations of this study

The methodology for this research was designed to be robust, but it has some limitations, as is typical for this type of research. Firstly, there was a lower response rate than was initially anticipated, with not all contacts able to make time for interviews for the project. The reasons for these are unclear, but are thought to include:

• The availability of employers was impacted by timing of calls over the summer holidays, when typically more people are on annual leave.
• Although employers were largely introduced to the research through known project contacts, this did not always translate into them having time and resource available to take part in an interview.
• In addition, there was a challenge due to the large number of contacts to be made, which may have been limited by capacity of contacts to make introductions to employers, and to promote this research.
• Finally, while interviews were conducted throughout the summer of 2022, it was found that later interviews were often providing less additional information, due to a critical mass of key information already being derived.

For these reasons, it was felt that the total number of interviews was providing a robust understanding of a range of employer and practitioner perspectives.