STEM strategy for education and training: second annual report

Annex B: Key Performance Indicators - Summary

Note

In December 2017, we published a set of Key Performance Indicators for the STEM Strategy. These KPIs were selected on the basis that they related most closely to the key changes that we want to see resulting from delivery of the actions set out in the Strategy and, where possible, to primarily reflect progress as a result of the actions that will be taken and not significantly influenced by other external factors such as demographic and general labour market changes. Where well established data sets existed, we set stretch aims for the targets. For others, some further data collection has been needed or they did not lend themselves to stretch aims. We reviewed the KPIs in 2019 in the light of additional data collections and analysis. This annex provides an update on the status of each of the KPIs. In some cases the data sets or the factors being measured have changed as a result of the review.

For a few of the KPIs, only baseline data is provided as further data is not yet available. As with the first Annual Report, it remains the case that the changes to the KPIs since the baseline may not be fully attributable to the Strategy actions as these were not underway during the full period that the data refers to, and/or are the product of decisions and activities that took place before the Strategy was in place. Time lags in availability of published data means that some of this information will refer back to a period when the Strategy actions were only just beginning to be implemented. Overall, it may be some time before the impact of the Strategy can be fully assessed. Details about underlying data for the indicators is published separately. In all cases we have used the most up to date published information to determine the baseline and the status of the KPI. Because they use different data collections, the actual baseline years vary because of timing differences between collection and publication of data and publication of this report. We will continue to monitor and evaluate individual actions and KPIs as new data becomes available to assess performance and determine if any changes in approach are required.

I. Increases in the proportion of people undertaking STEM related learning, engagement, study and training across all sectors including in school-level qualifications and awards, and participation in apprenticeship programmes. (Excellence and Inspiration)

I a. Meet Initial Teacher Education student intake targets for all STEM subjects.

Data Source: Higher Education Statistics Agency (HESA): Student data

In order to address teacher shortages in certain STEM subjects, targets are set for the number of students to be taken onto initial teacher education courses in these subjects.

While the targets themselves have still been missed in most subject areas, overall, intakes for the STEM secondary teacher training courses have increased in 2018-19 to 595 entrants compared to 510 in 2016-17 and 2017-18 and 380 in 2015-16 (the baseline year).

In addition to the figures above, there were 85 entrants in Home Economics (which was added to the STEM Bursaries in 2019-20). Further information on entrants to Home Economics are provided in the KPI spreadsheet.

Alternative routes into teaching are helping towards achieving targets for individual secondary subjects and will continue to do so in 2020-21, along with STEM bursaries. The 'Teaching Makes People' recruitment campaign will continue to promote teaching as a career in all subjects.

I b. Increase the number of passes at SCQF level 5 in Mathematics by 10% by 2022.

Data Source: Scottish Qualifications Authority (SQA): National Qualifications data

We have taken the 2017 SQA results as the baseline year from which to measure any increase as this was the year the Strategy was implemented.

In 2019, there were 29,873 passes at SCQF level 5 in mathematics (Mathematics and Application of Mathematics combined). This is higher than the baseline and also higher than in 2014. These changes should be seen in the context of a falling cohort size with the S4-S6 cohort falling each year from 135,548 in 2014 to 126,067 in 2019 (figures from December 2019 pupil census). It is worth noting that some learners may be included twice in these figures if they have entered for both the Mathematics and Application of Mathematics Courses.

	2014	2015	2016	2017	2018	2019
No. of passes	28,849	24,676	27,515	28,166	28,336	29,873
Change	---	-14.5%	+11.5%	+2.4%	+0.6%	+5.4%

52.9% of passes in Mathematics at SCQF level 5 were by females. The trend since 2014 in the proportion of passes by females in the post-review data has been:

2014	2015	2016	2017	2018	2019
51.5%	51.4%	52.4%	52.4%	52.8%	52.9%

I c. Increase overall provision of Foundation Apprenticeship opportunities to 5,000 by 2019 and expand provision and Foundation Apprenticeship opportunities across all Scottish secondary schools.

Data Source: Skills Development Scotland (SDS): Foundation Apprenticeship statistics

Cohort 1 (2016-2018 delivery) of the Foundation Apprenticeships (FAs) is the first year of delivery after the initial pilot phase and is the baseline year. The overall number of starts in cohort 1 was 346 of which 161 were STEM starts (46.5%). In cohort 2 (2017-2019) this increased to 1,244 starts of which 552 were STEM starts (44.4%), and in cohort 3 (2018-2020) the total number of FA starts was 1,532 of which 722 were STEM starts (47.1%). This increase in starts might be expected whilst the programme is establishing. SDS have contracted for 5,000 opportunities in 2019-20. Further SDS statistics on FAs are expected to be published later in March 2020.

From next year onwards we will revise this KPI to look at increasing the percentage of school leavers attaining STEM Foundation Apprenticeships. Information about attainment of STEM FAs is included in the detailed data published with this report.

I d. Increase the number of apprenticeship opportunities in STEM related subjects at SCQF Level 9 and above.

Data Source: Skills Development Scotland (SDS): Graduate Apprenticeship statistics

The Graduate Apprenticeship (GA) programme is still relatively new. The first cohort began in September 2017, the second cohort in September 2018 and the third cohort in September 2019. The overall number of starts in 2017-18 was 278, all of which were STEM starts (243 at SCQF Level 9 and above). In 2018-19 the total number of GA starts was 921 of which 607 were STEM starts (583 at SCQF Level 9 and above).

In 2018-19, females represented 19.4% of starts in STEM GA subjects, up from 17.6% in 2017-18.

Higher level Modern Apprenticeships (MAs) are also available at SCQF Levels 9 (Technical Apprenticeships) and 10 and 11 (Professional Apprenticeships). These are not included in the figures but may be included in KPI I d in subsequent annual reports.

I e. Increase the proportion of those who successfully completed a recognised qualification at college in a STEM subject.

Data Source: Scottish Funding Council (SFC): College FES data

For college courses at FE level, the proportion of successfully completed courses that were in STEM subjects was 27.4% in 2014-15, 27.7% in 2015-16, 26.8% in 2016-17, 27.0% in 2017-18 and 26.6% in 2018-19. For college courses at HE level, the proportion of successfully completed courses that were in STEM subjects was 30.3% in 2014-15, 30.2% in 2015-16, 30.0% in 2016-17, 29.2% in 2017-18 and 26.8% in 2018-19.

I f. Increase the proportion of Scottish Domiciled qualifiers on Full-time First Degree STEM courses.

Data Source: Higher Education Statistics Agency (HESA): Student data

2015-16 is the baseline year. The proportions of Scottish domiciled qualifiers on first time degree courses were similar across the four years: there has been decreases in the last two years but there is a slight increase overall since the baseline year. The proportions in 2015-16 were 34.6% (excluding medical courses) and 51.2% (including medical courses). In 2017-18 the proportions were 35.2 (excluding medical) and 52.5% (including medical). In 2018-19 the proportions were 34.7% (excluding medical) and 51.7% (including medical).

I g. Increase in the number of participants in STEM related Youth and Adult awards.

Data Source: Education Scotland: Youth Awards & Adult Awards data collection

The national Awards Network includes most of the main providers of Youth Awards in Scotland (28 member organisations). They have provided the following baseline estimates figures of the number and percentage of youth awards with a STEM learning embedded within their core curriculum:

• 39,900 youth awards in 2016-17 (43% of all youth awards achieved in Scotland in that year).
• 45,787 in 2017-18, representing 49% of awards achieved.
• 48,574 in 2018-19, representing 42% of all 114,939 awards achieved.

The rise in total STEM related awards achieved in 2018-19 is largely attributable to continued growth in achievement of the John Muir Award and Archaeology Scotland's Heritage Hero Award programme. The percentage of the total of awards achieved with an identified STEM element is marginally down, at 42% - but this is a reflection of a broadening of the range of awards included in the Awards Network census rather than any diminution in interest in STEM.

Examples of Youth Awards included are: The Duke of Edinburgh's Award, The John Muir Award, Young Enterprise Awards, Outward Bound Trust awards, awards from uniformed organisations and the Cadets and SQA wider achievement awards.

Baseline information for Adult Awards is not yet available but overall numbers of participants are much lower (400 in 2016-17).

II. Increased practitioner confidence in STEM learning in the early years, primary years and in CLD settings and increased practitioner engagement in STEM professional learning opportunities. (Excellence)

II a. Increase the cumulative hours of STEM professional learning accessed by early learning and childcare practitioners, schools, college and CLD practitioners annually.

Data Source: Education Scotland: Annual practitioner and STEM CLPL provider surveys

Education Scotland introduced new data gathering measures in 2017-18 to track provision of professional learning in STEM. In 2019, 49 STEM provider organisations responded to Education Scotland's data gathering exercise. Collectively, they provided 128,171 cumulative hours of STEM professional learning between 1 August 2018 and 31 July 2019. Some 30,505 practitioner engagements resulted from 2,132 professional learning sessions run by these providers over this period. This is in comparison to the total of 109,969 cumulative hours of STEM professional learning provided by the 44 providers that responded in 2018.

At the same time, practitioners from a range of sectors were invited to complete the Annual STEM Practitioner Survey 2019. There were a total of 1,187 responses to this survey from early learning and childcare, ASN, primary and secondary practitioners. The findings showed they accessed an average of 16.1 hours of STEM professional learning between 1 August 2018 and 31 July 2019, compared with 21.3 hours in 2018. The reasons for this decrease are not clear but Education Scotland's efforts to promote the survey to a much wider and more representative group of practitioners could be a contributing factor to this decrease. For example, there was an increase of 35% in the number of responses to the survey in 2019. Responses from early learning and childcare increased by 42%, with this sector accessing the lowest number of hours of STEM professional learning.

In 2018, an average of 63.4% of practitioners agreed or strongly agreed with the statement, 'I am confident in delivering STEM learning in my practice. In 2019, Education Scotland sought more granular responses to give an indication of the relative confidence levels across sectors and aspects of STEM. The results relating to practitioner confidence highlighted the variability in confidence across the sectors and disciplines. Details are provided in the separate KPI spreadsheet. This information will be used to inform targeting of activity as we move forward.

A separate STEM survey was also issued to community learning and development practitioners in 2019. In total there were 141 responses to this survey. Questions on professional learning hours and practitioner confidence were introduced for the first time in the 2019 survey. The findings showed that the respondents each accessed an average of 15.3 hours of STEM professional learning between 1 August 2018 and 31 July 2019.

III. Significant reductions in the equity gaps in participation and achievement in STEM learning, engagement, study, courses and training across all sectors in relation to gender, deprivation, rurality, race, disability and for care leavers. (Equity)

III a. Reduce the gap between the percentage of school leavers with 1 or more award in STEM subjects at SCQF level 6 or better from the least and most deprived SIMD quintiles to 31 percentage points by 2020 and to 25 percentage points by 2022.

Data Source: Scottish Government: Summary Statistics for attainment, leaver destinations and healthy living - school leaver attainment

2015-16 is the baseline year when the gap was 36.8 percentage points. Data from previous years show that the current value (a 34.4 percentage point gap) is smaller than the previous year when it was 36.6, and any year previous.

Between 2017-18 and 2018-19 there was a decrease in the proportion of school leavers with 1 or more STEM award at SCQF level 6 in the most deprived quintiles and least deprived quintiles. This led to the gap decreasing to 34.4 percentage points.

III b. Improve the gender balance in attainment in key STEM related subjects at SCQF level 6 by increasing the number of females passing Physics by 15% and Computing by 20%, by 2022

Data Source: Scottish Qualifications Authority (SQA): National Qualifications data

2017 SQA data is the baseline year. Prior to that, female passes in Physics had declined from 2,262 in 2014 to 1,899 in 2017 - with some variations across the years. Since 2017, they have remained broadly static, decreasing slightly in 2018, and increasing slightly in 2019. For computing, now including software development and cyber security, female passes declined from 670 in 2014 to 617 in 2017 - again, with variations from year to year - but have now increased for two years in a row to 759.

III c. Improve the gender balance in STEM subjects studied at college and university.

Data Source: Higher Education Statistics Agency (HESA): Student data Scottish Funding Council (SFC): College FES data

The percentage of female enrolments across STEM courses at universities has, in general been increasing marginally from 2014-15 to 2018-19. In 2018-19, it ranged from 67% for the biological sciences to 20% for Engineering and Technology and 21% for Computer Science. There were very slight increases in the proportion of female enrolments in engineering/technology and computer science between 2014-15 and 2018-19 but a slight decrease for mathematical sciences.

In the college sector, while there remains a lot to be done, progress has been encouraging. The percentage of female enrolments across STEM courses at HE level had been gradually increasing from 13.9% in 2014-15 to 17.8% in 2017-18, but fell to 13.9% in 2018-19. In 2018-19, they ranged from 53.8% for Business Management and Administration to 4.2% for Nautical Studies and 6.9% in Engineering. At FE level, over the five year period there has been an increase in the proportion of females taking STEM courses from 30.8% in 2014-15 to 31.3% in 2018-19. While figures are variable across the types of course, there are promising increases for Engineering with female enrolments increasing from 14.8% in 2014-15 to 24.0% in 2018-19.

III d. Increase gender balance in the uptake of STEM related Foundation Apprenticeship opportunities in the senior phase of school.

Data Source: Skills Development Scotland (SDS): Annual Equality Action Plan report

Females represented 8.1% of starts in STEM Frameworks for cohort 1 (2016-18). This increased to 13.1% of starts in STEM frameworks in cohort 2 (2017-19) and 20.6% of starts in STEM Frameworks for cohort 3 (2018-20).

More females are choosing the social services frameworks and more males are focusing on engineering ('Civil Engineering' and 'Engineering' frameworks). However, the proportion of females participating in the male dominated frameworks has increased since cohort 2, so there has been some progress made.

III e. Increase the proportion of schools from most deprived quintile that receive a quality STEM engagement experience from funded Science Centres.

Data Source:
Science Centre quarterly and annual reports & Annual Science Festival reports to Scottish Government
Scottish Government: Summary Statistics for Schools in Scotland;
Scottish Government analysis of School contact and SIMD Data

Baseline taken as 37.9% in 2016-17, the year prior to the launch of the STEM Strategy and the first to use data from SIMD16.

The proportion in 2018-19 was 62.5% of eligible schools, which represents a large increase on previous figures. Changes in policy and bedding in of increased funding in 2016-17 could account for the majority of this. However, figures may also be affected by the re-configuration of SIMD data zones in 2016 and a slight decrease in the total number of eligible schools.

III f. Increase the number of members of community groups from the most deprived or rural areas participating in quality engagement with Science Centres and festivals to 15,000 by 2022.

Data Source:
Science Centre quarterly and annual reports & Annual Science Festival reports to Scottish Government

The baseline has been taken as 8,235 visits by members of eligible community groups in 2016-17, the year prior to the launch of the Strategy and the first year of increased subsidy. The trend since 2012-13, when the subsidy was initiated separately, has been for increasing numbers of participants as the Centres have developed relations with community groups in their areas. The subsidy was increased in 2016-17 with an increase of over 1,000 participants from the previous year. In 2017-18 the number of visits increased again to 8,604, and, in 2018-19, 11,505 people from community groups across Scotland visited our Science Centres, exceeding the 10,000 target. As such, we have increased this target to 15,000 by 2022.

IV. Increased numbers of people who understand the benefits and value of STEM for themselves, their families and their communities. (Inspiration)

IV a. Increase the proportion of young people who say they feel studying STEM is important for their future careers in the Young People in Scotland Survey.

Data Source:
Young People in Scotland Survey

Results from the 2019 Young People in Scotland survey showed that the proportion of respondents who said they had or would choose to study STEM dropped from 65% in 2017 to 61% in 2019. Although it may take time for the actions in the strategy to affect this KPI, we will review our actions on inspiring young people for STEM in light of these results.

V. Increased collaboration between schools, colleges, universities and employers (Connection)

V a. Increase the number of employers engaged with education to support young people of all ages to understand STEM career opportunities and develop skills for work (including career advice, work inspiration, work experience placements, etc.)

Data Source:
Education Scotland: Annual practitioner and STEM CLPL provider surveys DYW Regional Group KPIs

Of the 1,187 early learning and childcare, primary, ASN and secondary practitioners that responded to Education Scotland's Annual STEM Practitioner Survey 2019, 42.1% said that their establishment has a STEM partner(s) from the private, public or third sector in the period from 1 August 2018 to 31 July 2019. This is a significant increase on the figure of 25.9% in the 2018 survey.

New KPIs have been put in place for the DYW regional groups that ask them to split their core activity (employer engagement) in to STEM and non-STEM. This activity is benchmarked for 2018-19 although the figures are only estimates and will not capture all employer-school activity and partnerships in the area. Some of the data now being collected will be available on a quarterly basis, but others only yearly.

Provisional information from the DYW Regional Groups indicates that, in the first half of the current reporting year:

• The Regional Groups arranged opportunities for 78,910 young people to engage with employers.
• Of those young people 19,850 of them involved engagements with employers from STEM industrial sectors.
• This amounts to 25.3% of young people's engagement in the first half of the year.

VI. Increased employment in STEM-related occupations and employers are more satisfied with the STEM skills and capability of the people they employ from schools, colleges, universities and from apprenticeship programmes. (Connection)

VI a. Increase the numbers of placements and internships with employers for college learners within STEM curricular areas.

Data Source:
Scottish Funding Council (SFC)

The Scottish Funding Council collects data from the college sector on student work placement opportunities, and the number of enrolments to STEM courses with work experience elements has been provided here. Figures have increased from 1,152 in 2015-16 to 2,223 in 2018-19.

VI b. Reduce the proportion of STEM employers in Scotland experiencing skills shortages.

Data Source:
UK Employer Skills Survey (ESS)

The proportion of STEM employers in Scotland reporting at least one skills shortage vacancy was 6.4% in 2015 and 7.7% in 2017. This is higher than the rate for all employers in Scotland which was 6.0% in both 2015 and 2017. Skill shortage vacancies as a proportion of all vacancies amongst STEM employers was 27.4% in 2015 and 24.2% in 2017. This is higher than the rate for all employers in Scotland which was 24.1% in 2015 and 23.6% in 2017. However, amongst STEM industries with health industries excluded, skills shortage vacancies as a proportion of all vacancies was 22.8% in 2017 and this is lower than for all employers in Scotland.

Opportunities are being explored to further revise the measurement of this KPI through data on STEM employer attitudes on the preparedness of education leavers (EPS data).