Innovation data baseline: final report

2. Measuring Innovation

2.1 Why Support Innovation?

Innovation has been a consistent policy priority for many years, both in Scotland and the UK and internationally. Widely considered essential for economic growth and productivity, innovation drives the development of new products and services or improving existing ones, increasing sales, and thus economic output, and ultimately creating wealth and employment.

Innovation in Scotland

Founded in 2014, Edinburgh Molecular Imaging is helping doctors and clinicians see the invisible with molecular imaging technology. Based on fluorescent imaging, the technology has the potential to detect disease in real time during interventional procedures including surgery, providing more accurate treatment while sparing healthy tissue. This is done by literally illuminating diseased tissue with fluorescent dye, providing doctors with a clear view of the extent of the disease

Research and development (R&D) is often considered central to innovation and innovation policy, but innovation is, of course, more than R&D, important as that is. While many innovations are technological - faster computers, more powerful phones and more fuel efficient cars - innovation is also about doing things better.

While the link between innovation and productivity is complex, there is evidence to suggest that innovation plays a key role in productivity growth. Indeed, work by the OECD and Nesta suggest that innovation could account for between 25% and 50% of labour productivity gains.^[14] Productivity growth will come through increasing output at a rate faster than employment growth, but also through increasing the efficiency of firms' operations.

However, the benefits arising from R&D and innovation more widely are not just economic. Innovation helps us develop better medicines, more effective public services and greener energy with resulting social and environmental benefits.

Although comparable estimates are not available for innovation as more widely conceptualised (as compared to R&D), a review of existing evidence by RAND Europe^[15] argued that there may be even greater benefits across society than the economic R&D estimates suggest (through 'spillover' effects), including benefits on culture, public engagement, social cohesion and environment, even if these are difficult to measure.

Innovation in Scotland

Facing high and rising levels of violent crime, the police in Scotland adopted in the mid 2000s a new public health approach to the issue. Interdisciplinary, science-based and multi-partner, this innovative model pioneered a new approach to identifying the drivers of violent crime and targeting resources at prevention as well as law enforcement. Since its adoption, crime statistics have dramatically reduced, and countries across the world have sought to learn from Scotland's approach and implement similar models.

Therefore, there are compelling reasons for governments to invest in measures that encourage and support innovation.

2.2 Measuring Innovation

Comprehensive and robust measurement of innovation and its impacts is challenging from both a conceptual and a practical perspective. We have reviewed the measures and indicators currently collected by the Agencies to track their investments and note that these are predominantly in line with their Business Plan objectives/targets and take account of all of the Agencies' activity. A sample of these are considered later (and in more detail in the Appendix).

We have instead adopted a more conceptual approach as our starting point. Rather than attempt to develop a measurement framework based on current practice, this approach seeks first to articulate an 'ideal world' model for measuring innovation support and its impacts, and then to assess current practice against this model. This will help identify gaps and suggest areas for future data collection.

Defining the Scope

The definition of innovation that has been developed by the Scottish Government's Enterprise and Skills Analytical Unit to guide this work is as follows:

"new ways of combining existing (and/or new) resources to better address existing (and/or new) needs".

This is an undeniably broad definition, and it is worth pointing out that much of the literature on innovation, and innovation measurement, is based on economics and management science. This has given the field a strong bias towards economic models of innovation that posit economic motives and outcomes.

While these economic outcomes will be a primary focus, it is important to bear in mind that innovation extends beyond the economic domain, and indeed is a crucial driver of wider social progress in fields as diverse as health and wellbeing, environmental protection and public services. This is also consistent with the ambition of the Scottish Government to be in the top quartile of OECD countries not just for productivity but also equality, wellbeing and sustainability.

An 'ideal world' framework for the measurement of innovation support should be able to accommodate this diversity.

2.3 Conceptual Framework

The measurement framework must be underpinned by a robust conceptual model that explains the processes (and direct/indirect linkages) by which the target outcomes and impacts may be achieved. In the context of innovation, there is a rich and complex literature on which to draw, and different frameworks and models of innovation have evolved over time.

A useful review of these can be found in the OECD Oslo Manual (2018),^[16] which provides detailed guidance on measuring innovation at a macro level. While very useful as a reference guide, the Oslo Manual is intended to inform national survey design and data collection rather than to assess the more specific impacts of particular innovation support activities. Nonetheless, it does provide much in the way of valuable insight, particularly around the nature of innovation processes and the roles of different actors within an innovation system.

It is helpful to begin with a consideration of innovation and how it occurs. Innovations derive from knowledge, existing or new - but innovations are more than ideas. They depend on the implementation of new or existing ideas/ knowledge. Thus there are at least two distinct processes at work - the creation of knowledge and its application (with numerous interactions in-between). This is implicit in the definition above.

This could be considered an oversimplification, and one that supposes this to be a linear process, when the reality is more complex and iterative. It also does not account for the fact that knowledge may be created in a wide range of ways, and the processes for acquiring and applying knowledge or ideas are also many and varied. Similarly, it is important to note that innovation is not solely undertaken for commercial purposes, and that a wide range of organisations will have a role in the innovation system.

Rather than seek to tackle the complexities of innovation systems directly, we have instead sought to simplify the conceptual framework and accompanying theory of change while retaining sufficient breadth that a wide range of innovation activities and interventions can be accommodated.

The key categories have been mapped out against an 'idealised' notion of an innovation journey from knowledge creation to eventual exploitation (commercial or otherwise). For now, we are disregarding the fact that such a linear model is not the reality. In fact, one step does not necessarily lead to another and there may be feedback loops. Failures will occur along the way, and some ideas will take a longer time than others to be developed. Some innovations may also need more development than others before reaching application, and others may move forwards and backwards through the process from idea to application. Some will never progress at all.

It is also the case, as demonstrated by the Covid-19 crisis, that new and even unforeseen demands and issues can impact on the process and on the timing and scale of any outcomes at different stages in the innovation journey.

Thus, when interpreting and applying the proposed framework, it is important to bear in mind these complexities. The key elements of the proposed framework are summarised in Figure 2.1, over, and described in the section that follows.

Figure description:

This diagram models the journey of innovation. It shows the types of activities present in each stage of the innovation journey and how they link up with one another.

Knowledge Creation

This includes a variety of means through which knowledge/ ideas are created. Academic and scientific research are obvious inclusions, and interventions here would include investment into research activities, but also investment into research infrastructure and into the development of research capacity and capability. These are typically subject to considerable public investment as the potential for commercial return is highly uncertain, and in many cases, unlikely. In the context of the current project this is very much the focus of SFC's innovation interventions.

Beyond the academic community, this would also include early-stage (e.g. Technology Readiness Level stage 1 - 3) research activities within firms, and other organisations, as well as individual invention. Models of open innovation and user innovation rightly argue that knowledge and ideas can come from a broad range of sources, including competitors, suppliers and customers, as well as within organisations themselves. Interventions in this area would include support for organisations to undertake research, market research and internal review work (e.g. for inclusive business model innovation). Some support of this nature may be provided by the enterprise agencies, for example through R&D grants.

Innovation Capacity

Just as research capacity is essential to the creation of new knowledge, so an innovative economy depends on the capacity of organisations and individuals across the economy to undertake innovation activities. Knowledge creation and innovation capacity could be considered the essential building blocks for an innovative economy.

The first aspect here is awareness and understanding of innovation and its potential value (to organisations, the economy and society). Thus, interventions which serve to raise awareness and build understanding of the value of innovation would belong here. This could also include teaching and training at all levels of the education system, essential to developing innovative capacity within individuals. However, this is outwith the defined scope of the current review.

The capacity for innovation will also depend on the assets and resources available to organisations, including the skills and expertise of staff (business leaders and 'operational' staff), the availability of suitable investment (public and private sources) and the organisational structures that can support innovation.

This category includes a broad range of possible interventions from skills and leadership to organisational development support. Many of these kinds of interventions are delivered by the enterprise agencies (and arguably also by Skills Development Scotland, although not included within the current scope).

Knowledge Flows and Diffusion

Innovation rarely happens in isolation, and is more often a collaborative effort and is typically shared or developed in collaboration whether between academic researchers and external organisations, or organisations and partners, competitors or customers or even amongst staff within organisations. An effective innovation system is therefore one that supports and enables the flow and exchange of information.

This is rich territory for intervention and support here includes infrastructure and investment that supports, encourages and incentivises knowledge sharing. Most obviously, there are substantial investments in knowledge exchange activities between universities (and colleges) and external organisations (such as industry). This category also includes business to business innovation support and activities that support the wider uptake of existing knowledge, ideas or technologies that can support innovation (e.g. high speed broadband roll out).

The former is a major area for SFC investment while the enterprise agencies tend to be more focussed on the latter, although in practice there will be programmes in this category that are supported by all three agencies.

Innovation Development

As noted above, innovation is about the implementation of ideas or knowledge (new or existing). This process overlaps with knowledge flows and diffusion but generally extends further through the iterative development and testing of new products, services, processes and business models until they are ready for market application.

This essentially comprises the development phases for research and experimental development and can include feasibility work, proof of concept testing, prototyping, user research and testing, design and development. Again, this is a common area for intervention, not least as commercial investment may still perceive this to be too high risk. Thus, there is a strong tradition of public investment in organisational innovation development.

This is the main focus for the innovation support (by value of investment) provided by HIE and SE.

Application and Exploitation

The final category, or stage in the innovation process, would be the application of innovations, usually in the form of new products, services or business models being brought to market (commercial or otherwise) or new processes being introduced.

This area is less often the subject of public sector intervention largely because private investment is more likely here than elsewhere on the innovation journey due to the greater predictability of likely financial returns. There is also a less compelling 'market failure' rationale for public sector intervention at this stage (although this does not prohibit the public sector investing in "public goods", which is a recognised market failure).

For example, a technology firm will invest in taking a new technology to market having received support (of various kinds) to develop the technology to this stage. However, some forms of public sector support may apply including investment and loan finance, supply chain sourcing and export development. Again, the enterprise agencies are more likely to invest in this area. SFC would not invest in these kinds of activities.

It is important to note that intervention and investment occur at each stage in the innovation process or journey as summarised above. Thus, for each of these categories there will be a set of inputs, which support specific activities, generate outputs and lead ultimately to the intended impacts. Some of these outputs will then feed into subsequent stages of the innovation process, while others will end at that point.

This reflects both the different drivers and motivations at play for different actors across the innovation system (with respect to the objectives of the interventions these can be quite varied and divergent), and also the failure rate along the journey from knowledge creation to market application.

This framework is, as noted, a simplification of real world practice in innovation. However, the categories are purposely designed to be sufficiently broad to accommodate the range of innovation support provided by the public sector.

What is does not include is consideration of the external factors that can affect the nature and extent of innovation within individuals, organisations and an economy, such as:

• market forces and unforeseen shocks (such as COVID 19);
• policy and fiscal measures (e.g. R&D tax credits); and
• social and environmental factors (e.g. climate change prompting innovation in low carbon technologies).

2.4 Theory of Change

In order to be implementable and robust as a framework for measurement, the identified categories should provide a coherent account of a route to impact from the initial investment to the intended outputs and eventual impacts (economic, social, environmental, etc.).

Knowledge Creation

Investment in knowledge creation, and the capacity for knowledge creation, would be expected to result in new knowledge and ideas, or novel interpretation of existing knowledge. That resulting knowledge would then require to be developed and used such that it then returns value in different ways. Some of this knowledge would be shared such that its primary impact will be in contributing to understanding in a particular domain or discipline, while other forms of knowledge will continue to be developed more widely.

A new interpretation of a Shakespeare play, for example, would contribute to the understanding of the work of the playwright, but may go no further than this (an end in its own right), while scientific research might help develop a new technology which then might progress through the innovation stages described above to result in a new product reaching the market and delivering financial, economic and other wider social returns. As such, while basic research activity is often focussed primarily on the advancement of knowledge, a proportion of the outputs from this activity will contribute (eventually) to economic impacts such as productivity gains, albeit usually as a result of subsequent investment and development activity. In this way, research activity and its immediate outputs in the form of codified knowledge might be considered 'capital stock' which might, at any point in time, be utilised for the purposes of innovation.

The immediate or direct outputs arising from the investment in research activity would be mainly in the form of traditional academic outputs that seek to codify knowledge - research papers, conference presentations etc. - or could also be in the form of new technologies e.g. software code, or even cultural outputs. It is, however, too simple to propose that these outputs would then be picked up by other actors in the innovation system and developed to final application. It is more likely that the knowledge gained through research effort would be exchanged with other actors through interaction between researchers and others.

Indeed, such interactions also often result in new knowledge being created. Thus the linkages between knowledge creation and subsequent stages in the innovation process are not always straightforward.

It is also important to note that the drivers for supporting basic research in universities will often be based more on the value of knowledge as a public good than they are on the potential for future economic impact.^[17] This is not to say that there will be no benefit or impact from investing in this activity. High quality scholarship (as measured by strong research performance) would be expected to result in reputational gains for institutions, attracting further research talent and investment, as well as students. Strong scientific research capacity is also a magnet for often high value inward investment as witnessed, for example, in the Cambridge biotech cluster.

The knowledge created may also eventually deliver benefits to society in ways that are not directly economic but may have economic consequences. For example, psychological research into child development may lead to revisions to guidance on road safety education that could subsequently develop into practice, leading to a reduction in traffic accidents amongst children, and savings in relation to hospital admissions. In this way, the original investment in research provides the inputs to a process that delivers both social and economic benefit.

Innovation Capacity

As noted above, an effective innovation system requires that sufficient organisations (and individuals) have the capacity and skills required to undertake innovation. Building this capacity is therefore a valid policy objective.

Support in this respect would be expected to increase the numbers of organisations that understand the potential benefits/impacts from being more innovation active, and have the capacity to undertake innovation. This in turn would be expected to lead to more organisations investing in, and undertaking innovation ('innovation active'). This activity could lead to the development of new products, services and processes and increased levels of economic and social benefits (e.g. through new sales, increased efficiency, employment and productivity gain). In particular, more innovation active companies should be more productive.

Knowledge Flows and Diffusion

Ensuring the flow of knowledge, ideas and information across the innovation system is critical to its effectiveness. It is often the case that the sources of knowledge are not best placed to turn that knowledge into innovation. Facilitating and enabling these connections helps build a pipeline of possible innovation prospects that can then, through a series of further steps, be developed into innovations that are then applied and exploited in different market contexts.

As noted above, new knowledge may be created through these interactions (for example between universities and industry), and the process is not always as simple as the transfer of knowledge from one party to another. This suggests some overlaps between the stages of the innovation journey as set out above, and is an example of the complexity in innovation processes.

An example of the types of direct intervention here includes knowledge exchange programmes that connect universities and firms. For example, firms may take advantage of research expertise and specialist equipment within the university to validate a technology, or a university may recruit commercial experience/expertise from the private sector to support a university spin-out to secure external investment.

This activity may result in a successful output where the actors take forward a collaborative project to the next stage of the innovation journey (innovation development). Similarly, there may be no quantifiable output although it could help the actors recognise the value in collaboration/knowledge exchange, making them more open to engaging and collaborating in the future, thereby stimulating the innovation pipeline.

Knowledge flows and diffusion do not always require university (or college) involvement. Knowledge flows between companies and their suppliers, competitors and customers are also important, and knowledge may also transfer (or leak) with changing personnel within firms. These effects may be facilitated by interventions designed to stimulate these networks.

At this stage again, it should be expected that a proportion of activities arising within and from these interactions will progress no further.

Innovation Development

Within this broad category, investment into innovation projects will, through a variety of means, move ideas and knowledge closer to application via a process of product, service and process development (including business model and workplace innovation).

This may include taking forward the outputs of knowledge exchange activities, but may also develop ideas created or acquired in other ways (e.g. from within organisations). This is also likely to be a somewhat iterative process where projects may need to 'go back' a few steps in the process to seek further inputs (e.g. go back to researchers for input on feasibility or advise on amendments to products). As with other stages in the journey, a degree of failure is also to be expected, and should be considered in assessing the outcomes of testing and development processes.

The outputs should be a series of innovations (for example in the form of prototypes, market tested propositions etc.) that are closer to being market ready, having been tested, developed, designed and refined to improve their chances of eventual success. As a result, investment and support for the development phases of innovation might be expected to result in more innovations reaching the market with a higher change of success, and therefore greater prospect for economic and/ or social impact. This would again include productivity gains.

Application and Exploitation

In this final stage, innovations are brought to market. While knowledge itself is essentially a non-rival good (it is available to all), organisations may wish to protect their investments by making relevant knowledge excludable. The primary mechanisms for this are various forms of legal protection for intellectual property (IP). Of course, IP protection could take place at a much earlier stage in the innovation process but is included here as a means of exploiting innovation (usually for commercial purposes).

This stage is also when innovations are launched on the market and investments and activities here would be expected to lead directly to economic and social benefits (again there will be successes and failures). Agency interventions have a role in bringing these innovations to market and, in particular, promoting Scottish innovations to a global audience via export markets.

Thus a new product may lead to increases in sales and a new or improved process may result in productivity gains. A new healthcare treatment may result in public health benefits (e.g. people living healthier lives for longer) and a consequent reduction in spending on acute care services.

Linkages across the Model

Across the model, attributional links can be made (direct, indirect and leveraged) but these are more complex than may first appear. So, while knowledge is an essential input to innovation, not all knowledge will lead to innovation.

In addition, the outputs of research activity may not be a direct input to subsequent development - as noted above, much of the transfer of knowledge is mediated by interactions between people. Measuring these interactions can be challenging unless they fall into the predefined categories of exiting data collection systems (discussed below). So, attributing and tracking/measuring eventual gains in economic output to general investment in basic research in universities is something of an inferential leap for a number of reasons:

• the original investment is likely to have been made with different policy goals in mind (e.g. education and knowledge as a public good rather than an economic driver);
• numerous intermediate steps (including potential failures) and investments are required to implement the outputs of research in society or the economy, these benefits cannot be solely attributed to the original research investment; and
• many of the activities supported by the original investment may never contribute to innovation in this way, therefore economic measures alone should not be used to judge the success (or otherwise) of the original investments.

Nonetheless, in this example, the investment in research is necessary but not sufficient to support innovation. The scale of the investment into research capacity in the academic base (through the SFC Research Excellence Grant) makes this particularly relevant to the current study. If a (possibly sizeable) proportion of research activity will not ever deliver a significant economic return (bearing in mind that this may not be its primary purpose), then a simple financial or economic return on investment calculation may not do justice to its wider social value. Instead, measures of social impact would be required.

Elsewhere in the model, linkages are again present, and in some cases may be easier to measure. Innovation requires organisations with the capacity to undertake innovation activities. Thus, this can be considered an essential input again to the innovation process. Within the Innovation Development category, product or service development may proceed through a series of stages from early feasibility through testing and prototyping to production (for example), the output of a new product, service or process is then an input to the final category of application and exploitation.

This model and theory of change presents one way of considering innovation, and seeks to provide a framework to articulate the interconnectedness of different activities (and different areas of innovation support) and how they contribute to the expected end goals of social and economic benefit. Two further points are worth making:

• it should be expected that a proportion of projects within each category may not advance further for a variety of reasons, not all of which may be negative (e.g. where research identifies that a technology will not work, thereby saving further investment);^[18] and
• the outputs of different categories may feed into further activities elsewhere in the model but may also lead directly to economic and social benefit (e.g. productivity) without further support.

Spillovers

One of the (often unintended) outcomes and benefits from engaging in, or undertaking, innovation is "spillover effects". The basic principle is that new knowledge created through, for example, academic research has a number of potential applications by a number of different actors, outwith its intended application. In the context of innovation these can be regarded as "knowledge spillovers" and whilst these effects can occur at every stage of the innovation journey (for example, technology spillovers within the upstream and downstream supply chains or market spillovers where an applied/exploited innovation benefits society more generally), they are most commonly recognised at the knowledge creation stage.

The presence of spillovers are often referenced as a market failure/efficiency rationale to justify public sector intervention - the underlying theory being that, while the input costs of innovation (e.g. on R&D) may be known, the impact and wider benefit cannot be known and therefore priced/monetised. The end result being a firm chooses not to invest in, for example, a particular innovative process or product.

There is also the potential for these spillover effects to lead to externalities, whereby competitors or "free riders" are able to absorb or "appropriate" this knowledge in ways that the resulting benefits cannot be captured by the knowledge producer / product innovator. Again, the end result is often under investment in innovation.

Research undertaken by BIS^[19] provides evidence on the positive relationship between knowledge spillover and productivity, which will; often have the positive effects of "inducing complementarities in R&D efforts" amongst actors (e.g. competitors), which could have a further downstream positive impact on productivity.

This leads to an interesting dichotomy, in that, at the individual knowledge producer / product innovator level they may seek to minimise the opportunity for competitors or other actors to benefit from their knowledge, whilst at the macro level the presence of spillovers is viewed as a positive outcome as it often leads to greater investment (and returns) in knowledge creation.

While we would note the importance of capturing/highlighting knowledge spillover effects as a longer term outcome of innovation activity within the theory of change and logic model, developing metrics to capture these effects may not be appropriate and best undertaken via other approaches such as large scale econometric analyses, qualitative evaluation and case studies, etc.

For example, at the individual academic institution or firm level (where the responsibility for gathering and reporting performance monitoring data often resides) the direct benefits and impacts of developing a new product or process may be measured through straightforward metrics such as sales or profit per unit of output.

However, what is harder to identify and measure is if this knowledge has been adsorbed or appropriated by other actors in the economy and, if so, the extent to which it is having a measurable positive impact. There may also be significant time lags in the creation of knowledge and the emergence of spillover benefits for other actors.

It is therefore extremely challenging to accurately attribute, track and quantify the wider knowledge spillover impacts/benefits from one particular intervention to other firms, industry sectors, countries, society, etc.