Triple Helix

A Journal of University-Industry-Government Innovation and Entrepreneurship

Triple Helix Cover Image
Open Access

Assessing economic impact of research and innovation originating from public research institutions and universities—case of Singapore PRIs

Triple HelixA Journal of University-Industry-Government Innovation and Entrepreneurship20163:6

DOI: 10.1186/s40604-016-0037-6

Received: 12 July 2015

Accepted: 31 March 2016

Published: 13 April 2016

Abstract

Research and innovation activities from universities and public research institutes (PRIs) contribute to economic growth in significant ways. Given the hefty investment in R&D in both public and private sectors, it is essential for policy makers to have meaningful, relevant and practical metrics for measuring the impact of public-funded research and innovation. However, the definition of impact and its measurements seem to differ across the literature, making it difficult for scholars and policy makers to inform policy and direct future research. This paper aims to consolidate all parts of the proverbial elephants by reviewing the extant literature for university/PRI impact studies, analysing and synthesising the erstwhile disparate studies, discussing the current metrics and their issues and identifying gaps among the different research strands. Building on the endogenous growth theory and the Triple Helix, we contribute to the body of knowledge by offering a multi-level Impact Assessment Framework for assessing the economic impact of public research. In the context of the framework, we applied the transaction cost economics theory and net present value concept to propose a new indicator, realised imputed commercial value (RICV). We empirically establish that RICV is a better predictor than the traditional licensing revenue metric used by universities/PRIs of firms’ propensity to repeat licensing agreement with them, thereby indicating academic innovation impact.

Keywords

Innovation impact Technology transfer Commercialisation Public R&D Net present value

Resumen

Las actividades de investigación e innovación de las universidades e institutos de investigación públicos (PRI) contribuyen al crecimiento económico de forma significativa. Dada la fuerte inversión en I&D en los sectores público y privado, es esencial para legisladores contar con medidas significativas, relevantes y prácticas para medir el impacto de la investigación y la innovación con fondos públicos. Sin embargo, la definición de impacto y sus medidas parecen diferir a través de la literatura. Esto dificulta la toma de decisiones óptimas sobre el financiamiento y la dirección de la investigación pública. En este trabajo deseamos consolidar todas las partes del elefante proverbial mediante la revisión de la literatura de impacto, el análisis y la síntesis de estudios existentes, la discusión de fortalezas y debilidades de medidas cuantitativas en uso, y la identificación de brechas entre las diferentes líneas de investigación. Basándonos en la teoría del crecimiento endógeno y la Triple Hélice, contribuimos al cuerpo de conocimiento, ofreciendo un marco de evaluación multi-nivel del impacto económico de la investigación pública. Combinando este marco, con la teoría de costos de transacción y el concepto del valor actual neto proponemos un nuevo indicador: Valor comercial imputado realizado (VCIR). Establecemos empíricamente que VCIR es un mejor predictor de la propensión de las empresas para repetir negocios con con universidades/PRI que el indicador tradicional de ingresos por licencias.

Keywords

Innovation impact Technology transfer Commercialisation Public R&D Net present value

Résumé

Les activités de recherche et d’innovation des universités et instituts de recherche publics contribuent à la croissance économique de façon significative. Vu le lourd investissement dans la R&D à la fois dans les secteurs public et privé, il est essentiel que les décideurs politiques disposent d’indicateurs significatifs, pertinents et pratiques pour mesurer l’impact de la recherche et de l’innovation financée sur des fonds publics. Cependant, la définition de l’impact et ses mesures semble varier à travers la littérature, rendant difficile l’élaboration de la politique et l’orientation de la recherche future par les universitaires et les décideurs politiques. Cet article a pour objectif de lever les différents non-dits sous tous leurs aspects en passant en revue la littérature existante sur l’impact des universités et instituts de recherche publics, en analysant et en synthétisant les études autrefois disparates, en discutant les mesures actuelles et leurs insuffisances, et en identifiant les lacunes des différents aspects de la recherche. En nous basant sur la théorie de la croissance endogène et la Triple Hélice, nous contribuons au corpus de la connaissance en mettant à disposition un cadre d’évaluation à plusieurs niveaux de l’impact économique de la recherche publique. Dans ce cadre, nous avons appliqué la théorie économique du coût de transaction et le concept de la valeur actuelle nette pour proposer un nouvel indicateur, la Valeur commerciale réalisée imputée (VCRI). Nous établissons empiriquement, que la VCRI est un meilleur prédicteur que la mesure traditionnelle des revenus de licence utilisée par les universités et les instituts de recherche publics de la propension des entreprises à renouveler avec eux un accord de licence, ce qui indique l'impact de l'innovation universitaire.

Keywords

Innovation impact Technology transfer Commercialisation Public R&D Net present value

摘 要

起源于大学和公共研究机构(PRIs)的研究与创新活动对经济增长方式贡献显著。鉴于公共和私营部门都在研发方面大量投资,政策制定者们必需有有意义的、相关的和实际的指标,来衡量公共资助研究和创新的影响。然而,影响的定义及其测量在文献资料中似乎存在着不同,很难让学者和政策制定者们提出政策和进行未来的研究。本文旨在通过回顾对大学/PRIs的影响研究的现有文献,分析和综合以往零散的研究成果,讨论当前的度量标准及其问题,确定这些不同研究的缺失。基于内生增长和三螺旋理论,我们通过提供一个多层次影响评估框架评估公共研究的经济影响,以丰富相关的知识宝库。在此框架背景下,我们应用交易成本经济学理论和净现值概念提出一个新的指标 —— 已实现的估算商业价值(RICV)。凭经验可以确定:照比传统的被企业倾向的大学/PRIs用于重复许可协议的许可收入指标而言,RICV是一个更好的预言家,能表明学术创新的影响。

Keywords

Innovation impact Technology transfer Commercialisation Public R&D Net present value

Аннотация

Исследовательская и инновационная деятельность в университетах и публичных исследовательских институтах (Public Research Institution, далее – PRIs) оказывает влияние на экономический рост различными путями. Осуществляя разумные инвестиции в исследования и разработки как в публичном, так и частном секторе, представителям власти необходимо иметь результативные, актуальные и практические инструменты для измерения воздействия на исследования и инновации, финансируемые публично. Однако, методики определения влияния и его изменения могут различаться в различных литературных источниках, усложняя оценку для представителей правительства и ученых при определении достигнутых результатов и координации дальнейших исследований. Целью настоящей статьи является объединение накопленных знаний в данной сфере путем обзора релевантной литературы по тематике управления PRIs, последующего анализа и синтеза нестандартных теорий, обсуждения известных методологий и их оценки, а также идентификации недостатков в различных исследовательских подходах.

Формируя эндогенную теорию роста в рамках Тройной спирали, мы осуществили вклад в базу знаний, предложив многоуровневую модель оценки влияния публичных исследований на экономику. В рамках данной модели мы применили теорию трансакционных издержек и концепцию оценки чистой приведенной стоимости, предложив новый показатель – Реализованная Вмененная Коммерческая Ценность (Realised Imputed Commercial Value, далее – RICV)). Эмпирическим путем нами было показано, что RICV представляет собой лучший индикатор, чем традиционная методика определения выручки, используемая создаваемыми при университетах фирмами для продления лицензионных соглашений с ними, тем самым характеризуя влияние академических инноваций.

Keywords

Innovation impact Technology transfer Commercialisation Public R&D Net present value

Resumo

As atividades de pesquisa e inovação em universidades e institutos de pesquisa públicos (IPPs) contribuem de maneira significativa para o crescimento econômico. Dado os fortes investimentos em P&D em ambos os setores público e privado, é essencial para os formuladores de políticas terem métricas úteis, práticas e relevantes para mensurarem o impacto do financiamento público em pesquisa e inovação. No entanto, a definição de impacto e suas mensurações parecem divergir ao longo da literatura, tornando difícil para os pesquisadores e formuladores de políticas pública retornar informações para a política e direcionar estudos futuros. Este artigo visa consolidar todos estes aspectos através de uma revisão da literatura existente sobre estudos de impacto para Universidade e IPPs, analisando e sintetizando estudos díspares de outrora, discutindo as métricas atuais e as suas questões, e identificando as lacunas entre as diferentes vertentes de pesquisa. Com base na teoria do crescimento endógeno e na Hélice Tríplice, nós contribuímos para o conhecimento teórico através de um quadro de avaliação de impacto multi-nível, para inferir o impacto econômico da pesquisa pública. No contexto desta estrutura, nós utilizamos a teoria econômica dos custos de transação e o conceito do valor presente para propor um novo indicador, o Valor Comercial Teórico Realizado (VCTR ou RICV- Realised Imputed Commercial Value ). Nós estabelecemos empiricamente que o VCTR é um indicador melhor do que as métricas de receita de licenciamento tradicionais utilizadas por Universidades e IPPs para medir a propensão de empresas em repetir os acordos de licenciamento, indicando, assim, o impacto da inovação acadêmica.

Keywords

Innovation impact Technology transfer Commercialisation Public R&D Net present value

Multilingual abstract

Please see Additional file 1 for translation of the abstract into Arabic.

Introduction

The role of public-funded research in the creation of new knowledge in the national innovation system is fast growing in importance (Etzkowitz et al. 2005). Research and research-related commercialisation activities from universities, public research institutes (PRIs) and health institutions contribute to economic growth in significant ways. The development of human capital in the academic and PRIs and their subsequent knowledge transfer through publications, consulting or migration to the industry could have profound social and economic impact (Cohen et al. 2002).

Although it is widely acknowledged that public-funded research has contributed to the intellectual capital, human capital and industrial capital to fuel economic development and advance the society, the assessment of such contributions has presented challenges to the government funding agencies that pursue national innovation system as part of their economic growth strategy. These issues are pertinent to the policy makers not only in North America (Vincett 2010; Roessner et al. 2013) but also in Europe and Asia Pacific. In the USA, for example, over USD37 billion had been spent annually on R&D by the National Science Foundation and the National Institutes of Health (NSF 2015; NIH 2015). Based on a recent survey of public research institutes in OECD countries, the absolute real R&D expenditure in the public sector increased from USD69.1 billion in 1998 to USD 84.3 billion in 2008, with the strongest being in Spain and Turkey at annual growth of over 9 and 16 %, respectively (OECD 2011). For a few OECD countries that reduced their government’s R&D intensity on PRIs during the period, a corresponding increase was found in the university’s R&D expenditure, thereby indicating the significance of PRIs and universities as two main actors of the public research systems.

Given the significant expenditures on the public research centres, it is essential for scholars and public policy makers to have meaningful, relevant and practical metrics for measuring the impact of public-funded research and innovation. These indicators serve as important tools to inform policy and decision-making, shape research agenda and optimise resource allocation. The public research commercialisation metrics currently used by many research commercialisation offices include the number of intellectual property (IP) licenses, IP licensing revenues and spin-out formation rate. While these primary metrics are important, relevant and easy to capture (AUTM 1991–2012), variation in methodological approaches across the studies does not facilitate benchmarking with similar organisations.

According to Metrics for Knowledge Transfer from Public Research Organisations in Europe published by European Commission (2009), differences in target populations and expected response rates, as well as the normalisation approach for variation in research expenditures and research resource input pose challenges for policy makers in comparing inter-university/PRI performance in their contribution to their regional economy. In addition, there is concern that the current indicators do not adequately reflect the value of the technological innovation that public research organisations have brought to the firm and industry (Roessner et al. 2013). Large-scale investments in academic R&D are generally rationalised by a broad set of public principles or priorities. However, the evaluations are typically made in strictly commercial terms. Although some R&D projects that might have been shelved or aborted did not yield any economic return, prior studies have shown that failed R&D projects could still contribute to the overall innovation process (Elmquist and Le Masson 2009). The first global attempt to measure the socio-economic impact of public-funded research comprehensively was made for the Australian Research Quality Framework. Case study approach was used and indicators verified and proposed. However, the framework was never implemented due to a change in government in 2007 (Penfield et al. 2014).

It is evident that the current public research impact literature has several issues that require attention. First, most studies assumed linear knowledge flow from universities/PRIs to the industry to achieve commercial spill-overs and deliver economic impact (Smith et al. 2011). Second, the lack of clear and consistent definition about the term “impact” and its host of related indicators may be seen in the diverse metrics used in the studies: R&D intensity (Nelson 1986), invention disclosures (Carlsson and Fridh 2002), publications (Nelson 2009), patents (Fischer and Varga 2003), technology licenses with companies (Bach and Llerena 2007), academic spin-offs (Colombo et al. 2010), innovative sales from licensees (Vincett 2010), as well as employment and income (Armstrong 1993). Third, the theoretical frameworks adopted varied widely from resource-based theory (Colombo et al. 2010) and CDM (Barajas et al. 2012) to input-output model (Roessner et al. 2013) and total factor productivity (Fischer and Varga 2003), while the methodologies ranged from individual case study (Brownrigg 1973) to econometric analysis (Felsenstein 1996). The use of different theoretical and methodological approaches has certainly enriched the literature with multi-disciplinary perspectives. However, these have also fragmented the literature, making it difficult for scholars and policy makers to perform meaningful international benchmarking of their innovation impact to inform policy and direct future research (Roessner et al. 2013, Penfield et al. 2014).

This paper aims to consolidate all parts of the proverbial elephants by reviewing the extant literature for university/PRI impact studies, analysing and synthesising the erstwhile disparate studies, discussing the current metrics and their issues, and identifying gaps among the different research strands. Building on the endogenous growth theory (Romer 1990) which advocates knowledge-based economic growth strategy and the Triple Helix model1 (Etzkowitz et al. 2005), we contribute to the body of knowledge by offering a multi-level framework for assessing the economic impact of public research through structuring the indicators into four groups of stakeholders: (a) inventive output of universities/PRIs, (b) commercialisation output of universities/PRIs by their technology transfer organisations (TTOs), (c) innovation of industry/firms and (d) community/ economy.

According to the Transaction Cost Economics (TCE) theory, market transactions are not costless and firms come into existence due to market failures. Buyers may have to bear various types of transaction costs ranging from costs of coordination to hazard of hold-up2 arising from asset specificity or IP appropriability3 (Williamson 1979). Before firms decide to procure external intellectual property (IP), such as in-license technologies from universities/PRIs, they will typically consider the “make internal” versus “buy external” options and weigh the relative costs and benefits of each option. Prior studies have shown that firms with satisfactory prior licensing experience and perception of high licensing benefits relative to costs have a higher propensity for in-licensing technology (Atuahene-Gima 1993).

Adopting the ‘objectivist’ approach (Klaes 2000) of objectively measuring the value of IP transaction to firms, we used the net present value (NPV) methodology (Fisher 1930; Magni 2009) that is regarded robust in theory and prevalent in practice for financial investment decisions to quantify the benefit of in-licensing PRIs’ IPs to the firms. This is done by totalling the sum of projected or extrapolated commercial revenue streams from the royalty-bearing IP licensing transactions - also known as Realised Imputed Commercial Value (RICV).

RICV could thus be objectively computed for every one-time or royalty-bearing IP licensing deal to estimate the sales revenue that firms generate from IP commercialisation. Unlike the resource-intensive methods that could estimate project benefits upon project completion (e.g. RVM4, BETA5 or NIST6), the RICV7 method uses annual revenue streams received by the TTO/PRIs as units of observations by relaxing the assumption that the R&D project benefits should or could be attributed to the percentage of PRI’s IP contribution. RICV also has the other advantages of being aggregated for firms at industry and country levels for micro-, meso- and macro-perspectives.

Building on the TCE theory and earlier works that established the positive relationship among licensing experience, benefits and propensity, we postulated in this study that firms’ benefits from in-licensing of university/PRIs’ IPs, represented by RICV, would be positively associated with their propensity to repeat in-licensing transactions with PRIs.

To examine empirically the relative significance of the concept of RICV, we posed the following research question: Is RICV a suitable predictor of firms’ propensity to repeat in-licensing agreements with PRIs, compared to the traditional licensing revenue indicator? Two hypotheses were formulated and binary logistic regression was used to test the model. The first hypothesis (H1) postulates that firms that incur higher IP licensing fees (costs) with universities/PRIs have lower propensity to repeat their IP licensing transactions, thereby reducing academic innovation impact on firms. The second hypothesis (H2) hence posits that firms that attain higher levels of RICV with universities/PRIs (benefits) have higher propensity to repeat their license transaction, indicating an increase in academic innovation impact on firms. With results confirming our hypotheses, we contribute to the literature of impact studies by proposing RICV as new secondary indicators to estimate the economic impact of public-funded technological innovation at the firm, industry and national levels.

We conclude the paper with discussion of the results and their implications.

Methodology

This study comprises two parts: (a) development of an Impact Assessment Framework and (b) the empirical investigation of the RICV concept as part of the framework. An exploration process was used for the first part while regression analysis of archived data of Singapore PRIs was performed for the second.

To develop a comprehensive framework for impact studies on public-funded research, we applied an exploration process comprising three stages: planning, execution and reporting (Tranfield et al. 2003). In the first stage of planning, we identified the key data source as the ISI Web of Knowledge’s Social Sciences Citation Index (SSCI) as it provides peer-reviewed journals which are regarded as validated knowledge of high impact (Podsakoff et al. 2005). During the second stage of execution, we determined initial selection criteria in the form of keywords and various combinations8. With these search keys, we refined our search of SSCI database to articles in English language under the research categories as business economics, educational research, public administration, urban studies, social sciences and science and technology for the period from 1973 to 2013. This results in an initial list of 299 papers. After reviewing their abstracts, we identified 192 relevant papers. To avoid reference bias, we searched Google Scholar9 with similar keyword combination queries to arrive at an initial list of 108 peer-reviewed journal papers, of which 89 were found relevant. After checking against duplicates, we compiled our consideration set10 of 275 papers for analysis and classification by the types of the impact indicators used and synthesis of conceptual framework. The framework will be presented in the next section.

For the empirical investigation of the RICV concept as part of the framework, we used licensing agreement transactions as the units of analysis in non-contrived settings. To control for inter-nation and inter-PRI variations, this study focused on PRIs governed within the same IP, science and technology and organisational framework of the national research agency of Singapore. Operating under the same agency, the PRIs are assumed to have the same propensity to out-license their technologies to enterprises. This study used the IP commercialisation database the agency maintains on organisations that had signed licensing agreements with them to deploy technologies developed by their PRIs for in-house use or commercial purposes for a stipulated period of time. The dataset comprised 835 licensing agreement transactions that took place between 2006 and 2012, innovation outcome of the firms in terms of innovative product count and RICV, as well as their status as repeat licensees.

In H1, the independent variable is the annual IP licensing fees incurred by a firm for in-licensing IP developed by the PRI, payable from the firm to the TTO representing the PRI, as executed in the TTO firm licensing agreement. The dependent variable is the firm’s propensity to repeat licensing, which is used as a proxy indicator of the impact that prior PRIs’ IP licensing and knowledge transfer has on the firm’s innovation output. This variable is binary with a value of 1 if the firm has repeated licensing agreements with the TTO or 0 if it has not.

In H2, the dependent variable is the same as that for H1, and the independent variable is annual RICV generated by the firm for executing an IP licensing agreement with the TTO. The annual RICV is made up of projected or extrapolated commercial revenue streams from royalty-bearing IP licensing deals for a particular year. Please refer to the Appendix for computation details and examples.

To control for variations contributed by other factors, control variables such as licensee organisation type and technological orientation of IP licensed are included in the study.

To validate H1, positing that firms that incur higher IP licensing fees with universities/PRIs have lower propensity to repeat licensing transactions, binary logistic regression is performed, as the dependent variable, firm’s propensity to repeat licensing is a binary variable, to examine how the independent variable, IP licensing fees, affect the dichotomous outcome. This approach of binary logistic regression had been used in prior study by Ambos et al. (2008) on the factors affecting the commercial output of academic research, also coded as a binary variable. In H2, the same binary-dependent variable used in H1, firm’s propensity to repeat licensing, is regressed using binary logistic regression on the independent variable, RICV.

Results

Public research impact studies

This section presents a descriptive analysis of our consideration set of 275 articles and reviews the dimensions of public research impact assessment posited in these studies. Figure 1 shows that 275 articles were published in a total of 50 journals.
Fig. 1

Public research impact articles published per academic journal 1973–2013

Given that impact studies may be undertaken at various levels of analysis, ranging from individual through institutional to national, the range of journals publishing such studies is correspondingly wide, from the fields of business economics to public administration. However, a closer examination of the journal titles shows that the distribution of journal titles is rather skewed, with three journals accounting for more than half of the journal population: Research Policy (35 %), Journal of Technology Transfer (12 %) and Technovation (5 %). While Research Policy has consistently topped the chart, Journal of Technology Transfer saw an explosion of publications in this area in the recent decade. There may be several explanations for this phenomenon. Using the lens of life cycle model to the development of new inquiry topic, the topic of technology transfer was in the nascent stage of development. It is noteworthy that following the US Bayh–Dole Act of 1980 and similar developments in other innovating economies, which encouraged universities and PRIs to patent their inventions with government funds, a rise in technology transfer from public research to the industry spear-headed a concomitant growth in the literature of technology transfer, particularly in the USA and Europe. As new topics of inquiry gain attention, the editorial teams of more prestigious journals that might be initially sceptical of their validity became more willing to publish them or new journals such as the Journal of Technology Transfer being developed to carry the scholastic conversations (Kuhn 1962). During the period of 1973–2013, we observe an increasing volume of impact studies, as presented in Fig. 2.
Fig. 2

Public research impact articles published per year

Our review of public research and innovation impact studies shows that interest in this topic started back in the 1970s, when the US tertiary institutions were recommended to assess the direct and indirect consequences of their investment and expenditures on their regional economy by the American Council on Education (Goldstein 1989).

In the 1980s, the national innovation system (NIS) concept gained attention with the works of Freeman (1987) in Japan and Nelson (1993) in the USA in the area of public-funded research. Their works highlighted the significant roles that universities, PRIs and government agencies play in pursuing technological progress and contributing to economic growth. The public policy makers’ rising expectations of academic innovation impact on the industry and society may be seen in the proliferation of studies on this topic as politicians, economists and technology management scholars questioned about the diverse aspects of public research impact ranging from the rationale and feasibility of measuring such impact to the conceptual framework and methodology of monitoring and managing it.

Brownrigg’s (1973) case study of UK University of Stirling found that its impact on the local economy in terms of income and employment might be marginal due to the significance of other factors such as local industrial structure and its ability to promote trade and investment flow. However, Brownrigg highlighted the limitations of traditional input-out modelling used by the economists in such impact studies of local economy. A similar case study approach coupled with university panel data was adopted by Keane and Allison (1999) in their study of Australian University of Sunshine Coast, Queensland, to establish the strength of university-industry connection and the degree of its “embeddedness” in the region.

In the USA, Bozeman and his colleagues developed the research value mapping (RVM) approach to assess the socio-technical impact of applied R&D projects with the Department of Energy (Bozeman and Kingsley 1997; Bozeman and Klein 1999). Using a combination of qualitative (case studies) and quantitative techniques (data developed from case studies), Bozeman and his team spent a considerable amount of effort to track the knowledge flow of R&D projects and model the possible project outcomes such as technologies patented or licensed, new companies created, new or improved products or processes developed or enhanced human capital. The RVM was found to work well for summative evaluation; however, its adoption beyond the USA has been limited due to the extensive resources required to trace and perform value mapping. According to Hicks et al. (2002) report submission to the Japanese National Institute of Science and Technology Policy, the RVM’s identification of projects with significant milestone achievement to be units of analysis could provide administrative convenience from the funding perspective but did not reflect how the researchers view their work and capture other non-economic payoffs such as the development of human capital (Georghiou and Roessner 2000).

Jaffe (1989, p. 957), on the other hand, used US state-level time-series data to present the significance of “geographically mediated spill-overs from university research to commercial innovation” in the form of corporate patents, particularly in the drugs, chemicals and electronics industrial sectors. These findings were corroborated by Fischer and Varga’s (2003) work on 72 Austrian districts using knowledge production function framework, as well as Beugelsdijk and Cornet’s (2002) study of 1510 Dutch firms, who found that their proximity to technical universities had significant spill-over effects on the firms in the form of new products. Mansfield’s (1991) study of 76 major US firms attributed 11 % of their new products and 9 % of new processes to academic research.

Harhoff’s (1999) econometric analysis of German engineering industry sectors yielded similar evidence of regional spill-over effects from academic institutions on the sectors in the form of new business formation. However, Lööf and Broström’s (2008) study in Sweden found significant academic impact only on large manufacturing firms in the form of innovative sales, while the average service firms did not seem to benefit from university collaboration.

More recent studies are also found in Spain by Barajas et al. (2012) who used the CDM (Crépon et al. 1998) model to measure academic spill-over effects in the form of sales per employee; and Italy by Colombo et al. (2010) who applied the resource-based theory (Wernerfelt 1984) to assess innovation impact in terms of spin-offs.

In France, Bach and Llerena (2007), in their study with BETA, urged to look beyond traditional technology innovation indicators, such as R&D expenditures and licensing revenue, to capture the interaction dynamics among the institutes. It is noteworthy that BETA of France and NIST of the USA had introduced their respective methods in the 1990s to assess the impacts of large-scale R&D projects such as the European Space Programme (Bach and Lambert 1992) and Measurement and Standards Laboratory Programme (Tassey 1999). These methods were subsequently adopted by other countries such as Japan in their study of techno-economic impact of large-scale R&D projects (Kondo 2012). While these methods have been useful for estimating the impact of R&D projects, their limitation on the timing and size of the impact studies has been highlighted as the projects selected for studies must have either completed or achieved some significant benefits. As the time elapsed between project start-up and project benefit realisation might range two to seven  years, based on past studies (Tassey 1999), BETA and NIST used these methods only for large-scale projects, while employing other forms of intermediate indicators, such as new companies started and product or process innovation for periodic and more pragmatic assessment of their innovation impact.

To build a comprehensive multi-dimensional framework, we applied the R&D-based endogenous growth theory (Romer 1990) and Triple Helix (Etzkowitz et al. 2005) to model the flow of knowledge among the major stakeholders of the industrialised economies: universities and PRIs, firms and industry, as well as the community and society. The endogenous growth theory (Romer 1990) postulates that knowledge and technology, unlike physical objects, are characterised by increasing returns, thereby driving economic growth. The theory forms the basis of knowledge-based economy and emphasises the critical role that the economic processes of knowledge creation and diffusion play in shaping growth of firms, industries, communities and nations. The Triple Helix, on the other hand, underscores the central role of universities and its relations with the industry and government in advancing the knowledge-based society (Etzkowitz et al. 2005).

To synthesise the data gathered into the framework, we categorised the dimensions of impact assessment which emerged from our literature review by the different types of knowledge generated and associated output indicators from the various stakeholders: (1) inventive output of universities/PRIs, (2) commercialisation output of universities/PRIs by their TTOs, (3) innovation of industry/firms and (4) community/economy. Figure 3 shows the distribution of impact study papers by categories 1 to 4.
Fig. 3

Distribution of papers by output category

While the majority of papers (84 %) fall into only one of the four categories, some articles belong to more than one category. For example, some studies (e.g. Derrick and Bryant 2013) used both the inventive output of the universities (publications—category 1) and commercialisation output of universities/PRIs (licenses generated—category 2) to measure the impact of research incentives on Australian medical research organisations, thereby falling into in category (1/2) as illustrated in Fig. 3.

Figure 4 presents the conceptual framework depicting the processes and associated output captured by studies among these four stakeholder output categories.
Fig. 4

Impact Assessment Framework

The categories are represented as blocks and the knowledge flows among the blocks depicted by bilateral arrows. These bilateral flows among the blocks are better representation of knowledge flow directions than the traditional linear knowledge flows from universities/PRIs through the firms/industry to the economy as illustrated by most studies in the current literature (Gulbrandsen and Rasmussen 2012). In practice, there is much knowledge exchange between universities/PRIs and the industry, as the inventive output from the former could be inspired by the problems encountered by the latter through research collaboration. In a similar vein, the commercialisation output of the former could be shaped and refined by the requirements of the latter through the open innovation approach adopted by firms (Chesbrough 2007). In addition, universities/PRIs could have direct impact on the economy through their direct expenditures and employment. This direct interaction between universities/PRIs and economic development is represented in our conceptual framework by enveloping universities/PRIs/TTOs (block 1 and block 2) and industry/firms (block 3) with the larger community/economy (block 4). The knowledge flows are therefore indicative of the four phases of the RIE value chain.

It is important to note that this framework is distinct from the chain-linked model postulated by Kline and Rosenberg (1986), which focuses on the relationships among research, invention, innovation and production within the boundaries of an organisation. The Impact Assessment Framework, in contrast, looks beyond the organisational boundaries to access complementary assets and open innovation. The categorisation of block 1 and block 2 is supported by Gulbrandsen and Rasmussen (2012), who made similar distinction between inventive output of universities/PRIs and their commercialisation output, articulating that the former measures the potential rather than the actual commercial value of public science. Block 3 is consistent with the findings of Crépon et al. (1998), who distinguished between innovation input (R&D) and innovation outcome (product and process innovation). Block 4 is corroborated by the work of Goldstein and Renault (2004, p. 735), who advocated that universities could have potential impacts including “direct and indirect spending impacts”.

Case of Singapore PRIs

In the case of Singapore PRIs, all the indicators highlighted in blocks 1 and 2 were used to measure their inventive and commercialisation output for tracking progress and benchmarking with local and international counterparts, as indicated in Fig. 5.
Fig. 5

Impact Assessment Framework used by Singapore PRIs

To assess the commercialisation impact of PRIs’ research at firm, industry and economy levels, the PRIs actively tracked since 2006 the RICV of the intellectual property (IPs) generated from the licensees’ commercialisation of the IPs and aggregated them at the various levels. RICV was captured for two purposes: (a) obtain the actual commercial value generated by the IP and (b) validate their pre-deal assumptions of projected cash flows as feedback mechanism.

According to the descriptive statistics in Table 1, more than 50 % of the licensing agreement transactions were with small and medium enterprises (SMEs), 40 % with multi-national corporations (MNCs) and large enterprises and the remaining with start-ups. About one third of the licensing agreement transactions came from the biomedical sciences sector, one third from info-communication sector and the rest from engineering, manufacturing, electronics, materials, chemicals and other sectors.
Table 1

Descriptive statistics of organisations participating in licensing transactions

 

Mean

Standard deviation

Size

  

 Start-ups

0.12

0.32

 Small and medium enterprises (SME)

0.52

0.50

 Multi-national corporations (MNC) and large enterprises

0.40

0.49

Sector

  

 Biomedical sciences

0.37

0.48

 Info-comm

0.30

0.46

 Engineering and manufacturing

0.21

0.41

 Electronics, materials, chemical and other

0.12

0.33

Our regression results for H1 are presented in models 1 and 2 of Table 2. Model 1 is a baseline model that examines the variances contributed by the control variables. Model 2 describes how IP licensing fees affects a firm’s propensity to repeat licensing pertaining to H1. Model 2 does not appear to be an improvement over model 1, as evident from the similar pseudo R square values, Cox and Snell R square (0.08) and Nagelkerke R square (0.13), as well as the percentage of correct classification (78 %). However, with a coefficient (b = −0.23, p < 0.1), IP licensing fees is found to be a significant but negative predictor of the firm’s propensity to repeat licensing, indicating support for H1 that firms that incur higher IP licensing costs have lower propensity to repeat their IP licensing transactions, thereby reducing academia innovation impact on firms.
Table 2

Regression results

 

Binary logistic regression (H1, H2)

   

Variables

DV: firm’s propensity to repeat licensing (N = 835)

 

Model 1

Model 2

Model 3

Model 4

Constant

−0.44(0.37)

0.47(0.61)

0.29(0.22)

0.49(0.65)

Start-up

0.13(0.31)

0.20(0.32)

−0.66***(0.35)

−0.66***(0.35)

SME

1.79**(0.30)

1.74**(0.30)

0.72*(0.32)

0.67*(0.33)

MNC and large enterprises

1.85**(0.30)

1.82**(0.30)

0.91**(0.32)

0.88*(0.32)

Biomedical sciences sector

0.61***(0.31)

0.42(0.32)

−0.01(0.35)

0.01(0.35)

Info-comm sector

0.49(0.31)

0.47(0.31)

0.24(0.33)

0.24(0.33)

Engineering and manufacturing sectors

−0.65*(0.30)

−0.64*(0.3)

−0.79*(0.33)

−0.82*(0.33)

Log_IP licensing costs

 

−0.23***(0.12)

−1.79**(0.23)

−1.85**(0.24)

Log_RICV

  

1.67**(0.21)

1.70**(0.21)

Log_RICV × Log_IP licensing costs

   

−0.10(0.11)

Cox and Snell R square

0.08

0.08

0.17

0.17

Nagelkerke R square

0.13

0.13

0.27

0.27

Model chi-square

69.76

73.42

157.73

158.57

Significance

0

0

0

0

Classification correct

78 %

78 %

82 %

83 %

*p < 0.05; **p < 0.01; ***p < 0.10; robust standard errors are in parentheses

The regression results for H2 are presented in model 3 of Table 2. Model 3 is an improvement over the baseline model 1, as seen from the increase in the Cox and Snell R square (from 0.08 to 0.17), Nagelkerke R square (0.13 to 0.27) and the percentage of correct classification (78 to 82 %).

With a positive coefficient (b = 1.67, p < 0.05), RICV is established to be a significant and positive predictor of the firm’s propensity to repeat licensing, thereby validating H2 that firms that attain higher levels of RICV with universities/PRIs have higher propensity to repeat their license transaction.

To explore possible interaction effects between IP licensing fees and RICV, model 4 is a full model demonstrating the main and interaction effects of all variables, including the interaction term between IP licensing fees and RICV. The interaction term is found to be not significant. In models 1 to 4, some control variables such as firm size and engineering and manufacturing sector seem to be significant. After controlling for them, there is no change in the significant contributions of the independent variables to the dependent variable, confirming support for H1 and H2. Combining the results of H1 and H2, we can establish that RICV is a better indicator than IP licensing fees in predicting a firm’s propensity to repeat licensing and indicating academic innovation impact on firms. With more than 100 cases per predictor variable, these models satisfy Field’s (2005) guidelines of minimum 10 cases per predictor for reliable regression results. With a variance inflation factor (VIF) value of 1.234 that is lower than the cut-off value of 10 recommended by Chatterjee and Price (1991), the probability of multi-collinearity issues affecting the findings is relatively low.

Combining the results of H1 and H2, we establish that while IP licensing fees and RICV have opposite effects on firm’s propensity to repeat licensing with PRIs and academic innovation impact on them. The findings are consistent with Atuahene-Gima’s (1993) findings that firms’ ITL propensity was determined by their perceived value, namely, their benefits relative to their costs.

Although this study involves PRIs, its results may be seamlessly extended to universities for several reasons. First, since the implementation of the US Bayh–Dole Act and its equivalents in other knowledge-based economies, PRIs and universities in most countries have been charged with similar missions to create socio-economic impact through promoting knowledge generation and transfer to the industry (Link et al. 2011). Second, although PRIs are expected to work on more applied research while universities on basic research, both share similar strategic roles in setting future R&D directions, especially in areas critical to their nations, e.g., energy or climate. As a result of their longer planning horizon, their technologies are at an earlier stage of development compared to the industrial R&D centres, making their time to commercialisation longer and more risky than that of their industrial counterparts (Link and Scott 2013). As both universities and PRIs share similar goals, roles, pressures and challenges in their technology transfer processes, many studies in the technology transfer literature have treated them as a homogeneous group and extend their findings across them (Park et al. 2010).

Discussion

In the field of public research impact, the definition of the term “impact” differed across various stakeholders. To academia, the term is usually associated with intellectual contribution to one’s discipline within academia while socio-economic impact is regarded as beyond academia. From the government perspective, the definition is much broader as that adopted by the Australian RQF and the UK Research Excellence Framework (REF). According to the UK Research Excellence Framework (REF2014 2011), impact is an effect on change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia.

How broad should the definition be? Its breadth really depends on what and how it is to be assessed. To countries that focus on science, technology and innovation (STI) to fuel economic growth and generate wealth (e.g. Singapore), impact is typically defined and assessed with economic measures and STI indicators (Donovan 2011)

We made several contributions in this paper. First, we consolidated a large body of knowledge on impact studies with a theoretically grounded, multi-level framework for assessing public research and innovation impact, connecting the four stakeholders of the knowledge production process—PRIs/universities, TTOs, firms/industry, and community/economy.

Second, we identified literature gaps that create opportunities for future research. In this section, we will discuss several gaps, including the relationships among the various categories of stakeholder outputs, depicted as Impact Assessment Framework blocks 1 to 4 of Fig. 4.

Third, we established empirically the relative significance of RICV as a better indicator than IP licensing fees in predicting firms’ propensity to repeat licensing from PRIs, thereby indicating some form of academic innovation impact on firms. We will elaborate in this section our findings of the RICV concept in the context of the integrated framework.

Inventive output of universities/PRI

In the first block of the framework in Fig. 4, the studies generally adopted the approach of using the inventive output of universities/PRIs as proxy indicators to estimate their social and economic impact. The notion of knowledge as key driver for the competitive advantage of firms and nations has its roots in the literature of economics building on Romer’s endogenous growth theory (Romer 1990; Nonaka and Takeuchi 1995; Grant 1996). The capacity and capability to develop novel ideas and translate them into new products and processes for economic growth form the knowledge base of an economy (Huggins and Johnston 2009). These innovation development capabilities are increasingly being associated with universities and PRIs. As knowledge becomes publicly accessible either in the form of spill-overs or diffusion, it is widely regarded as a desirable outcome for the greater good of the society, spurring greater innovation and economic growth (Fischer and Varga 2003). Table 3 provides a summary of key studies using inventive output indicators.
Table 3

Summary of studies using university/PRI inventive output indicators

Inventive output indicators

Authors

Disclosures

Chapple et al. 2005

Disclosures, patents

Carlsson and Fridh 2002

Gulbrandsen and Rasmussen 2012

Park et al. 2010

Knowledge infrastructure

Keane and Allison 1999

Non-resident tuition rate

Baryla and Dotterweich 2001

Patent citations

Beaudry and Kananian. 2013

Jaffe et al. 1993

Meyer 2006

Owen-Smith and Powell 2003

Spencer 2001

Wang et al. 2011

Patents

Agrawal and Henderson 2002

Bach and Llerena 2007

Baldini 2010

Bekkers and Freitas 2008

Bozeman and Kingsley 1997

Bozeman and Klein 1999

Fischer and Varga 2003

Georghiou and Roessner 2000

Huggins and Johnston 2009

Lissoni et al. 2013.

PRI-firm collaboration

Sáez et al. 2002

Publication citations

Abramo et al. 2009b

Tornquist and Hoenack 1996

Publication co-authorship

Abramo et al. 2011

Abramo et al. 2009a

Publications

Arundel and Geuna 2004

Cohen et al. 2002

Publications, budget

Rogers et al. 1999

Publications, patents

Dietz and Bozeman 2005

Nelson 2009

Publications, patents and citations

McMillan et al. 2000

Narin, Hamilton and Olivastro 1995

Narin et al. 1997

R&D intensity

Nelson 1986

R&D stocks, patents, citations

Hall et al. 2005

University-industry research collaboration

Villasana 2011

As centres of knowledge production, universities/PRIs consistently engage in inventive process, where researchers develop ideas that could be novel and inventive. These inventions are then submitted as required by the Bayh–Dole Act or equivalent in the form of invention disclosures for evaluation by the TTOs for the appropriate intellectual property (IP) strategy to pursue. For invention disclosures that satisfy both internal and external patenting guidelines, the TTOs would proceed to file patent applications, while the researchers would submit their findings for publications. Some high-technology industries may find these publications or patents relevant and cite them or even participate in research collaboration with the universities/PRIs in the course of their R&D work.

Given the significant government R&D expenditures in universities and PRIs to develop the knowledge base of the economy, the studies of this category frequently built on the total factor productivity or knowledge production function frameworks to examine the various outputs in the inventive process as indicators to determine the economic benefits directly or indirectly generated (Jaffe 1989). Building on Cobb–Douglas production function, Pakes and Griliches (1984) pioneered the development and use of the full model of knowledge production function and the final output production function. In their study, they used R&D expenditures as knowledge creation investment and patent counts as economically valuable knowledge produced. Besides economic impact, many researchers have also adopted bibliometric approaches to assess the degree of international scientific influence using citation, mapping and network analysis of papers and patents (Tornquist and Hoenack 1996; Spencer 2001). Others have studied co-authored publications in their bibliometric examination of public-private research collaboration (Abramo et al. 2009a). While citation indicators may reflect productivity and impact, co-authorship indicators may illustrate informal network access, knowledge diffusion and skill transfer.

Cohen et al. (2002) found that publications are important knowledge channels from the public sector to the manufacturing sector. This is supported by Agrawal and Henderson’s (2002) interview of MIT faculty, who established that publications were two-and-one-half times more significant than patents as knowledge channels. Jaffe (1989) found that the proximity between universities/PRIs and firms could have mediating effects on spill-overs from public-funded research to corporate patents. Subsequent studies suggested similar association between patent citation patterns (as indicator of “economic benefit”) and geographical proximity in Austria (Fischer and Varga 2003) and in the USA (Jaffe et al. 1993).

Building on the RVM project, Dietz and Bozeman (2005) discovered significant differences between publication and patent productivity in their socio-technical human capital project. Career spent in industrial settings was found to be negatively related to publication productivity but positively related to patent productivity. In a similar vein, the source of industry support was identified to be unrelated to publication productivity but positively related to patent productivity. These findings suggest that governments that aim to improve the commercial relevance of their public-funded research should encourage “revolving door” policy between academia and industry in their development of science and technical human capital to enhance their university/PRI inventive output.

Pakes and Griliches (1984) established a strong association between R&D and patent counts at the firm and industry levels to demonstrate that patents are appropriate measures of inventive output. Although patent and patent citations have frequently been used as economic indicators, there is concern that patents do not provide the complete picture of economic contribution by universities/PRIs (Fischer and Varga 2003). First, patents do not capture all forms of knowledge, such as codified knowledge (trade secrets and copyrights) and tacit knowledge (know-how and experience), which is important for spill-over. Second, patent count is dependent on the intensity of patenting activity, which is in turn determined by the patenting policy of the institutions. Therefore, patent counts and citations are not necessarily reflective of the commercial value of underlying IP.

According to a study by Intellectual Ventures in 2008, although 60 % of the patents were developed by academia, PRIs and individual inventors, they contributed to less than 1 % of the licensing revenue. On the other hand, large enterprises which were the originators of 40 % of the patents received more than 99 % of the licensing revenue (Hagiu et al. 2011). The stark contrast in licensing revenue between these groups that is disproportionate to that in patent origination signals critical issues with patents as economic indicators.

This demonstrates the urgency for more commercialisation-related indicators that policy makers can implement if they are committed to encouraging more firms to license and commercialise public science (Cheah and Zalan 2013).

Commercialisation output of universities/PRIs by TTOs

The second block of the Impact Assessment Framework in Fig. 4 dwells on the output of commercialisation process that TTOs use in taking IPs from their universities/PRIs to the market. Since the 1990s, there has been an increasing awareness that technological innovation may not be confined within the boundaries of an organisation. It has been widely acknowledged that technology is an essential resource that can be converted into a capability and competitive advantage for firms and nations. Firms have been giving more attention to the sourcing of external knowledge to complement and enhance their existing R&D stock and the acquisition of external technology as part of their R&D process. A summary of key studies using university/PRI commercialisation output indicators is listed in Table 4.
Table 4

Summary of studies using university/PRI commercialisation output indicators

Commercialisation output indicators

Authors

Licensees that go IPO, royalties

Powers and McDougall 2005a

Licenses

Bach and Llerena 2007

Bekkers and Freitas 2008

Carlsson and Fridh 2002

Heisey and Adelman 2011

Hewitt-Dundas 2012

Huggins and Johnston 2009

Lach and Schankerman 2008

Nelson 2009

Sine et al. 2003

Licenses, spin-offs

Bozeman and Kingsley 1997

Bozeman and Klein 1999

Georghiou, and Roessner 2000

Gulbrandsen and Rasmussen 2012

Maia and Claro 2013

McAdam et al. 2009

Pries and Guild 2011

Rogers et al. 1999

Rogers et al. 2000

New biotech enterprises

Zucker et al. 1998

No of spin-offs that go IPO

Powers and McDougall 2005b

Spin-off rate

Bania et al. 1993

Colombo et al. 2010

Di Gregorio and Shane 2003

Spin-offs

Bach and Lambert 1992

Harhoff 1999

Jacobsson and Vico 2010

Lehrer et al. 2009

Lockett and Wright 2005

Marion et al. 2012

Miner et al. 2012

O’Shea et al. 2005

O'Shea et al. 2007

Probert et al. 2013

Rasmussen and Borch 2010

Tassey 1999

Technology transfer

Coccia 2008.

According to Hemmert (2004), external technology acquisition takes place by (a) licensing technology, (b) research collaboration with other organisations and (c) purchasing technology from other organisations. TTOs typically do not sell the technologies originating from universities/PRIs, but prefer to assign non-exclusive licenses to firms if the inventions are broad in scope and wide in applications across industries, so as to encourage product development in more fields for maximum economic impact. Notwithstanding this, TTOs may consider exclusive licensing terms for inventions that require substantial investments for commercialisation. Nelson’s (2009, p. 994) comparison of IP licenses, publications and patents found IP licenses to be the most reliable indicator of “downstream knowledge utilisation” as the “competing economic interests” between licensors and licensees ensure that the licensing agreement is a fair representation of industry demand. Besides licensing IPs to existing firms, TTOs may facilitate the formation of spin-offs by providing technology and business incubation services to their researchers.

As both developed and developing nations put in place the appropriate science, technology and education policies to facilitate technology commercialisation and academic entrepreneurship from universities/PRIs, there has been much scrutiny on their commercialisation output, namely, the number of IP licenses signed, the IP licensing revenues collected and the number spin-offs generated.

To date, more than 14 periodic surveys had been launched to capture such output indicators across different regions such as the USA (AUTM 1991–2012), Canada (AUTM 2000–2012), Europe (ProTon 1991–2012) and Singapore (A*STAR 2002–2011). The data collected from these surveys form the dataset for many studies in this literature strand. For example, AUTM data were used by Di Gregorio and Shane (2003) to show that the rate of university spin-offs depends on inventors’ quality and equity stake in the spin-offs. Calls have been made to both AUTM and ASTP to standardise their variable definitions and methodological approaches to facilitate comparability of PRI indicators across the studies. However, there is a dearth of impact studies in the Asia.

One frequently used theoretical framework in these studies is the resource-based theory (Wernerfelt 1984), which postulates that a firm’s performance is determined by its resources. Barney (1991, p. 117) defined resources as assets that are specific to the firm, “valuable, rare, imperfectly imitable, and non-substitutable”. Building on the resource-based theory, O’Shea et al. (2005), Colombo et al. (2010) and Powers and McDougall (2005b) established that the quality of researcher and research has significant effect on spin-off formation and performance, which were commercialisation output indicators used to estimate public research economic impact.

While licensing number and revenue are useful and relevant indicators to measure commercialisation process efficiency, several issues have been raised about their adequacy in reflecting the economic benefits of public-funded research. First, as license data collected by TTOs only reflect commercialisation activities during the licensing period, which is capped between patent grant date and expiration date, they do not capture any further innovative activity that may continue to impact the economy beyond the licensing period. Second, it is widely recognised that the pathway of commercialisation from invention (that the firm has licensed from universities/PRIs) to innovation (that the firm can generate sales in the form of products and services) requires significant investment as it involves early-stage technology development. This pathway is often alluded to as the valley of death due to the high failure rate, and the role of the public sector in bridging this valley has been much debated upon in the UK and USA (Link and Scott 2013).

The current issues with commercialisation output indicators call for other indicators that would better reflect the downstream innovation process taking place in the firms to provide a more complete picture of public science impact on the industry.

Innovation of industry or firms

The third block of the Impact Assessment Framework focuses on the innovation of industry or firms. Table 5 summarises the studies using firm’s innovation input and outcome indicators.
Table 5

Summary of studies using industry/firm innovation input/outcome indicators

Innovation indicators

Authors

Innovation input

 Corporate patent linked to university papers

Agrawal and Cockburn 2003

 Corporate patents

Jaffe 1989

 Corporate patents, R&D expenditure

Autant-Bernard 2001

 Corporate R&D

Boardman and Gray 2010

 Corporate R&D and innovation

De Fuentes and Dutrenit 2012

 Corporate R&D stocks linked to universities

Adams 2002

 Corporate patenting rate

Hu 2008

 Induced investment

Giesecke 2000

Kramer et al. 1997

Pressman et al. 1995

 Industry-owned patents

Lawson 2013a

Lawson 2013b

 Innovative expenditure

Beugelsdijk and Cornet 2002

 Investment in building and equipment

Florax and Folmer 1992

 R&D intensity

Hauknes and Knell 2009

 Research collaboration

Martin 1998

Innovation outcome indicators

 Innovation sales

Beugelsdijk and Cornet 2002

Lööf and Broström 2008

 Innovative count

Anselin et al. 1997

 Innovative output of firms

Acs et al. 1994

 Innovative products

Beise and Stahl 1999

Varga 2000

 New product innovation sales

Eom and Lee 2010

 New products

Mansfield 1991, 1995, 1998

 New products and processes

Rosenberg and Nelson 1994

 No. of drugs entering each clinical R&D stage (from NIH grant expenditure)

Blume-Kohout 2012

 Product and process innovation

Bozeman and Kingsley 1997

Bozeman and Klein 1999

Coccia 2008

Conceição et al. 1998

Georghiou, and Roessner 2000

Kingsley et al. 1996

Klevorick et al. 1995

 Sales and profitability

Bozeman 2000

 Sales growth rate

Evangelista and Vezzani 2010

 Sales of new products, improved products and products with no or incremental changes

Faems et al. 2005

 Sales of products and services of spin-offs

Vincett 2010

 Sales per employee

Barajas et al. 2012

Lööf and Heshmati 2002

Some of the studies referred to the knowledge production function (Jaffe 1989). Acs et al. (1994) built on knowledge production function to ascertain that small firms could benefit (with increase in innovation count) from the knowledge of universities and the R&D centres of large firms. Varga (2000), on the other hand, applied the Griliches-Jaffe knowledge production function framework to establish that firms located in close proximity to the research university can benefit economically with an increase in their number of innovative products.

In 1998, Crépon et al. (1998) integrated both Cobb–Douglas production function and knowledge production function to develop the CDM model, which depicts two key relationships pursuant to a firm’s decision to invest in innovation. The first relationship describes the link between innovation input (primarily R&D) and innovation output (product or process innovation) as accepted in the knowledge production function (Pakes and Griliches 1984). The second relationship established the connection between innovation output and productivity based on Cobb–Douglas production function. Since then, many studies have built on the CDM model using labour productivity (sales per employee) in Spain (Barajas et al. 2012) and in the Netherlands (Lööf and Heshmati 2002) and turnover growth in Italy (Evangelista and Vezzani 2010).

Based on the CDM model, we have classified the following indicators as input to the innovation process of firms: (a) corporate R&D stocks, (b) innovation expenditures, (c) corporate patents, (d) investment in building and equipment for R&D and (e) induced investments. In particular, Pressman et al. (1995, p. 30) surveyed a sample of MIT licensees by collecting information on their induced investment defined as “Money spent developing new products and efficient ways to produce and market these products. It excludes the costs of producing (or investment required to produce) mature products”. From the survey, they extrapolated their data to estimate an economic impact of US$922 million.

Using a similar approach, Kramer et al. (1997) estimated that University of Pennsylvania’s licensees created an economic impact of US$4.6 billion in the form of induced investments. Although Pressman et al. (1995) and Kramer et al. (1997) have used induced investments to measure economic impact, these values are actually input to the firm’s innovation process that should be distinguished from its innovation outcome. According to Eurostat (2012), innovation outcome may be in the form of product innovation (e.g. sales from innovative products) or process innovation (e.g. productivity gain from innovative process) (Goldstein and Drucker 2006). The distinction between innovation input and innovation outcome indicators is clearly depicted in block 3 of the framework in Fig. 4.

While many studies continue to use innovative product sales or proportion of new products as indicators of firms’ innovative outcome, there are recommendations to refine innovative outcome modes and indicators. In a recent study, Evangelista and Vezzani (2010) expanded Eurostat’s (2012) definition of innovation outcome from two to four modes to examine economic impact: (a) pure product innovation, (b) pure process innovation, (c) organisation innovation and (d) mix of product, process and organisational innovations. Penfield et al. (2014) further articulated the need to distinguish among output, outcome and impact.

Community/economy

The fourth block of the framework comprises studies that use economic indicators such as employment, income, output and GDP growth, as summarised in Table 6.
Table 6

Summary of studies related to economic impact

Economic impact indicators

Authors

Country competitiveness

van Hemert and Nijkamp 2010

Earnings of non-farm worker

Goldstein and Drucker 2006

Goldstein and Renault 2004

Economic growth

Bozeman 2000

Employment

Conceição et al. 1998

Kramer et al. 1997

Pressman et al. 1995

Riddel and Schwer. 2003

Swann and Prevezer 1996

Employment and income

Armstrong 1993

Brownrigg 1973

Lewis 1988

Employment and output

Glasson 2003

Employment growth

Henderson and Weiler 2010

Lendel 2010

Employment, income and output

Felsenstein 1996

Harris 1997

Pastor et al. 2013,

Roessner et al. 2013

Employment, output and income

Roessner et al. 2010

Expenditures from staff and students

Steinacker 2005

GDP contributed by university R&D

Martin 1998

GDP growth and income growth

Baldwin and McCracken 2013

Human capital

Huffman and Quigley 2002

Nagle 2007

Perkmann et al. 2011

Income and output

Bleaney et al. 1992

Jobs created

Gulbrandsen and Rasmussen 2012

Miner et al. 2012

Wallmark 1997

Physical capital or labour

Tassey 2005

Private and social rates of return on projects

Weiler 2000

Private and social rates of return to public R&D

Jamison and Jansen 2000

Salter and Martin 2001

Property rent

Sivitanidou and Sivitanides 1995

Social rate of return on academic research

Mansfield 1991, 1995, 1998

Value-add

Huggins and Johnston 2009

The impact studies in this category generally applied the input-output modelling or Keynesian multiplier approach. In UK, Harris’ (1997) study of the University of Portsmouth with the input-output approach estimated that the direct expenditure by the university sector resulted in an output multiplier effect of 1.24 to 1.73. Glasson’s (2003) input-output model found an output multiplier of between 0.70 and 1.12 for Sunderland University in Northeast England.

Using Keynesian multiplier approach, Brownrigg (1973) arrived at income multiplier between 1.45 and 1.80 based on a case study of the University of Stirling, while Huggins and Cooke (1997) found the corresponding multiplier at 1.46 to 1.52 at the Cardiff University. In the USA, Felsenstein (1996) applied econometric model on the input-output model to quantify the contribution of the Northwestern University to the Chicago metropolitan area at an output multiplier of 3.1 and employment multiplier of 1.55 in 1993. In a more recent study, Roessner et al. (2013) estimated the impact of university licensing on the whole US economy with a range of hypothetical licensing royalty rates at 2, 5 and 10 %, based on AUTM data (1996–2010) and product substitution effect adjustment.

Despite the popularity of the input-output and Keynesian multiplier approaches, these methodologies have raised some concerns. First, these techniques are restricted to estimating the backward linkages arising only from university expenditures and investment. The dynamic impact of university knowledge production and other forms of output on the business community are not captured (Martin 1998). Second, there is a lack of counterfactual hypotheses for comparison of impact with and without the university’s presence, giving rise to wide variation in results (Goldstein and Drucker 2006, p. 24). Third, the input-output model assumes that product sales arising from university/PRI-licensed IPs are all contained within the local economy to contribute to GDP. This assumption becomes invalid when a significant proportion of IPs are commercialised overseas due to the small domestic market size. Fourth, the input-output model treats university/PRI licensing and research income as expenditures that can have direct and indirect effect on the output and employment of other industries within the local economy. However, in a small country with an open economy, it is unlikely that it has the capability and capacity to provide all the equipment, materials and talent required to conduct R&D.

Toward an integrated framework and the role of RICV

We understand that different indicators are useful and relevant for tracking the effects of intermediate output or outcome at the different stages of the long journey of research, innovation and enterprise (RIE) value-added activities involving multiple stakeholders from science produced by universities/PRIs through the innovation outcome generated by the industry/firms to economic development:
  1. 1.

    Inventive output of universities/PRIs, e.g. R&D intensity, invention or technology disclosures, publications and patents;

     
  2. 2.

    Commercialisation output of universities/PRIs, e.g. licenses executed, spin-offs;

     
  3. 3.

    Firm/industry’s innovation input, e.g. corporate R&D stocks and its innovation outcome, e.g. innovative product sales and RICV;

     
  4. 4.

    Economic development in the form of employment, income and RICV at the economy level.

     

Given that RIE is a complex and iterative value chain with multiple feedback loops, it is apparent that no single indicator can furnish a complete picture of public-funded research and innovation impact. We therefore propose an integrated Impact Assessment Framework for universities/PRIs as a starting point to manage their value chain. The framework provides the policy makers and scholars the flexibility to define and refine indicators at different stages of the RIE journey while keeping a holistic view of the entire value chain, to facilitate periodic monitoring, analytics and adaptation at micro-, meso- and macro-levels. In particular, the framework includes a relatively new indicator, RICV, to estimate innovation outcome at firm or industry level, as well as economic impact at a regional or national level.

Implications and conclusion

In this paper, we addressed our research question whether RICV is a suitable predictor of firms’ propensity to repeat their in-licensing agreements with PRIs, compared to the traditional licensing revenue indicator. Our results show that firms that incur higher IP licensing fees with universities/PRIs have lower propensity to repeat their IP licensing transactions, thereby reducing academic innovation impact on firms. On the other hand, firms that attain higher levels of RICV with universities/PRIs are found to have higher propensity to repeat their license transactions, indicating an increase in academic innovation impact on firms. With results confirming our hypotheses, we contribute to the literature of impact studies by proposing new secondary RICV indicators to estimate the economic impact of public-funded technological innovation at the firm, industry and national levels.

Our finding of RICV as a predictor of firms’ propensity to repeat licensing engagement with universities/PRI has contributed to the TCE theory. In most studies of the current literature, the TCE is used to explain the propensity of firms to license technologies from other firms (pecuniary sources as traditional licensors). Given that these public institutions are non-pecuniary sources as they have socio-economic rather than pure financial objectives, the determinants of in-licensing propensity in firm-public institution engagement are likely to be different from that in firm-firm transaction. This study therefore adds a new dimension of RICV to the TCE theory.

In addition, our results show that the NPV theory from corporate finance, which has erstwhile been useful in evaluating investment decisions at the firm level, can be extended to the technology transfer discipline to estimate the impact of public research and innovation at the industry and economy level.

This study has made several managerial implications. As shown in Fig. 2, Singapore PRIs started using RICV since 2010 as part of their balanced scorecard strategy for performance management to measure and monitor the quality of the licenses signed and their impact at the firm level. Totalling the RICVs of all licensees for a particular year allows impact estimation at the economy level for the year. From 2010 to 2012, the PRIs have seen a significant increase in total RICV by more than 80 % per annum, with corresponding increase in total licensing revenues. Second, by grouping the RICV of firms by industry, the PRIs could obtain a better grasp of research and innovation impact at the industry level. Trending analysis of RICV by technology has enabled the identification of promising and complementary technologies with rising RICVs to form innovation clusters to serve growing industry sectors. To date, three innovation clusters have been formed and increased their contribution from less than 5 % of total RICV in 2010 to more than one third in 2012.

The findings of this study indicate that RICV is a better indicator than the traditional licensing revenue metric used by universities/PRIs to measure and monitor innovation impact in terms of firm innovation output. Although this study was carried out with the public research organisations of Singapore, the findings are generalisable to those of other countries that pursue the strategy of innovation-driven economic development. Universities/PRIs of knowledge-based economies face similar challenges in measuring the intermediate input and output of their respective RIE value chain activities, as evident in their adoption of common traditional indicators for benchmarking despite their limitations (AUTM 2014). To address the challenges and limitations, the universities/PRIs of the other innovation-driven economies are invited to use the RICV approach to inform strategic decision-making in shaping research agenda, allocating resources, influencing inventive outputs and determining the effectiveness of commercialisation outputs. The implementation of the RICV indicator in various universities/PRIs will build confidence in the indicator.

The policy makers of innovating economies may use the RICV and related indicators to benchmark the impact of research and innovation on enterprise and economy. Firms could also adopt and adapt the RICV approach to actively manage their IP and technology portfolio so as to increase the net present value of revenue streams generated by their products and services enabled by their IP and technology portfolio.

As an extension for this study, we propose development of a practical implementation guide for institutions seeking to use the framework and RICV indicator. The guide should comprise the development of tools, processes and training to enable the setup of a scalable and flexible system that can cater to the needs and objectives of the institutions.

Our study has identified a few areas for future research. First, while our paper focuses on the economic impact and STI indicators gleaned from our review of 275 articles, we acknowledge that there is much value in combining narratives with both qualitative and quantitative metrics. Although qualitative approaches such as field interviews and case study interrogations could provide rich and useful data, they could be very resource-intensive. The public institutions may therefore consider embarking on in-depth case studies (e.g. RVM) for completed large-scale R&D projects once every few years, not only to study about the performance of specific projects but also to use them to baseline and compare the relative significance of the current set of applicable indicators (as synthesised in our proposed Impact Assessment Framework), as well as analyse their relationships to evaluate their effectiveness.

Second, we are aware that RIE process from PRIs/universities’ receipt of R&D grant award (to perform R&D) to their TTOs’ successful commercialisation of IP generated (with licensing firms) may take an average of 3 to 8 years, involving multiple organisations and departments. Although it is widely acknowledged that the process is non-linear, there is increasing interest among PRIs and governments (Smith et al. 2011) to trace the efficiency of the various stages of the process to evaluate its overall performance. Future studies may embark on this area to identify the determinants of the conversion efficiency among the intermediate outputs, for example, from invention disclosures to patents, from patents to licenses and from licenses to products.

Footnotes
1

The Triple Helix model emphasises the central role of universities in research and innovation and the significance of university-industry-government relations in advancing knowledge-based society (Etzkowitz et al. 2005).

 
2

Asset specificity refers to the extent to which assets can be transferred to other purposes. The TCE model postulates that the more specific the nature of the assets provided by the seller, the greater is the risk that the buyer faces in the transaction.

 
3

IP appropriability refers to the risk of disclosing IP to expropriation.

 
4

RVM is research value mapping approach developed by Bozeman and Kingsley (1997) in the US to assess the socio-technical impact of large applied R&D projects with the Department of Energy002E

 
5

Bureau d’Economie Théorique et Appliquée (BETA), France

 
6

National Institute of Standards and Technology (NIST), US–BETA and NIST developed methods in the 1990s to assess the impacts of large-scale R&D projects.

 
7

RICV is treated as “imputed” for the following reasons: (a) the sales revenue is a projection or extrapolation that was mutually agreed between licensee and licensor; and more importantly, (b) the imputed commercial value is attributed in whole rather in part regardless of the contribution of the IP(s) licensed to the final product/services. In other words, it is assumed that all IPs play an integral part to the final innovation outcome to simplify computation for all practical intents and purposes. A similar approach was used by Roessner et al. (2013) in computing sales revenues from university IP royalties based on AUTM data in their estimation of economic impact using input-output model.

 
8

Keyword combinations or variations include but not limited to “invention” or “publication” or “patent” or “commercialisation”, “innovation” and “academic” or “university” or “public research” and “impact”.

 
9

For robustness, the keyword search constraints were relaxed using broad queries for a subset of journals (e.g., American Economic Review, Applied Economics, Economic Development Quarterly, Higher Education, Industry and Innovation, Journal of Technology Transfer, R&D Management, Research Policy, Review of Economics and Statistics, Science and Public Policy, Technovation and Urban Studies). We submit that any additional papers identified did not have a material influence on the model we inferred from our literature synthesis.

 
10

Although newly published papers may not have time to accumulate citations, they could contribute to the literature by indicating research directions. We therefore did not adopt the citation-based method in our literature search to avoid discriminating against recent papers.

 

Declarations

Acknowledgements

We thank the Chief Executive of A*STAR ETPL, Mr Philip Lim, for his guidance and support of this paper. However, all errors of fact or interpretation in this paper remain ours.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
National University of Singapore Business School
(2)
Agency for Science, Technology and Research (A*STAR) ETPL

References

  1. A*STAR (2002-2011) National survey of R&D. Agency for Science, Technology and Research. Google Scholar
  2. Abramo G, D’Angelo CA, Di Costa F (2011) University-industry research collaboration: a model to assess university capability. Higher Educ 62(2):163–181View ArticleGoogle Scholar
  3. Abramo G, D’Angelo CA, Di Costa F, Solazzi M (2009a) University–industry collaboration in Italy: a bibliometric examination. Technovation 29(6):498–507View ArticleGoogle Scholar
  4. Abramo G, D’Angelo CA, Di Costa F (2009b) Research collaboration and productivity: is there correlation? Higher Educ 57(2):155–171View ArticleGoogle Scholar
  5. Acs Z, Audretsch D, Feldman M (1994) R&D spillovers and recipient firm size. Rev Econ Stat 76:336–40View ArticleGoogle Scholar
  6. Adams J (2002) Comparative localization of academic and industrial spillovers. J Econ Geogr 2:253–78View ArticleGoogle Scholar
  7. Agrawal A, Cockburn I (2003) The anchor tenant hypothesis: exploring the role of large, local, R&D-intensive firms in regional innovation systems. Int J Ind Organ 21:1227–53View ArticleGoogle Scholar
  8. Agrawal A, Henderson R (2002) Putting patents in context: exploring knowledge transfer from MIT. Manag Sci 48(1):44–60View ArticleGoogle Scholar
  9. Ambos TC, Mäkelä K, Birkinshaw J, D’Este P (2008) When does university research get commercialized? Creating ambidexterity in research institutions. J Manag Stud 45:1424–47View ArticleGoogle Scholar
  10. Anselin L, Varga A, Acs Z (1997) Local geographic spillovers between university research and high technology innovations. J Urban Econ 42:422–48View ArticleGoogle Scholar
  11. Armstrong H (1993) The local income and employment impact of Lancaster University. Urban Stud 30:1653–1668View ArticleGoogle Scholar
  12. Arundel A, Geuna A (2004) Proximity and the use of public science by innovative European firms. Econ Innov New Technol 13(6):559–580View ArticleGoogle Scholar
  13. Atuahene-Gima K (1993) Determinants of inward technology licensing intentions: an empirical analysis of Australian engineering firms. J Prod Innov Manag 10:230–240View ArticleGoogle Scholar
  14. Autant-Bernard C (2001) Science and knowledge flows: evidence from the French case. Res Policy 30(7):1069–1078View ArticleGoogle Scholar
  15. AUTM (1991-2012) The AUTM Licensing Survey: Fiscal Year 1991–2012. Norwalk: Association of University Technology ManagersGoogle Scholar
  16. AUTM (1996-2010) The AUTM Licensing Survey: Fiscal Year 1996-2010. Norwalk: Association of University Technology ManagersGoogle Scholar
  17. AUTM (2000-2012) The AUTM Licensing Survey: Fiscal Year 2000–2012. Norwalk: Association of University Technology ManagersGoogle Scholar
  18. AUTM (2014) The AUTM Licensing Survey: Fiscal Year 2014. Norwalk: Association of University Technology ManagersGoogle Scholar
  19. Bach L, Llerena P (2007) Indicators of higher-education institutes and public-research organizations technology transfer activities: insights from France. Sci Public Policy 34(10):709–721View ArticleGoogle Scholar
  20. Baldini N (2010) Do royalties really foster university patenting activity? An answer from Italy. Technovation 30(2):109–116View ArticleGoogle Scholar
  21. Baldwin J, McCracken W III (2013) Justifying the Ivory Tower: higher education and state economic growth. J Educ Finance 38(3):181–209Google Scholar
  22. Bania N, Eberts R, Fogarty M (1993) Universities and the startup of new companies: can we generalize from Route 128 and Silicon Valley? Rev Econ Stat 75:761–66View ArticleGoogle Scholar
  23. Barajas A, Huergo E, Moreno L (2012) Measuring the economic impact of research joint ventures supported by the EU Framework Programme. J Technol Transf 37(6):917–942View ArticleGoogle Scholar
  24. Barney J (1991) Firm resources and sustained competitive advantage. J Manag 17:99–120Google Scholar
  25. Baryla Jr, Dotterweich D (2001) Student migration: do significant factors vary by region? Educ Econ 9:269–80View ArticleGoogle Scholar
  26. Beaudry C, Kananian R (2013) Follow the (industry) money—the impact of science networks and industry-to-university contracts on academic patenting in nanotechnology and biotechnology. Ind Innov 20(3):241–260View ArticleGoogle Scholar
  27. Beise M, Stahl H (1999) Public research and industrial innovations in Germany. Res Policy 28:397–422View ArticleGoogle Scholar
  28. Bekkers R, Freitas IMB (2008) Analysing knowledge transfer channels between universities and industry: to what degree do sectors also matter? Res Policy 37(10):1837–1853View ArticleGoogle Scholar
  29. Beugelsdijk S, Cornet M (2002) A far friend is worth more than a good neighbour: proximity and innovation in a small country. J Manag Gov 6:169–88View ArticleGoogle Scholar
  30. Bleaney M, Binks M, Greenaway D, Reed G, Whynes D (1992) What does a university add to its local economy. Appl Econ 24:305–311View ArticleGoogle Scholar
  31. Blume-Kohout ME (2012) Does targeted, disease‐specific public research funding influence pharmaceutical innovation? J Policy Anal Manage 31(3):641–660View ArticleGoogle Scholar
  32. Boardman C, Gray D (2010) The new science and engineering management: cooperative research centers as government policies, industry strategies, and organizations. J Technol Transf 35(5):445–459View ArticleGoogle Scholar
  33. Bozeman B (2000) Technology transfer and public policy: a review of research and theory. Res Policy 29:627–655View ArticleGoogle Scholar
  34. Bozeman B, Kingsley G (1997) R&D value mapping: a new approach to case study-based evaluation. J Technol Transf 22(2):33–41View ArticleGoogle Scholar
  35. Bozeman B, Klein HK (1999) The case study as research heuristic: lessons from the R&D value mapping project. Eval Program Plann 22(1):91–103View ArticleGoogle Scholar
  36. Brownrigg M (1973) The economic impact of a new university. Scott J Pol Econ 20:123–139View ArticleGoogle Scholar
  37. Carlsson B, Fridh AC (2002) Technology transfer in United States universities: a survey and statistical analysis. J Evol Econ 12:199–232View ArticleGoogle Scholar
  38. Chapple W, Lockett A, Siegel D, Wright M (2005) Assessing the relative performance of UK university technology transfer offices: parametric and non-parametric evidence. Res Policy 34(3):369–384View ArticleGoogle Scholar
  39. Chatterjee S, Price B (1991) Regression analysis by example. Wiley, New YorkGoogle Scholar
  40. Cheah S, Zalan T (2013) Why are some firms more successful than others in commercial adoption of technologies licensed from public research institutes in Singapore? The Proceedings of XXIV ISPIM Conference, HelsinkiGoogle Scholar
  41. Chesbrough H (2007) The market for innovation: implications for corporate strategy. Calif Manage Rev 49(3):45–66View ArticleGoogle Scholar
  42. Coccia M (2008) Spatial mobility of knowledge transfer and absorptive capacity: analysis and measurement of the impact within the geoeconomic space. J Technol Transf 33(1):105–122View ArticleGoogle Scholar
  43. Cohen W, Nelson R, Walsh J (2002) Links and impacts: the influence of public research on industrial R&D. Manag Sci 48:1–23View ArticleGoogle Scholar
  44. Colombo M, D’Adda D, Piva E (2010) An empirical analysis. J Technol Transf 35(1):113–140View ArticleGoogle Scholar
  45. Conceição P, Heitor MV, Oliveira P (1998) University-based technology licensing in the knowledge based economy. Technovation 18(10):615–658View ArticleGoogle Scholar
  46. Crépon B, Duguet E, Mairesse J (1998) Research and development, innovation and productivity: an econometric analysis at the firm level. Econ Innov New Technol 7(2):115–58View ArticleGoogle Scholar
  47. De Fuentes C, Dutrenit G (2012) Best channels of academia–industry interaction for long-term benefit. Res Policy 41(9):1666–1682View ArticleGoogle Scholar
  48. Derrick GE, Bryant C (2013) The role of research incentives in medical research organisations. R&D Manag 43(1):75–86View ArticleGoogle Scholar
  49. Di Gregorio, Shane S (2003) Why do some universities generate more start-ups than others? Res Policy 32:209–227View ArticleGoogle Scholar
  50. Dietz JS, Bozeman B (2005) Academic careers, patents, and productivity: industry experience as scientific and technical human capital. Res Policy 34(3):349–367View ArticleGoogle Scholar
  51. Donovan C (2011) State of the art in assessing research impact: introduction to a special issue. Res Eval 20(3):175–179View ArticleGoogle Scholar
  52. Elmquist M, Le Masson P (2009) The value of a ‘failed’ R&D project: an emerging evaluation framework for building innovative capabilities. R&D Manag 3(2):136–152View ArticleGoogle Scholar
  53. Eom BY, Lee K (2010) Determinants of industry–academy linkages, and their impact on firm performance: the case of Korea as a latecomer in knowledge industrialization. Res Policy 39(5):625–639View ArticleGoogle Scholar
  54. Etzkowitz H, Mello J, Almeida M (2005) Towards “meta-innovation” in Brazil: the evolution of the incubator and the emergence of a triple helix. Res Policy 34(4):411–424View ArticleGoogle Scholar
  55. European Commission (2009) Metrics for knowledge transfer from public research organisations in Europe. Report from the European Commission’s Expert Group on Knowledge Transfer Metrics, BrusselsGoogle Scholar
  56. Eurostat (2012) viewed 20 Feb 2012. http://epp.eurostat.ec.europa.eu/
  57. Evangelista R, Vezzani A (2010) The economic impact of technological and organizational innovations. a firm-level analysis. Res Policy 39:1253–1263View ArticleGoogle Scholar
  58. Faems D, Van Looy B, Debackere K (2005) The role of interorganizational collaboration within innovation strategies: towards a portfolio approach. J Prod Innov Manag 22(3):238–250View ArticleGoogle Scholar
  59. Felsenstein D (1996) The university in the metropolitan arena: impacts and public policy implications. Urban Stud 33:1565–80View ArticleGoogle Scholar
  60. Field A (2005) Effect sizes. Discovering Statistics Using SPSS. 2nd edit. Sage Publcations, London Google Scholar
  61. Fischer MM, Varga A (2003) Spatial knowledge spillovers and university research: evidence from Austria. Ann Reg Sci 37:303–22View ArticleGoogle Scholar
  62. Fisher I (1930) The theory of interest. Macmillan, New YorkGoogle Scholar
  63. Florax R, Folmer H (1992) Knowledge impacts of universities on industry: an aggregate simultaneous investment model. J Reg Sci 32:437–66View ArticleGoogle Scholar
  64. Freeman C (1987) Technology policy and economic performance: lessons from Japan. Pinter, LondonGoogle Scholar
  65. Georghiou L, Roessner D (2000) Evaluating technology programs: tools and methods. Res Policy 29:657–678View ArticleGoogle Scholar
  66. Giesecke S (2000) The contrasting roles of government in the development of biotechnology industry in the US and Germany. Res Policy 29(2):205–223View ArticleGoogle Scholar
  67. Glasson J (2003) The widening local and regional development impacts of the modern universities—a tale of two cities (and North-South perspectives). Local Econ 18:21–37View ArticleGoogle Scholar
  68. Goldstein H (1989) Estimating the regional economic impact of universities: an application of input-output analysis. Plann Higher Educ 18:51–64Google Scholar
  69. Goldstein H, Drucker J (2006) The economic development impacts of universities on regions: do size and distance matter? Econ Dev Q 20:22–43View ArticleGoogle Scholar
  70. Goldstein HA, Renault CS (2004) Contributions of universities to regional economic development: a quasi-experimental approach. Reg Stud 38:733–46View ArticleGoogle Scholar
  71. Grant R (1996) Toward a knowledge-based theory of the firm. Strategic Management Journal 17: 109-122Google Scholar
  72. Gulbrandsen M, Rasmussen E (2012) The use and development of indicators for the commercialisation of university research in a national support programme. Technol Anal Strateg Manag 24(5):481–495Google Scholar
  73. Hagiu A, Yoffie D, Wagonfeld AB (2009) Intellectual Ventures. HBS Case: 710-423Google Scholar
  74. Hall BH, Jaffe AB, Trajtenberg M (2005) Market value and patent citations. RAND J Econ 36(1):16–38.Google Scholar
  75. Harhoff D (1999) Firm formation and regional spillovers. Econ Innov New Technol 8:27–55View ArticleGoogle Scholar
  76. Harris R (1997) The impact of the University of Portsmouth on the local economy. Urban Stud 34:605–26View ArticleGoogle Scholar
  77. Hauknes J, Knell M (2009) Embodied knowledge and sectoral linkages: an input–output approach to the interaction of high-and low-tech industries. Res Policy 38(3):459–469View ArticleGoogle Scholar
  78. Heisey PW, Adelman SW (2011) Research expenditures, technology transfer activity, and university licensing revenue. J Technol Transf 36(1):38–60View ArticleGoogle Scholar
  79. Hemmert M (2004) The influence of institutional factors on the technology acquisition performance of high-tech firms: survey results from Germany and Japan. Res Policy 33:1019–1040View ArticleGoogle Scholar
  80. Henderson J, Weiler S (2010) Entrepreneurs and job growth: probing the boundaries of time and space. Econ Dev Q 24(1):23–32View ArticleGoogle Scholar
  81. Hewitt-Dundas N (2012) Research intensity and knowledge transfer activity in UK universities. Res Policy 41(2):262–275View ArticleGoogle Scholar
  82. Hicks D, Kroll P, Narin F, Thomas P, Ruegg R, Tomizawa H, Kobayashi S (2002) Quantitative Methods of Research Evaluation Used by the US Federal Government. Second Theory-Oriented Research Group, National Institute of Science and Technology Policy (NISTEP). Tokyo, Japan Google Scholar
  83. Hu MC (2008) Knowledge flows and innovation capability: the patenting trajectory of Taiwan’s thin film transistor-liquid crystal display industry. Technol Forecasting Soc Change 75(9):1423–1438View ArticleGoogle Scholar
  84. Huffman D, Quigley J (2002) The role of the university in attracting high tech entrepreneurship: a Silicon Valley tale. Ann Reg Sci 36:403–19View ArticleGoogle Scholar
  85. Huggins R, Cooke P (1997) The economic impact of Cardiff University: innovation, learning and job generation. GeoJournal 41(4): 325-337Google Scholar
  86. Huggins R, Johnston A (2009) The economic and innovation contribution of universities. a regional perspective. Environ Plann C Govern Policy 27(6):1088–1106Google Scholar
  87. Jacobsson S, Vico EP (2010) Towards a systemic framework for capturing and explaining the effects of academic RD. Technol Anal Strateg Manag 22(7):765–787View ArticleGoogle Scholar
  88. Jaffe A, Trajtenberg, Henderson R (1993) Geographic localization of knowledge spillovers as evidenced by patent citations. Q J Econ 108:577–98View ArticleGoogle Scholar
  89. Jaffe AB (1989) Real effects of academic research. Am Econ Rev 79:957–70Google Scholar
  90. Jamison D, Jansen C (2000) Technology transfer and economic growth. J Assoc Univ Technol Managers 12:23–45Google Scholar
  91. Keane J, Allison J (1999) The intersection of the learning region and local and regional economic development: analysing the role of higher education. Reg Stud 33:896–902View ArticleGoogle Scholar
  92. Kingsley G, Bozeman B, Coker K (1996) Technology transfer and absorption: an ‘R & D value-mapping’ approach to evaluation. Res Policy 25(6):967–995View ArticleGoogle Scholar
  93. Klaes M (2000) The birth of the concept of transaction costs: issues and controversies. Ind Corp Change 9(4):567–593View ArticleGoogle Scholar
  94. Klevorick A, Levin R, Nelson R, Winter S (1995) On the sources and significance of inter-industry differences in technological opportunities. Res Policy 24:185–205View ArticleGoogle Scholar
  95. Kline SJ, Rosenberg N (1986) An overview of innovation. In: Landau R, Rosenberg N (eds) The positive sum strategy: Harnessing technology for economic growth. National Academy Press, Washington, DC, pp 275–304Google Scholar
  96. Kondo M (2012, June) Techno-economic impacts of large-scale national R&D projects. In Management of Innovation and Technology (ICMIT), 2012 IEEE International Conference: 88-93. IEEEGoogle Scholar
  97. Kramer PB, Scheibe SL, Reavis DY, Berneman LP (1997) Induced investments and jobs produced by exclusive patent licenses: a confirmatory study. J Assoc Univ Technol Managers 9:79–100Google Scholar
  98. Kuhn TS (1962) The structure of scientific revolutions. University of Chicago Press, Chicago, ILGoogle Scholar
  99. Lach S, Schankerman M (2008) Incentives and invention in universities. Rand J Econ 39(2):403–433View ArticleGoogle Scholar
  100. Lawson C (2013a) Academic inventions outside the university: investigating patent ownership in the UK. Ind Innov 20(5): 385-398Google Scholar
  101. Lawson C (2013b) Academic patenting: the importance of industry support. J Technol Transf 38(4): 509-535Google Scholar
  102. Lehrer M, Nell P, Garber L (2009) A national systems view of university entrepreneurial ISM: inferences from comparison of the German and US experience. Res Policy 38(2):268–280View ArticleGoogle Scholar
  103. Lendel I (2010) The impact of research universities on regional economies: the concept of university products. Econ Dev Q 24(3):210–230View ArticleGoogle Scholar
  104. Lewis JA (1988) Assessing the effect of the polytechnic, Wolverhampton, on the local community. Urban Stud 25:53–61View ArticleGoogle Scholar
  105. Link AN, Scott J (2013) Bending the Arc of Innovation: Public Support of R&D in Small, Entrepreneurial Firms. Palgrave MacmillanGoogle Scholar
  106. Link AN, Siegel DS, van Fleet DD (2011) Public science and public innovation: assessing the relationship between patenting at US National Laboratories and the Bayh-Dole Act. Res Policy 40(8):1094–1099View ArticleGoogle Scholar
  107. Lissoni F, Pezzoni M, Poti B, Romagnosi S (2013) University autonomy, the professor privilege and academic patenting: Italy, 1996–2007. Ind Innov 20(5):399–421View ArticleGoogle Scholar
  108. Lockett A, Wright M (2005) Resources, capabilities, risk capital and the creation of university spin-out companies. Res Policy 34(7):1043–1057View ArticleGoogle Scholar
  109. Lööf H, Broström A (2008) Does knowledge diffusion between university and industry increase innovativeness? J Technol Transf 33(1):73–90View ArticleGoogle Scholar
  110. Lööf H, Heshmati A (2002) Knowledge capital and performance heterogeneity: a firm-level innovation study. Int J Prod Econ 76(1):61–85View ArticleGoogle Scholar
  111. Magni C (2009) Investment decisions, net present value and bounded rationality. Quant Finance 9(8):967–979View ArticleGoogle Scholar
  112. Maia C, Claro J (2013) The role of a proof of concept center in a university ecosystem: an exploratory study. J Technol Transf 38(5):641–650View ArticleGoogle Scholar
  113. Mansfield E (1991) Academic research and industrial innovation. Res Policy 20:1–12View ArticleGoogle Scholar
  114. Mansfield E (1995) Academic research underlying industrial innovations: sources, characteristics, and financing. Rev Econ Stat 77(1):55–65View ArticleGoogle Scholar
  115. Mansfield E (1998) Academic research and industrial innovation: an update of empirical findings. Res Policy 26:773–776View ArticleGoogle Scholar
  116. Marion TJ, Dunlap DR, Friar JH (2012) The university entrepreneur: a census and survey of attributes and outcomes. R & D Manag 42(5):401–419View ArticleGoogle Scholar
  117. Martin F (1998) The economic impact of Canadian university R&D. Res Policy 27:677–687View ArticleGoogle Scholar
  118. McAdam R, McAdam M, Brown V (2009) Proof of concept processes in UK university technology transfer: an absorptive capacity perspective. R & D Manag 39(2):192–210View ArticleGoogle Scholar
  119. McMillan GS, Narin F, Deeds DL (2000) An analysis of the critical role of public science in innovation: the case of biotechnology. Res Policy 29(1):1–8View ArticleGoogle Scholar
  120. Meyer M (2006) Are patenting scientists the better scholars? An exploratory comparison of inventor-authors with their non-inventing peers in nano-science and technology. Res Policy 35(10):1646–1662View ArticleGoogle Scholar
  121. Miner AS, Gong Y, Ciuchta MP, Sadler A, Surdyk J (2012) Promoting university startups: international patterns, vicarious learning and policy implications. J Technol Transf 37(2):213–233Google Scholar
  122. Nagle M (2007) Canonical analysis of university presence and industrial comparative advantage. Econ Dev Q 21(4):325–338View ArticleGoogle Scholar
  123. Narin F, Hamilton KS, Olivastro, D (1995) Linkage between agency supported research and patented industrial technology. Research Evaluation 5(3): 183–187 Narin F, Hamilton KS, Olivastro D (1997) The increasing linkage between US technology and public science. Res Policy 26(3):317–330Google Scholar
  124. Nelson AJ (2009) Measuring knowledge spillovers: what patents, licenses and publications reveal about innovation diffusion. Res Policy 38:994–1005View ArticleGoogle Scholar
  125. Nelson RR (1986) Institutions supporting technical advance in industry. Am Econ Rev 76:186–189Google Scholar
  126. Nelson RR (1993) National innovation systems: a comparative study. Oxford Univ. Press, New YorkGoogle Scholar
  127. NIH (2015) About NIH: Budget. http://www.nih.gov/about-nih/what-we-do/budget. Accessed 26 Dec 2015
  128. Nonaka I, Takeuchi A (1995) The knowledge-creating company. Oxford University Press, NYGoogle Scholar
  129. NSF (2015) FY2014 Agency financial report. December 2014. http://www.nsf.gov/pubs/2015/nsf15002/pdf/nsf15002.pdf. Accessed 26 Dec 2015
  130. O’Shea R, Allen T, Chevalier A, Roche F (2005) Entrepreneurial orientation, technology transfer and spinoff performance of US universities. Res Policy 34(7):994–1009View ArticleGoogle Scholar
  131. OECD (2011) Public research institutions: mapping sector trends, OECD Publishing. http://dx.doi.org/10.1787/9789264119505-en. Accessed 26 Dec 2015
  132. O'Shea RP, Allen TJ, Morse KP, O'Gorman C, Roche F (2007) Delineating the anatomy of an entrepreneurial university: the Massachusetts Institute of Technology experience. R & D Manag 37(1):1–16View ArticleGoogle Scholar
  133. Owen-Smith J, Powell WW (2003) The expanding role of university patenting in the life sciences: assessing the importance of experience and connectivity. Res Policy 32(9):1695–1711View ArticleGoogle Scholar
  134. Pakes A, Griliches Z (1984) Patents and R&D at the firm level: a first look. In: Griliches Z (ed) R&D, patents and productivity. University of Chicago Press, Chicago, pp 55–71Google Scholar
  135. Park JB, Ryu TK, Gibson DV (2010) Facilitating public-to-private technology transfer through consortia: initial evidence from Korea. J Technol Transf 35(2):237–252View ArticleGoogle Scholar
  136. Pastor J, Pérez F, de Guevara J (2013) Measuring the local economic impact of universities: an approach that considers uncertainty. Higher Educ 65(5):539–564View ArticleGoogle Scholar
  137. Penfield T, Baker M, Scoble R, Wykes (2014) Assessment, evaluations, and definitions of research impact: a review. Res Eval 23:21–32View ArticleGoogle Scholar
  138. Perkmann M, Neely A, Walsh K (2011) How should firms evaluate success in university-industry alliances? A performance measurement system. R & D Manag 41(2):202–216View ArticleGoogle Scholar
  139. Podsakoff PM, MacKenzie SB, Bacharach DG, Podsakoff NP (2005) The influence of management journals in the 1980s and 1990s. Strateg Manag J 26:473–88View ArticleGoogle Scholar
  140. Powers JB, McDougall P (2005a) Policy orientation effects on performance with licensing to start-ups and small companies. Res Policy 34(7): 1028–1042Google Scholar
  141. Powers JB, McDougall P (2005b) University start-up formation and technology licensing with firms that go public: a resource-based view of academic entrepreneurship. J Bus Venturing 20(3): 291–311Google Scholar
  142. Pressman L, Guterman S, Abrams I, Geist D, Nelsen L (1995) Pre-production investment and jobs induced by MIT exclusive patent licenses: a preliminary model to measure the economic impact of university licensing. J Assoc Univ Technol Managers 7:49–82Google Scholar
  143. Pries F, Guild P (2011) Commercializing inventions resulting from university research: analyzing the impact of technology characteristics on subsequent business models. Technovation 31(4):151–160View ArticleGoogle Scholar
  144. Probert J, Connell D, Mina A (2013) R&D service firms: the hidden engine of the high-tech economy? Res Policy 42(6-7):1274–1285View ArticleGoogle Scholar
  145. ProTon (1991-2012) The ProTon Europe annual survey reports. Brussels: ProTon EuropeGoogle Scholar
  146. Rasmussen E, Borch OJ (2010) University capabilities in facilitating entrepreneurship: a longitudinal study of spin-off ventures at mid-range universities. Res Policy 39(5):602–612View ArticleGoogle Scholar
  147. REF 2014 (2011) Decisions on assessing research impact, (Pubd online) http://www.ref.ac.uk/media/ref/content/pub/assessmentframeworkandguidanceonsubmissions/GOS%20including%20addendum.pdf accessed 19 Dec 2012
  148. Riddel M, Schwer RK (2003) Regional innovative capacity with endogenous employment: empirical evidence from the U.S. Rev Reg Stud 33:73–84Google Scholar
  149. Roessner D, Bond J, Okubo S, Planting M (2013) The economic impact of licensed commercialized inventions originating in university research. Res Policy 42:23–34View ArticleGoogle Scholar
  150. Roessner D, Manrique L, Park J (2010) The economic impact of engineering research centers: preliminary results of a pilot study. J Technol Transf 35(5):475–493View ArticleGoogle Scholar
  151. Rogers E, Hall B, Hashimoto M, Steffensen M, Speakman K, Timko M (1999) Technology transfer from university-based research centers: the University of New Mexico experience. J Higher Educ 70:687–705View ArticleGoogle Scholar
  152. Rogers E, Yin J, Hoffmann J (2000) Assessing the effectiveness of technology transfer offices at U.S. research universities. J Assoc Univ Technol Managers 12:47–80Google Scholar
  153. Romer P (1990) Endogenous technological change. J Pol Econ 98:S71–S102View ArticleGoogle Scholar
  154. Rosenberg N, Nelson R (1994) American universities and technical advance in industry. Res Policy 23:325–348View ArticleGoogle Scholar
  155. Sáez CB, Marco TG, Arribas EH (2002) Collaboration in R&D with universities and research centres: an empirical study of Spanish firms. R&D Manag 32:321–41View ArticleGoogle Scholar
  156. Salter A, Martin B (2001) The economic benefits of publicly funded basic research: a critical review. Res Policy 30:509–532View ArticleGoogle Scholar
  157. Sine WD, Scott S, Di Gregorio D (2003) The halo effect and technology licensing: the influence of institutional prestige on the licensing of university inventions. Manag Sci 49(4):478–496View ArticleGoogle Scholar
  158. Sivitanidou R, Sivitanides P (1995) The intrametropolitan distribution of R&D activities: theory and empirical evidence. J Reg Sci 35:391–415View ArticleGoogle Scholar
  159. Smith S, Ward V, House A (2011) ‘Impact’ in the proposals for the UK’s Research Excellence Framework: shifting the boundaries of academic autonomy. Res Policy 40(10):1369–1379View ArticleGoogle Scholar
  160. Spencer J (2001) How relevant is university-based scientific research to private high-technology firms? A United States–Japan comparison. Acad Manage J 44:432–40View ArticleGoogle Scholar
  161. Steinacker A (2005) The economic effect of urban colleges on their surrounding communities. Urban Stud 42:1161–75View ArticleGoogle Scholar
  162. Swann P, Prevezer M (1996) A comparison of the dynamics of industrial clustering in computing and biotechnology. Res Policy 25:1139–57View ArticleGoogle Scholar
  163. Tassey G (1999) Lessons learned about the methodology of the economic impact studies: the NIST experience. Eval Program Plann 22:113–119View ArticleGoogle Scholar
  164. Tassey G (2005) The disaggregated technology production function: a new model of university and corporate research. Res Policy 34(3):287–303View ArticleGoogle Scholar
  165. Tornquist K, Hoenack S (1996) Firm utilization of university scientific research. Res Higher Educ 37:509–34View ArticleGoogle Scholar
  166. Tranfield D, Denyer D, Smart P (2003) Towards a methodology for developing evidence-informed management knowledge by means of systematic review. Br J Manag 14:207–22View ArticleGoogle Scholar
  167. van Hemert P, Nijkamp P (2010) Knowledge investments, business R&D and innovativeness of countries: a qualitative meta-analytic comparison. Technol Forecasting Soc Change 77(3):369–384View ArticleGoogle Scholar
  168. Varga A (2000) Local academic knowledge transfers and the concentration of economic activity. J Reg Sci 40:289–309View ArticleGoogle Scholar
  169. Villasana M (2011) Fostering university-industry interactions under a triple helix model: the case of Nuevo Leon, Mexico. Sci Public Policy 38(1):43–53View ArticleGoogle Scholar
  170. Vincett P (2010) The economic impacts of academic spin-off companies, and their implications for public policy. Res Policy 39:736–747View ArticleGoogle Scholar
  171. Wallmark JT (1997) Inventions and patents at universities: the case of Chalmers University of Technology. Technovation 17(3):127–139View ArticleGoogle Scholar
  172. Wang X, Zhang X, Xu S (2011) Patent co-citation networks of Fortune 500 companies. Scientometrics 88(3):761–770View ArticleGoogle Scholar
  173. Weiler S (2000) Information and market failure in local economic development: a new role for universities? Econ Dev Q 14:194–203View ArticleGoogle Scholar
  174. Wernerfelt B (1984) A resource-based view of the firm. Strateg Manag J 5:171–180View ArticleGoogle Scholar
  175. Williamson OE (1979) Transaction cost economics: the governance of contractual relations. J Law Econ 23(2):233–261View ArticleGoogle Scholar
  176. Zucker L, Darby M, Brewer M (1998) Intellectual human capital and the birth of the U.S. biotechnology enterprises. Am Econ Rev 88:290–306Google Scholar

Copyright

© Cheah and Yu. 2016