 |
Indicators on Science, Technology and Innovation.
History and new Perspectives |
 |
 |
|
Indicators on Science, Technology and Innovation.
History and new Perspectives |
|
|
|
Indicators on Science, Technology and Innovation. History and New Perspectives
The Conference will focus on two closely related issues:
One hundred years of STI indicators 2006 marks a centennial: that of the systematic collection of statistics on science, technology and innovation (see http://www.csiic.ca/centennial.html). In 1906 James McKeen Cattell, an American psychologist and editor of Science for fifty years (1895-1944), published the first edition of a directory of scientists entitled American Men of Science. Based on the directory, Cattell produced regular statistical analyses for thirty years on the demography, geography and what he called the performance of scientists. At the same time (early 1900s), psychologists developed the systematic use of bibliometrics (counting scientific papers) as a tool for measuring the advancement of psychology as a science. Since these very first exercises, the measurement of science, technology and innovation has changed considerably (Godin 2005a). At its very beginning, statistics was concerned with measuring the size of the scientific community (counting the number of “men of science”) and scientists’ activities (counting papers). The measurements were conducted by scientists themselves, among them psychologists and geographers. In the early 1920s, these statistics and their sources became institutionalized and, from the 1940s to the 1950s, new ones were constructed. It was no longer scientists, but government departments and national bureaus of statistics which produced the statistics, those on which most of us rely today. The main works conducted by these public institutions, unlike previous measurements, dealt with measuring a “national budget for science” by counting the money devoted to R&D. The focus was no longer exclusively on universities, as Cattell’s and others have been, but on all economic sectors: industry, government, university, and non-profit. The focus was no longer on “men of science” either, but on organizations and their R&D activities. Above all, the focus was on measuring the efficiency or “productivity” of the science system, defined as the output arising from research activities. Over the past hundred years, thanks to national statistical offices, we have gained in terms of the diversity, quality and robustness of statistics. However, we have also lost some fundamentals. The very first measurements on science centered on counting “men of science”, because human resources was considered to be what an influential American report of 1947 identified as the ultimate resource: “the ceiling on research and development activities is fixed by the availability of trained personnel, rather than by the amounts of money available. The limiting resource is manpower” (US President Scientific Research Board, 1947). Today, the statistics on human resources in science and technology is one of the poorest sets of statistics that we possess. National as well as international organizations are aware of this fact, and are trying to remedy the deficiency. But progress is slow. Another lost opportunity of the last hundred years, with regard to statistics, is measuring outcomes. If one distinguishes output from outcomes, he observes that there exist very few indicators on the outcomes of research, and those that exist are all of an economic type, among them productivity indicators. Representatives of the indicators of an economic type include: patents, innovations, high-technology trade, and the technological balance of payments. Generally, the measurement of the social impacts of science is totally missing: education, health, environment, quality of life, etc. These were precisely the outcomes that Cattell identified as arising from science. To him, economics and industry was a means, not an end. Today it is the market and the priorities of governments with regard to the market that drive entirely measurements. Admittedly, the challenges are many for anyone concerned with measuring intangible outcomes of a social type. There are currently several initiatives in many countries looking precisely at how to measure outcomes other than strictly economic outcomes. Unfortunately, these initiatives are not conducted by statistical offices, but by government departments, for their own ad hoc needs and not necessarily for developing national indicators of a systematic nature. Developing such indicators is nevertheless as important as measuring social capital or knowledge management. The past hundred years has been a very productive period with regard to statistics on science. Since the 1950s, we owe this progress mainly to governments and their statistical offices. Most of the time, academics are actually users of statistics produced by the state. Statistics on science are now at a crossroads. Users of statistics are asking for a lot more information than before because their analyses and/or decisions are more fine-grained. National aggregates are no longer enough, and neither are standard classifications. The new challenge: positioning indicators In the last fifty years, most of the work on STI indicators has been based on an Input/output model (Godin 2005b), where the indicators are structured along the categories of ‘input’, ‘ouput’ and ‘outcome’, following a production function logic, in a national accounts paradigm. As a consequence the level of analysis is the country and the indicators are built as aggregates at this level, the individual agents disappear and the national innovation systems is reduced to a few aggregated sectors, described in its different aspects. Good examples of this kind of indicators production are R&D statistics based on the Frascati manual (OECD 2002), human resources statistics based on the Canberra manual (reference) and innovation statistics and surveys based on the Olso manual. Despite some well-know shortcomings in the quality and comparability of data (Godin 2005a; Lepori 2005), this activity has reached a rather mature stage, with a clear division of work between international agencies like OECD and Eurostat playing a role of harmonization and of production of international comparative statistics, while national statistical agencies are in charge of collecting and elaborating data at national level (Esterle and Theves 2005). However, the last two decades have seen the emergence of a set of indicators based on a different framework, which some call positioning indicators (Barré 2005): the indicators aim at characterising the elements of national innovation system as such, considering it is made of differentiated, autonomous and strategic agents. It considers important to describe their interactions, linkages as well as their complementarities, competition and cooperation, to figure out the different types and categories they belong to. The indicators focus on the description of the actors within the system, the key notion being that of a distributed intelligence, where the various actors have a strategy of their own. Positioning indicators have a very different status, being an emerging activity, performed in an ad-hoc way, by a variety of institutions. The actors producing such indicators can be University research teams (Manchester university, Leiden University…), special units within National statistical Offices (Norway), Contract research organisation (FhG-ISI, TNO…), private consultancies (Technopolis…), dedicated institutions with a coordinating mission (OST in France and its positioning indicators cooperative, the Dutch indicators consortium…) or professional associations. An illustration of the situation is the Report on European S&T Indicators (REIST) issued by the Commission in 2003, where a good part of the indicators are positioning indicators based on ad-hoc treatment of data; these indicators are produced by a large variety of institutions. PRIME promoted during the last two years the development of specific positioning indicators concerning two central domains for the European Research Area: firstly, indicators on individual higher education institutions (AQUAMETH and Observatory of European Universities; CHINC project funded by IPTS; Bonaccorsi, Daraio and Lepori 2005) and, secondly, indicators concerning public funding of research activities and, especially, project funding (Lepori et al. 2005). Both activities proved the feasibility of the production of such indicators by collection and harmonization of data from various national sources: examples of interesting applications of this approach are comparative work on efficiency of higher education institutions in Europe (Bonaccorsi and Daraio 2006), the analysis of the evolution of higher education funding in Europe during the last ten years (Lepori et al. 2005) and the first quantitative comparison of project funding systems in European countries (Lepori et al. 2005a). Hence a session in the conference will be devoted to the presentation of examples of new indicators indicators and of their application for policy analysis, both from projects inside and outside PRIME. Towards an European platform for Science and Technology Indicators (ESTIP) Despite this considerable progress, PRIME activities showed a number of limitations in the development of positioning indicators, especially concerning methodology and comparability of data (Bonaccorsi, Daraio and Lepori 2005; Slipersaeter et al. 2005); thus, if their development has to go beyond the experimental phase, there is a need for the establishment of more systematic procedures for data collection and validation, requiring a durable institutionalization beyond individual projects. This is a pressing issue also for policy needs since positioning indicators are a key element for the development of the European Research Area. We underline that the situation in this context is much more complex in Europe than in the USA, because we are (still) confronted to 26 different research and higher education systems: these differences have a direct impact on the type and quality of the available data, but also on their meaning and interpretation; hence the need of always contextualizing the produced indicators in respect to the national system considered. To address these issues, PRIME launched an exploratory activity to develop scenarios for a future European STI Indicators Platform (ESTIP), chaired by Rémi Barré and Philippe Larédo; a preliminary discussion together with the main stakeholders has been organized at the Lisbon conference, while a working document will be presented and discussed at the PRIME annual conference in Paris in February 2006 (Barré 2005). Hence the final session of the Lugano conference will be devoted to the presentation and discussion of the results of this activity in view also of the launch of the 7th Framework Programme. References Barré 2005, Towards a European STI Indicators Platform (ESTIP), Position Paper, To be presented and discussed at the second PRIME annual conference, Paris. Bonaccorsi A. and Daraio C. (eds.) (2006), Universities as strategic units. Productivity and Efficiency Patterns in the European University System, forthcoming. Bonaccorsi A., Daraio C., Lepori B. (2005), “Indicators for the analysis of higher education systems”, paper presented at the Lisbon Workshop on S&T indicators production, Lisbon 22-23 September 2005. Cattell J. M. (1906), American Men of Science: A Biographical Directory, New York: The Science Press. Esterle L., Theves J. (2005), “Analysis of the different European systems for producing indicators”, paper presented at the Lisbon Workshop on S&T indicators production, Lisbon 22-23 September 2005. Godin, B. (2005a), Measurement and Statistics on Science and Technology: 1920 to the Present, London: Routledge. Godin B. (2005b), Science, Accounting and Statistics: the Input-Output Framework, Presented at the ENIP / PRIME International Workshop “S&T Indicators for Policy Analysis: Needs, Status and New Developments”, Lisbon, 22-23 September. Lepori B. (2005), “Methodologies for analysis of research funding and expenditures”, paper presented at the Workshop on S&T indicators production, Lisbon 22-23 September 2005. Lepori B. Dinges M., Potì B. Reale E. (2005a), “Public project funding of research activities. A methodology for comparative analysis and some preliminary results”, paper presented at the Workshop on S&T indicators production, Lisbon 22-23 September 2005. Lepori B., Benninghoff M., Jongbloed B., Salerno C. Slipersaeter S. (2005), Changing Patterns of Higher Education Funding. Evidence from the CHINC countries, CHINC report. OECD (1991): OECD Proposed Guidelines for Collecting and Interpreting Technological Innovation Data, (Oslo Manual), OECD: Paris. OECD (1995), Manual on the Measurement of Human Resources in Science and Technology (Canberra Manual), Paris. OECD (2002), Frascati Manual. Proposed Standard Practice for Surveys on Research and Experimental Development, Paris. Slipersaeter S., Lepori B., Jongbloed B., Salerno C. (2005), Collecting Institutional Level Data for European Higher Education Institutions: Evidence from the CHINC Project, CHINC report. US President Scientific Research Board (1947) [1980], Science and Public Policy, New York: Arno Press.
|
|
