Scale-independent Indicators and Research Evaluation

Katz, J. Sylvan (2000) Scale-independent Indicators and Research Evaluation . Science and Public Policy 27(1):pp. 23-36.

Abstract

Does the size of institution or system affect the amount of recognition it receives, the impact it has on others and the degree to which it collaborates? Is it possible to optimise size to maximise recognition, impact and co-operation? This paper demonstrates that some conventional indicators used in research evaluation may fail to account for the non-linearity between size and performance. This can result in an over- and under-estimation of the research performance of both large and small institutions and nations. This paper shows that a power law relationships exists between recognition or impact and (a) the publishing size of scientific communities within an OECD science system and (b) the publishing size of a research community across OECD science systems or institutions in a science system. Also, a power law relationship exists between the amount of various types of collaboration and the ublishing size of institutions. It also shows that there are power law relationships between publishing size and HERD or number of researchers. The exponent of the power law is sometimes greater than 1.0 indicating the existence of a "Matthew effect". Other times it is less than 1.0 indicating an "inverse Matthew effect". A power law is the common signature of a scale-independent process that can be typified by a geometric fractal and other self-similar properties. A new class of scale-independent indicators is developed to overcome the inequity produced by some non-linear characteristics commonly measured when evaluating research performance.