The excess-tail ratio: correcting journal impact factors for citation distributions

Abstract

Despite their widespread adoption, journal Impact Factors suffer well-known drawbacks that limit their usefulness in accurately and fairly assessing scientific quality. Among these is the extreme variance and skewness in the citations to articles published by a given journal, which results in their sensitivity to a few highly cited articles, and enables many infrequently cited articles to “free-ride” on citations to these “skewed few.” To address this problem, I adjust journal Impact Factors according to the relative citedness of the few highly cited articles in a journal’s h-core (i.e., the h articles that receive at least h citations) and the many infrequently cited articles in its h-tail (i.e., those that receive fewer than h citations). I gauge the skew of a journal’s citation distribution by e2/ t2, the excesstail ratio where e2 captures excess citations above the h2 citations received by a few highly cited h-core articles and t2 captures surplus citations received by the many infrequently cited h-tail articles that fall below the h-core. I employ e2/ t2 to adjust raw Impact Factors for 25 selected management and economics journals. The adjusted scores, IF, discriminate Impact Factors based on the shapes of journal citation distributions, leading to more accurate evaluation. I find e2/ t2 < 1 (often << 1) for 23 of these journals to be consistent with an overstatement of their quality resulting from the sensitivity of Impact Factors to a few highly cited articles. Adjusted Impact Factors also yield distinctive and more consistent journal rankings over standard two-year and five-year time horizons. I conclude that the “excess-tail” ratio and IF are a useful complement to journal Impact Factors, particularly given their increasing use in the evaluation of individual scholarly output.