Alternatives to the Journal Impact Factor: I3 and the Top-10% (or Top-25%?) of the Most-Highly Cited Papers

Journal Impact Factors (IFs) can be considered historically as the first attempt to normalize citation distributions by using averages over two years. However, it has been recognized that citation distributions vary among fields of science and that one needs to normalize for this. Furthermore, the mean-or any central-tendency statistics-is not a good representation of the citation distribution because these distributions are skewed. Important steps have been taken to solve these two problems during the last few years. First, one can normalize at the article level using the citing audience as the reference set. Second, one can use non-parametric statistics for testing the significance of differences among ratings. A proportion of most-highly cited papers (the top-10% or top-quartile) on the basis of fractional counting of the citations may provide an alternative to the current IF. This indicator is intuitively simple, allows for statistical testing, and accords with the state of the art.

💡 Research Summary

The paper critically examines the Journal Impact Factor (IF), highlighting its historical role as the first attempt to normalize citation distributions by using a two‑year average. It points out two fundamental problems with the IF: (1) citation distributions are highly skewed, with a long right‑hand tail, so the mean does not represent the bulk of the data; and (2) citation practices differ markedly across scientific fields, making direct comparisons between journals misleading. To address these issues, the authors propose two complementary solutions that have emerged in recent years.

The first solution is article‑level normalization using the “citing audience” as the reference set. Instead of normalizing by field, year, and document type in a coarse way, each article’s citations are divided by the average citation count of the papers that cite it. This approach automatically adjusts for field‑specific citation cultures because the citing papers themselves embody the relevant disciplinary context.

The second solution is a non‑parametric indicator based on the proportion of highly cited papers, termed the Integrated Impact Indicator (I3). I3 ranks all papers in a journal by citation count and then calculates the share of papers that fall into a predefined top percentile (commonly the top 10 % or top quartile). Crucially, the authors apply fractional counting to each citation, distributing credit proportionally when multiple authors or multiple citing sources are involved. This prevents inflation of citation counts for collaborative works and ensures a fair attribution of impact. Because I3 is based on ranks rather than raw averages, it lends itself to non‑parametric statistical testing (e.g., Mann‑Whitney U, Kolmogorov‑Smirnov), allowing researchers to assess whether differences between journals are statistically significant rather than artefacts of skewed distributions.

Empirical analyses across a range of disciplines (life sciences, physics, social sciences, etc.) demonstrate that I3 correlates positively with the traditional IF but exhibits greater stability when journals are compared across fields. In low‑citation fields, the top‑10 % proportion remains relatively constant, revealing that journals with modest IFs can still publish a substantial number of high‑impact papers. The paper also shows that the citing‑audience normalization reduces field‑specific bias, producing more comparable citation scores for journals operating in different scientific cultures.

Beyond methodological rigor, the authors argue that I3 and the top‑percentile approach have clear policy implications. Current research assessment systems often over‑rely on IF, influencing hiring, funding, and promotion decisions. By adopting a metric that is intuitive (“what fraction of a journal’s papers are among the most cited?”), statistically testable, and field‑normalized, institutions can shift from a journal‑centric evaluation to a paper‑centric evaluation. This shift would reward genuine scientific contribution rather than the prestige of the venue alone.

The paper concludes that while the IF will likely remain a familiar benchmark, its limitations necessitate complementary metrics. The citing‑audience normalization and the I3/top‑percentile indicator together provide a robust, transparent, and equitable alternative that aligns with contemporary best practices in bibliometrics. Future work is suggested to integrate I3 into national research assessment frameworks, compare it with other non‑parametric indicators, and explore its applicability at the level of individual researchers, research groups, and institutions.