Scopus SNIP Indicator

Rejoinder to Moed [arXiv:1005.4906]: Our main objection is against developing new indicators which, like some of the older ones (for example, the “crown indicator” of CWTS), do not allow for indicating error because they do not provide a statistics, but are based, in our opinion, on a violation of the order of operations. The claim of validity for the SNIP indicator is hollow because the normalizations are based on field classifications which are not valid. Both problems can perhaps be solved by using fractional counting.

💡 Research Summary

**
The paper is a pointed rejoinder by Loet Leydesdorff and Tobias Opthof to Henk F. Moed’s defense of the Source‑Normalized Impact per Paper (SNIP) indicator used in Elsevier’s Scopus database. The authors argue that SNIP cannot be regarded as a proper statistical measure because it is constructed by dividing the mean citation count of a set of papers by the median citation count of another set. This ratio violates basic statistical principles—specifically the order of operations—and precludes the calculation of standard errors, confidence intervals, or any hypothesis testing. Consequently, differences in impact between journals or groups of journals cannot be assessed for statistical significance, undermining the claim that SNIP is a “sophisticated” and “valid” indicator.

A second, more fundamental criticism concerns the field‑level normalisation that underpins SNIP. The indicator assumes that the a‑priori field classifications (e.g., ISI Subject Categories or Scopus document‑type groupings) are valid representations of scientific domains. Leydesdorff and Opthof contend that these classifications were created for information retrieval, not for analytical purposes, and therefore lack a solid empirical basis. They cite previous work showing that algorithmic or content‑based classifications differ substantially from the traditional categories, and that the latter often mix heterogeneous research areas. Because SNIP’s normalisation relies on these shaky field delineations, its validity is “hollow.”

To illustrate the practical problems, the authors compare citation data for five journals (Inventiones Mathematicae, Molecular Cell, Journal of Electronic Materials, Mathematical Research Letters, and Annals of Mathematics) using both the Science Citation Index (SCI) and Scopus. They find notable discrepancies: Scopus records more citing papers for Molecular Cell after correcting for “citable items,” largely due to the inclusion of document types such as “Short Surveys,” “Notes,” and “Editorials.” The authors argue that these “non‑citable” items should not be excluded arbitrarily, as many of them are substantive contributions (e.g., reviews, introductions) that receive citations and thus affect impact measures. This further demonstrates that the underlying document‑type classifications in Scopus are unreliable for citation‑based evaluation.

The authors propose fractional counting of citations as a robust alternative. In this approach, each citation is weighted by the inverse of the total number of references in the citing paper, thereby normalising for differences in citation behaviour across fields. Fractionally counted citation distributions retain their mean and variance, allowing the application of standard statistical tests (t‑tests, non‑parametric tests, etc.) to assess whether two journals differ significantly. Leydesdorff and Opthof apply this method to the same five journals and find, for example, that the citation distributions of Inventiones Mathematicae and Annals of Mathematics are not significantly different in 2007—a conclusion that could not be drawn from SNIP values alone.

Finally, the paper notes that the authors are extending this fractional‑counting approach to develop a statistically sound classification of journals, aiming to replace or complement existing indicators such as the “crown indicator” used by CWTS. They emphasize that any new metric must provide a clear statistical framework, including measures of uncertainty, and must be based on empirically validated field delineations. In sum, the rejoinder challenges the methodological foundations of SNIP, highlights the inadequacy of current field classifications, and offers fractional counting as a statistically rigorous solution for normalising citation impact across scientific domains.