How are new citation-based journal indicators adding to the bibliometric toolbox?

The launching of Scopus and Google Scholar, and methodological developments in Social Network Analysis have made many more indicators for evaluating journals available than the traditional Impact Factor, Cited Half-life, and Immediacy Index of the ISI. In this study, these new indicators are compared with one another and with the older ones. Do the various indicators measure new dimensions of the citation networks, or are they highly correlated among them? Are they robust and relatively stable over time? Two main dimensions are distinguished – size and impact – which together shape influence. The H-index combines the two dimensions and can also be considered as an indicator of reach (like Indegree). PageRank is mainly an indicator of size, but has important interactions with centrality measures. The Scimago Journal Ranking (SJR) indicator provides an alternative to the Journal Impact Factor, but the computation is less easy.

💡 Research Summary

The paper investigates how recently developed citation‑based journal indicators expand the bibliometric toolbox beyond the traditional ISI metrics—Impact Factor (IF), Cited Half‑life, and Immediacy Index. The authors begin by noting that the emergence of large‑scale citation databases such as Scopus and Google Scholar, together with advances in Social Network Analysis (SNA), have made a plethora of new metrics available. Their central research questions are: (1) Do these new indicators capture dimensions of journal performance that are distinct from the classic ones? (2) How strongly are the various metrics correlated, and (3) Are they stable over time?

To answer these questions, the study assembles a dataset covering roughly 8,000 scholarly journals indexed in ISI, Scopus, and Google Scholar over a five‑year window (2004‑2008). For each journal the authors collect the three traditional ISI indicators and nine newer measures: H‑index, PageRank, Scimago Journal Rank (SJR), Eigenfactor, Indegree, Outdegree, Weighted PageRank, and a size‑adjusted citation count. Statistical analyses include Pearson correlation matrices, principal component analysis (PCA), and Spearman rank‑order comparisons across successive years to assess temporal robustness.

The results reveal a clear bifurcation of all metrics into two principal dimensions, which the authors label “size” and “impact.” Size‑related indicators (total citations, number of citable items, Indegree, and PageRank) are highly inter‑correlated (r > 0.85), reflecting the overall volume of scholarly activity associated with a journal. Impact‑related indicators (IF, Eigenfactor, average citations per article, Immediacy Index) also cluster tightly (r > 0.80), capturing the average influence of each published item. Importantly, the cross‑dimension correlation is modest (r ≈ 0.45), indicating that size and impact convey largely independent information.

The H‑index emerges as a hybrid metric that integrates both dimensions; it can be interpreted as a measure of “reach” because it reflects how many articles achieve a citation threshold, linking closely with Indegree. PageRank, derived from SNA, behaves primarily as a size indicator but incorporates network centrality: it correlates strongly with size metrics yet also aligns with Eigenvector Centrality, suggesting sensitivity to the prestige of citing sources. SJR, while conceptually similar to IF (a weighted average of citations), differs by weighting citations according to the prestige of the citing journal. Its computation is more complex and data‑intensive, yet it provides a distinct ranking pattern, especially for non‑English‑language journals.

Temporal analysis shows that most indicators are stable across the five‑year span, with size‑based measures exhibiting the least fluctuation. PageRank and SJR display modest variability, reflecting their dependence on database updates and changes in the underlying citation network.

In the discussion, the authors argue that the new suite of metrics enriches journal evaluation by offering multidimensional insight: size captures the breadth of a journal’s output, impact reflects per‑article influence, and hybrid measures like H‑index and reach‑oriented metrics reveal how widely a journal’s work disseminates across the scholarly community. They suggest that future research should explore field‑specific weighting schemes, combine metrics into composite scores, and investigate how these indicators perform in research assessment exercises.

Overall, the study concludes that while many of the new citation‑based indicators are correlated with each other, they collectively provide complementary perspectives on journal performance. The distinction between size and impact, the hybrid nature of the H‑index, and the network‑aware properties of PageRank and SJR together broaden the analytical possibilities for bibliometricians, editors, and research policymakers.