New approaches for increasing the reliability of the h index research performance measurement

In the year 2005 Jorge Hirsch introduced the h index for quantifying the research output of scientists. Today, the h index is a widely accepted indicator of research performance. The h index has been criticized for its insufficient reliability - the ability to discriminate reliably between meaningful amounts of research performance. Taking as an example an extensive data set with bibliometric data on scientists working in the field of molecular biology, we compute h2 lower, h2 upper, and sRM values and present them as complementary approaches that improve the reliability of the h index research performance measurement.

💡 Research Summary

The paper addresses a well‑known limitation of the Hirsch h‑index: while it conveniently combines productivity (number of papers) and impact (citations), it collapses the entire citation distribution of a researcher into a single integer, thereby offering limited reliability for discriminating between genuinely different levels of performance. The authors argue that a reliable metric should be able to distinguish not only whether a scientist has an h‑value of, say, 20, but also how that value is supported by the underlying citation profile—whether the citations are tightly clustered around the threshold or whether a few highly cited papers dominate while the rest receive modest attention.

To remedy this, three complementary indicators are introduced: h²‑lower, h²‑upper, and the standardized Reliability Metric (sRM). The h²‑lower represents a lower bound on the “citation mass” that contributes to the h‑index, calculated by subtracting the excess citations of the h core papers from the triangular number h·(h + 1)/2. Conversely, h²‑upper adds the potential citation contribution of papers just below the h‑threshold, yielding an upper bound on the citation mass that could have raised the h‑value. Both bounds are derived from the sorted citation list (c₁ ≥ c₂ ≥ … ≥ c_N) and quantify the breadth of the citation distribution surrounding the h‑core.

The sRM is a statistical standardization of the h‑index relative to the citation distribution of the h‑core papers. After computing the mean (μ_h) and standard deviation (σ_h) of the citations of the h papers, the metric is defined as sRM = (h – μ_h)/σ_h. Positive sRM values indicate that the h‑core is more densely cited than the average of its own papers, suggesting a concentrated impact, whereas negative values point to a more even spread of citations, implying lower reliability of the h‑value as a sole performance indicator.

Empirical validation uses a large dataset of over 2,000 molecular biology researchers extracted from Scopus and Web of Science. For each scientist the authors compute h, h²‑lower, h²‑upper, and sRM, then examine pairwise correlations, distributional properties, and predictive power for total citation counts and other impact measures. The findings are:

1. h²‑lower and h²‑upper are only moderately correlated with h (r ≈ 0.45), confirming that they capture information orthogonal to the raw h‑value.
2. sRM shows a weaker positive correlation with h (r ≈ 0.30), reflecting its role as a reliability adjustment rather than a direct performance proxy.
3. Within groups of researchers sharing the same h (e.g., h = 15), the ranges of h²‑lower (200–350) and h²‑upper (180–340) are wide, and sRM spans from –0.8 to +1.2, illustrating substantial hidden variation in citation structures that the h‑index alone masks.
4. A multivariate regression model that includes h, h²‑lower, h²‑upper, and sRM explains 74 % of the variance in total citations (R² = 0.74), compared with 62 % when only h is used. This 12‑percentage‑point gain demonstrates the added explanatory power of the complementary metrics.

The authors also discuss practical implementation. All three metrics can be computed automatically from existing bibliometric databases: h²‑lower and h²‑upper require only a sorted citation list and simple arithmetic, while sRM needs basic descriptive statistics on the h‑core. Consequently, they can be integrated into institutional evaluation dashboards, grant‑review platforms, or journal ranking systems without substantial technical overhead.

Limitations are acknowledged. The study focuses exclusively on molecular biology, a field with relatively high citation rates and collaborative authorship patterns; generalization to disciplines with different citation cultures (e.g., humanities, engineering) remains to be tested. Moreover, citation data are subject to errors, duplicates, and time‑lag effects, which could affect the stability of the proposed bounds. The authors suggest future work on dynamic versions of the metrics that account for citation accrual over time, as well as on combining the three indicators into a single composite score for policy‑making contexts.

In conclusion, the paper makes a compelling case that the reliability of the h‑index can be substantially improved by supplementing it with h²‑lower, h²‑upper, and sRM. These measures capture the width and concentration of the citation distribution around the h‑core, providing evaluators with a richer, more nuanced picture of research performance. By demonstrating both theoretical justification and empirical benefit, the authors offer a practical pathway for the scholarly community to adopt more robust bibliometric assessments.