How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations

Although bibliometrics has been a separate research field for many years, there is still no uniformity in the way bibliometric analyses are applied to individual researchers. Therefore, this study aims to set up proposals how to evaluate individual researchers working in the natural and life sciences. 2005 saw the introduction of the h index, which gives information about a researcher’s productivity and the impact of his or her publications in a single number (h is the number of publications with at least h citations); however, it is not possible to cover the multidimensional complexity of research performance and to undertake inter-personal comparisons with this number. This study therefore includes recommendations for a set of indicators to be used for evaluating researchers. Our proposals relate to the selection of data on which an evaluation is based, the analysis of the data and the presentation of the results.

💡 Research Summary

The paper addresses the persistent problem of evaluating individual researchers in the natural and life sciences with a method that goes beyond the simplistic use of the h‑index. While the h‑index combines productivity (number of papers) and impact (citations) into a single figure, it fails to capture the multidimensional nature of scientific performance and does not allow fair interpersonal comparisons. To remedy this, Bornmann and Marx propose a set of bibliometric indicators grounded in citation percentiles, together with detailed procedural recommendations for data collection, analysis, and reporting.

First, the authors stress the importance of using a sufficiently large publication set. They recommend at least 50 papers per researcher, ideally the entire career output, to ensure statistical reliability and to avoid the need for inferential extrapolation from a sample. This approach shifts the focus from a snapshot of recent activity to a comprehensive view of a scientist’s career.

Second, the citation window is carefully defined. The most recent one to two years are excluded because citations have not yet stabilized. Based on empirical studies, a citation window of three to five years (or at least two years) is recommended, reflecting the typical peak of citations in most natural‑science disciplines. This window balances the need for timely assessment with the requirement for reliable impact measurement.

Third, the treatment of self‑citations is nuanced. Self‑citations are considered a normal part of scholarly communication and are included in the primary analysis, but the authors advise flagging cases where self‑citations exceed roughly 30 % of total citations, as this may indicate strategic self‑promotion.

Data sources are limited to the Web of Science (WoS) and Scopus, which provide reliable coverage and citation counts. The authors caution against using Google Scholar due to its lack of transparency and inconsistent coverage. To mitigate name ambiguity, they recommend cross‑checking database results with personal publication lists, institutional repositories, and unique researcher identifiers such as ORCID or ResearcherID.

Normalization of citation counts is performed using field‑ and year‑specific reference values supplied by Thomson Reuters’ National Citation Report or InCites. This yields a relative citation rate (percentile) for each paper, indicating how a publication performs compared to its peers in the same discipline and year.

The proposed indicator suite includes: (1) total number of publications (productivity), (2) average citation percentile (overall impact), (3) proportion of papers in the top 10 % percentile (high‑quality output), (4) self‑citation rate (extent of self‑referencing), and (5) citation trajectory over time (career‑stage dynamics). The indicators are deliberately chosen to minimize redundancy, as many bibliometric measures are highly correlated.

To illustrate the methodology, the authors apply the framework to three anonymized researchers (Person 1, 2, 3) who differ in age, career stage, and academic success. Detailed tables and graphs show how each indicator varies across the three cases, revealing differences that a single h‑index would obscure.

In conclusion, the paper argues that a percentile‑based, multi‑indicator approach provides a more nuanced, fair, and transparent assessment of individual researchers. It offers concrete guidance on data handling, citation window selection, self‑citation monitoring, database choice, and normalization procedures, thereby furnishing institutions and funding agencies with a robust tool for hiring, promotion, and resource allocation decisions.