Peer-review in the Internet age

arXiv:0810.0486v1 [physics.soc-ph] 2 Oct 2008 Peer-review in the Internet age Pawel Sobkowicz∗ Abstract The importance of peer-review in the scientific process can not be overestimated. Yet, due to increasing pressures of research and exponentially growing number of publications the task faced by the referees becomes ever more difficult. We discuss here a few possible improvements that would enable more efficient review of the scientific literature, using the growing Internet connectivity. In particular, a practical automated model for providing the referees with references to papers that might have strong relationship with the work under review, based on general network properties of citations is proposed. ∗Electronic address: countryofblindfolded@gmail.com 1 I. INTRODUCTION The process of validation of scientific results, especially at the stage of publication is one of the most important parts of scientific methodology, ensuring quality of the published work. Current discussions aimed at improving peer-review methods point out several problems that face the journal editors and referees (Alberts et al. [1], Raffet al. [2]). Moreover, the continuously increasing competitive pressure of securing own job, grants, publications, gives the task of reviewing others’ work an odium of a chore. Both technical and ethical problems connected with the peer-review process are widely recognized, and there are publications and web sites aimed at giving guidance to those who, either as the authors or as the referees, participate in the process, for example Rockwell [3], Wager et al. [13]. One way of improving the process is in helping the editors is selection of the referees who would have knowledge necessary to assess the submitted publications. Such solutions have been discussed already, for example by Rodriguez et al. [4]. These ideas are based on the networked nature of the scientific communications. In this paper we turn to the other side of the process, namely providing the already chosen referees with tools aimed at improving the speed and quality of the review process. Exponential growth of scientific literature, expressed both in terms of the page counts of individual journals and in the growth of the numbers of the journals themselves makes the task of conscientious publication review ever more difficult. Following current literature, even within one’s own field(s) of research, is quite difficult. Doing the same for fields that are somewhat remote is close to impossible. How then can a reviewer be asked to judge if an article contains enough new discoveries to warrant publication? How can the editors manage the review process with the necessary attention to detail and yet with speed and flexibility required by the growth of data? There are some initiatives, at both organizational and technical levels, that can improve the situation. II. RANDOM CITING SCIENTIST The model of Random Citing Scientist, introduced by Simkin and Roychowdhury [5, 8, 9] has been quite successful in modelling the popularity of research based on the number of quotations — using assumptions that have had nothing (or almost nothing) to do with actual 2 content, merit, and novelty. It was sufficient to assume a very simple citing process, in which when a scientist is writing a manuscript he picks up m random articles, cites them, and also copies some of their references, each with probability p. It turns out that the statistics of citations obtained within such model are very close to the actual data. Much better than in a purely statistical model. For example, there was — as in real life — a large fraction of papers with high number of citations. In fact, the model has reproduced remarkably well network structure of connections between the publications. The model has broken an unwritten taboo, treating scientists as subjects of research. Worse yet, Simkin and Roychowdhury [6, 7] have analysed the occurrences of errors in real citations, such as misprinted age numbers, or misspelled names. It turned out that quite often there are whole chains of such wrong citations, showing clearly that the assumption of copying without actually reading may well be a valid one. One of the papers was titled: Read before you cite!. If the authors do not read the papers they refer to, what can be expected from the reviewers? The initial model has been later expanded to allow for preference to cite recent articles by Vazquez [11, 12], giving even better fit to observed citation statistics. One may ask: what are these models and proofs that scientists sometimes do cite without reading have to do with peer-review process? The answer is that the simple model might teach us how to improve the process of finding how the reviewed paper stands in comparison to its “competitors” — works dealing with similar subjects. The network structure of links suggested by the model allows some degree of automation of the process of looking for works that share the same background with the checked one,

…(Full text truncated)…

This content is AI-processed based on ArXiv data.