Solving reviewer assignment problem in software peer review: An approach based on preference matrix and asymmetric TSP model

Optimized reviewer assignment can effectively utilize limited intellectual resources and significantly assure review quality in various scenarios such as paper selection in conference or journal, proposal selection in funding agencies and so on. However, little research on reviewer assignment of software peer review has been found. In this study, an optimization approach is proposed based on students’ preference matrix and the model of asymmetric traveling salesman problem (ATSP). Due to the most critical role of rule matrix in this approach, we conduct a questionnaire to obtain students’ preference matrices and convert them to rule matrices. With the help of software ILOG CPLEX, the approach is accomplished by controlling the exit criterion of ATSP model. The comparative study shows that the assignment strategies with both reviewers’ preference matrix and authors’ preference matrix get better performance than the random assignment. Especially, it is found that the performance is just a little better than that of random assignment when the reviewers’ and authors’ preference matrices are merged. In other words, the majority of students have a strong wish of harmonious development even though high-level students are not willing to do that.

💡 Research Summary

The paper tackles the reviewer‑author assignment problem that arises in software engineering courses where students conduct peer reviews of each other’s code. While reviewer assignment has been extensively studied for conference paper selection, funding proposals, and patent examination, little work has addressed the educational context in which every participant is simultaneously a reviewer and an author. To fill this gap, the authors propose an optimization framework that incorporates students’ expressed preferences and models the assignment as an asymmetric traveling salesman problem (ATSP).

First, a questionnaire was administered to a class of 30 software‑engineering students. Each student rated, on a five‑point Likert scale, (i) the peers they would like to review (reviewer‑preference) and (ii) the peers from whom they would like to receive feedback (author‑preference). The raw data produced two asymmetric preference matrices, PR (reviewer‑centric) and PA (author‑centric). Because the ATSP requires a cost matrix where lower values indicate more desirable assignments, the authors transformed the preference scores by taking reciprocals, normalising to the