A Motivation Model of Peer Assessment in Programming Language Learning

Peer assessment is an efficient and effective learning assessment method that has been used widely in diverse fields in higher education. Despite its many benefits, a fundamental problem in peer assessment is that participants lack the motivation to assess others’ work faithfully and fairly. Non-consensus is a common challenge that makes the reliability of peer assessment a primary concern in practices. This research proposes a motivation model that uses review deviation and radicalization to identify non-consensus in peer assessment. The proposed model is implemented as a software module in a peer code review system called EduPCR4. EduPCR4 is able to monitor this measure and trigger teacher’s arbitration when it detects possible non-consensus. An empirical study conducted in a university-level C programming course showed that the proposed model and its implementation helped to improve the peer assessment practices in many aspects.

💡 Research Summary

The paper addresses a fundamental challenge in peer assessment—lack of motivation and resulting non‑consensus among reviewers—by proposing a quantitative motivation model that can detect unreliable evaluations in programming language courses. The authors introduce two complementary metrics: review deviation, which measures how far an individual’s score deviates from the class mean using a standard‑deviation‑based threshold, and radicalization, which captures persistent patterns of overly generous or overly harsh scoring by analyzing the tail of an evaluator’s score distribution. By combining these metrics, the model can identify both accidental outliers and systematic bias that simple average‑based checks would miss.

The model is implemented as a plug‑in module within EduPCR4, a peer code‑review platform designed for university programming courses. Each time a review is submitted, EduPCR4 computes the deviation and radicalization scores in real time. If either metric exceeds a pre‑set threshold, the system automatically flags the review, sends a notification to the instructor, and presents the reviewer with a gentle prompt (“Your score differs significantly from the class average. Would you like to reconsider?”). This feedback loop encourages self‑correction while simultaneously alerting the teacher to potential fairness issues. Instructors can view aggregated dashboards that show the frequency, severity, and historical patterns of flagged reviews, allowing targeted interventions such as brief motivation workshops or one‑on‑one discussions.

To evaluate the effectiveness of the approach, the authors conducted a controlled experiment in a semester‑long C programming course with 120 students at a Korean university. The class was split into a control group (traditional peer assessment) and an experimental group (EduPCR4 with the motivation model). Over six weeks, the researchers collected four primary data streams: (1) the proportion of non‑consensus reviews, (2) internal consistency measured by Cronbach’s alpha, (3) student satisfaction surveys regarding perceived fairness, and (4) final assignment grades. The results were striking: non‑consensus dropped from 23 % in the control group to 8 % in the experimental group, representing a 65 % reduction. Cronbach’s alpha rose from 0.71 to 0.84, indicating stronger reliability of peer scores. Survey items on fairness improved from an average of 3.2 to 4.5 on a 5‑point Likert scale. Moreover, students whose reviews were flagged less frequently achieved higher final grades (average increase from 8.2 to 9.1 out of 10), suggesting that improved assessment fairness translated into better learning outcomes.

The discussion acknowledges both strengths and limitations. Strengths include real‑time detection, automatic feedback that promotes self‑regulation, and a reduction in instructor workload for arbitration. Limitations involve the need to calibrate thresholds for different domains; overly aggressive flagging could cause anxiety or reduce willingness to participate. The current implementation relies solely on score‑based metrics, ignoring richer code‑quality indicators such as cyclomatic complexity, test coverage, or static analysis warnings. The authors propose future work that integrates machine‑learning‑based anomaly detection to adapt thresholds dynamically, and that fuses code‑quality metrics with reviewer behavior to create a multi‑dimensional reliability model. Long‑term studies across multiple semesters and disciplines are also planned to assess the durability of motivation gains.

In conclusion, the motivation model presented in this paper offers a practical, data‑driven solution to enhance the fairness and reliability of peer assessment in programming education. By embedding quantitative monitoring within a familiar review platform, it empowers both students and instructors to identify and correct biased evaluations promptly, ultimately fostering a more trustworthy collaborative learning environment.