Quantifying the Reliability of Predictions in Detection Transformers: Object-Level Calibration and Image-Level Uncertainty

DETR and its variants have emerged as promising architectures for object detection, offering an end-to-end prediction pipeline. In practice, however, DETRs generate hundreds of predictions that far outnumber the actual objects present in an image. This raises a critical question: which of these predictions could be trusted? Addressing this concern, we provide empirical and theoretical evidence that predictions within the same image play distinct roles, resulting in varying reliability levels. Our analysis reveals that DETRs employ an optimal specialist strategy: one prediction per object is trained to be well-calibrated, while the remaining predictions are trained to suppress their foreground confidence to near zero, even when maintaining accurate localization. We show that this strategy emerges as the loss-minimizing solution to the Hungarian matching algorithm, fundamentally shaping DETRs’ outputs. While selecting the well-calibrated predictions is ideal, they are unidentifiable at inference time. This means that any post-processing algorithm poses a risk of outputting a set of predictions with mixed calibration levels. Therefore, practical deployment necessitates a joint evaluation of both the model’s calibration quality and the effectiveness of the post-processing algorithm. However, we demonstrate that existing metrics like average precision and expected calibration error are inadequate for this task. To address this issue, we further introduce Object-level Calibration Error (OCE): This object-centric design penalizes both retaining suppressed predictions and missed ground truth foreground objects, making OCE suitable for both evaluating models and identifying reliable prediction subsets. Finally, we present a post hoc uncertainty quantification framework that predicts per-image model accuracy.

💡 Research Summary

The paper tackles a fundamental reliability problem of Detection Transformers (DETR) and their variants: they output a fixed, large set of predictions (often hundreds) per image, far exceeding the number of true objects, and it is unclear which predictions can be trusted. The authors first provide both empirical observations and a theoretical analysis showing that the Hungarian matching loss, which is central to DETR training, induces an optimal “specialist strategy.” Under this strategy, for each ground‑truth object the model learns a single primary prediction that is both accurate in localization and well‑calibrated in its class confidence, while all remaining secondary predictions are forced to suppress their foreground confidence to near zero, even though they may still produce reasonably accurate bounding boxes. This behavior is mathematically derived as the loss‑minimizing solution to the Hungarian assignment problem, where background‑matched predictions are heavily penalized for high foreground scores.

Because primary predictions are indistinguishable from secondary ones at inference time, any post‑processing step (e.g., confidence thresholding, top‑k selection, NMS) risks mixing well‑calibrated and poorly calibrated outputs. Existing evaluation metrics fail to capture this nuance: Average Precision (AP) rewards retaining many predictions, inflating performance despite many suppressed secondary predictions, while Expected Calibration Error (ECE) can be artificially lowered by discarding low‑confidence predictions, ignoring missed objects entirely.

To address these shortcomings, the authors introduce Object‑level Calibration Error (OCE). OCE aggregates predictions per ground‑truth object rather than per prediction. For each object it identifies the highest‑confidence matched prediction (the primary) and measures the absolute difference between its confidence and its true accuracy (precision·IoU). Simultaneously, it penalizes any secondary predictions that remain above a tiny confidence threshold, thereby accounting for both missed objects and retained suppressed predictions. OCE thus serves as a joint metric for evaluating a model together with any post‑processing algorithm.

Leveraging the insight that primary predictions tend to have markedly higher confidence than secondary ones, the paper proposes a post‑hoc image‑level uncertainty quantification (UQ) framework. After applying an OCE‑based post‑processing step, the method computes a “confidence contrast” – the difference between the average confidence of the selected positive predictions and that of the remaining negative predictions. This contrast correlates strongly with the overall detection accuracy of the image. By training a lightweight regression model (linear or shallow MLP) on this contrast (and optionally other simple statistics), the framework can predict per‑image accuracy without ground‑truth labels.

Extensive experiments on COCO and Cityscapes validate each component. The specialist strategy is observed across multiple DETR variants (DETR, Deformable‑DETR, DINO, UP‑DETR). OCE reliably distinguishes well‑calibrated models and effective post‑processing configurations, outperforming AP and D‑ECE in reflecting true model reliability. The image‑level UQ method achieves high Pearson correlation (≈0.78) and low root‑mean‑square error (≈4 % of AP) across in‑distribution, near‑OOD (e.g., weather or illumination shifts), and far‑OOD (different domains) scenarios.

In summary, the paper makes three key contributions: (1) a theoretical and empirical characterization of the optimal prediction strategy induced by the Hungarian loss in DETR, (2) the Object‑level Calibration Error metric that jointly evaluates model calibration and post‑processing, and (3) a simple yet effective confidence‑contrast based framework for per‑image uncertainty estimation. These contributions provide practical tools for deploying DETR‑based detectors in safety‑critical or human‑in‑the‑loop applications where understanding and quantifying prediction reliability is essential.