Improved repeatability measures for evaluating performance of feature detectors

The most frequently employed measure for performance characterisation of local feature detectors is repeatability, but it has been observed that this does not necessarily mirror actual performance. Presented are improved repeatability formulations which correlate much better with the true performance of feature detectors. Comparative results for several state-of-the-art feature detectors are presented using these measures; it is found that Hessian-based detectors are generally superior at identifying features when images are subject to various geometric and photometric transformations.

💡 Research Summary

The paper addresses a long‑standing issue in the evaluation of local feature detectors: the traditional repeatability measure, while widely used, often fails to reflect the true matching performance of a detector under realistic image transformations. The authors first critique the classic definition, which counts a feature as “repeated” if its detected location in a second image falls within a fixed pixel tolerance of the projected location from the first image. This approach ignores the scale and orientation of the feature, treats all detections as point‑like rather than region‑like, and does not penalize false positives that survive the matching stage. Consequently, detectors that score high on this metric can still produce poor precision/recall when descriptors are used for actual correspondence.

To remedy these shortcomings, the authors propose two complementary extensions. The first, overlap‑based repeatability, models each keypoint as a circular (or elliptical) region whose radius is proportional to the detector’s scale estimate. After applying the known geometric transformation (homography or affine matrix) to the region, the overlap ratio between the transformed region and the region detected in the second image is computed. If the ratio exceeds a threshold (e.g., 0.6), the keypoint is considered repeated. The second extension, scale‑orientation consistency, checks that the relative scale change and rotation angle between the two detections lie within predefined tolerances (typically 20 % for scale and 15° for rotation). Only keypoints satisfying both criteria contribute to the final composite repeatability score, which is the product of the overlap indicator and the consistency indicator, yielding a value in the range