How important are Deformable Parts in the Deformable Parts Model?

The main stated contribution of the Deformable Parts Model (DPM) detector of Felzenszwalb et al. (over the Histogram-of-Oriented-Gradients approach of Dalal and Triggs) is the use of deformable parts. A secondary contribution is the latent discriminative learning. Tertiary is the use of multiple components. A common belief in the vision community (including ours, before this study) is that their ordering of contributions reflects the performance of detector in practice. However, what we have experimentally found is that the ordering of importance might actually be the reverse. First, we show that by increasing the number of components, and switching the initialization step from their aspect-ratio, left-right flipping heuristics to appearance-based clustering, considerable improvement in performance is obtained. But more intriguingly, we show that with these new components, the part deformations can now be completely switched off, yet obtaining results that are almost on par with the original DPM detector. Finally, we also show initial results for using multiple components on a different problem – scene classification, suggesting that this idea might have wider applications in addition to object detection.

💡 Research Summary

The paper revisits the three canonical contributions of the Deformable Parts Model (DPM) – deformable parts, latent‑SVM discriminative learning, and multiple components – and questions the widely held belief that deformable parts are the dominant source of its performance advantage over the earlier HOG‑based detector of Dalal and Triggs. To assess the relative importance of each component, the authors conduct a series of controlled experiments on the PASCAL VOC detection benchmark. First, they replace the original aspect‑ratio and left‑right flip heuristics used to initialise the mixture components with an appearance‑based clustering procedure. By increasing the number of mixture components from the standard three to six or more, and by seeding each component with a cluster of visually similar training examples, they obtain a substantial boost in average precision (approximately 2–3 percentage points) compared with the baseline DPM. This result demonstrates that the mixture model itself, when properly initialised, captures a large portion of the variability that was previously attributed to part deformation. Next, the authors disable the deformation cost altogether for the newly learned components, effectively fixing the relative positions of all parts. Surprisingly, the detection performance remains virtually unchanged; in some cases it even improves slightly. This finding suggests that the deformation mechanism, while conceptually elegant, contributes little to the final accuracy once a sufficiently expressive mixture of components is present, and may even introduce unnecessary model complexity. The latent‑SVM learning step continues to provide a modest gain, but its impact is comparable whether or not part deformation is active, indicating that the discriminative re‑training primarily refines component‑level classifiers rather than learning sophisticated spatial flexibility. Finally, the authors explore the generality of the multi‑component idea by applying the same clustering‑based mixture strategy to a scene‑classification task. By treating each scene image as a whole and learning several appearance‑based components (e.g., indoor, outdoor, natural), they achieve a modest but consistent improvement over a single‑component baseline (about 1.5 % absolute accuracy gain). This cross‑domain experiment hints that the benefits of component‑wise modelling extend beyond object detection. In sum, the study overturns the conventional ordering of DPM contributions: the number of components and their data‑driven initialisation appear to be more critical than the deformable part mechanism itself. The authors recommend that future work focus on richer component generation, better clustering techniques, and possibly simplifying or removing the deformation module, thereby streamlining the model while preserving – or even enhancing – detection and classification performance.