Learning with Adaptive Prototype Manifolds for Out-of-Distribution Detection

Out-of-distribution (OOD) detection is a critical task for the safe deployment of machine learning models in the real world. Existing prototype-based representation learning methods have demonstrated exceptional performance. Specifically, we identify two fundamental flaws that universally constrain these methods: the Static Homogeneity Assumption (fixed representational resources for all classes) and the Learning-Inference Disconnect (discarding rich prototype quality knowledge at inference). These flaws fundamentally limit the model’s capacity and performance. To address these issues, we propose APEX (Adaptive Prototype for eXtensive OOD Detection), a novel OOD detection framework designed via a Two-Stage Repair process to optimize the learned feature manifold. APEX introduces two key innovations to address these respective flaws: (1) an Adaptive Prototype Manifold (APM), which leverages the Minimum Description Length (MDL) principle to automatically determine the optimal prototype complexity $K_c^*$ for each class, thereby fundamentally resolving prototype collision; and (2) a Posterior-Aware OOD Scoring (PAOS) mechanism, which quantifies prototype quality (cohesion and separation) to bridge the learning-inference disconnect. Comprehensive experiments on benchmarks such as CIFAR-100 validate the superiority of our method, where APEX achieves new state-of-the-art performance.

💡 Research Summary

The paper identifies two fundamental shortcomings in current prototype‑based out‑of‑distribution (OOD) detection methods. First, the “Static Homogeneity Assumption” forces every class to share the same number and equal importance of prototypes, ignoring the intrinsic visual complexity differences among classes. This leads to prototype collisions, where prototypes of semantically similar but distinct categories (e.g., leopard vs. tiger) collapse onto the same region of the feature space, creating blind spots that degrade both classification and OOD detection. Second, the “Learning‑Inference Disconnect” discards the rich posterior information (soft assignments, prototype quality, intra‑class cohesion, inter‑class separation) learned during training, relying instead on simple confidence scores or energy functions at test time, which limits the theoretical performance ceiling.

To remedy these issues, the authors propose APEX (Adaptive Prototype for eXtensive OOD Detection), a two‑stage repair framework. In the first stage, Adaptive Prototype Manifold (APM) automatically determines an optimal prototype count K​c for each class c. Using a pretrained DINOv2 backbone, class‑wise feature vectors are clustered with Gaussian Mixture Models (GMMs) across a range of component numbers. The Bayesian Information Criterion (BIC) is applied to select the k that minimizes the description length, yielding K​c per class. This MDL‑driven allocation provides heterogeneous representational capacity, eliminating prototype collisions.

Training then proceeds with a hierarchical probabilistic model: each class conditional density is modeled as a mixture of von Mises‑Fisher (vMF) distributions parameterized by the adaptive prototypes. Soft assignments w​c​i​j between sample i and prototype j are computed via entropy‑regularized optimal transport (Sinkhorn‑Knopp), ensuring differentiable, probabilistic matching. The loss combines (1) a Maximum Likelihood Estimation term L​MLE that directly maximizes the posterior p(y|z) and (2) a Prototype Contrastive loss L​PC that pulls prototypes of the same class together while pushing different‑class prototypes apart. The total loss L​train = L​MLE + λ L​PC guides the encoder and projection head to learn a discriminative hyperspherical embedding with an adaptive manifold structure.

The second stage introduces Posterior‑Aware OOD Scoring (PAOS). Two quality metrics are defined for each prototype: Cohesion (average cosine similarity to its assigned samples) and Separation (inverse of the maximum cosine similarity to prototypes of other classes). Their sum Q(p​c​k) quantifies overall prototype quality. This quality is transformed into an energy E​q = –Q and a Gibbs distribution p​c(k) ∝ exp(Q/τ​q) over the K*​c prototypes of class c. The class‑wise confidence C​onf(c) is the negative free energy, i.e., τ​q log ∑​k exp(Q(p​c​k)/τ​q). During inference, the standard Mahalanobis distance is calibrated by this confidence:

S​PAOS(h) = min_c { (h–μ​c)ᵀ Σ⁻¹ (h–μ​c) /