DisCoPatch: Taming Adversarially-driven Batch Statistics for Improved Out-of-Distribution Detection

Out-of-distribution (OOD) detection holds significant importance across many applications. While semantic and domain-shift OOD problems are well-studied, this work focuses on covariate shifts - subtle variations in the data distribution that can degrade machine learning performance. We hypothesize that detecting these subtle shifts can improve our understanding of in-distribution boundaries, ultimately improving OOD detection. In adversarial discriminators trained with Batch Normalization (BN), real and adversarial samples form distinct domains with unique batch statistics - a property we exploit for OOD detection. We introduce DisCoPatch, an unsupervised Adversarial Variational Autoencoder (VAE) framework that harnesses this mechanism. During inference, batches consist of patches from the same image, ensuring a consistent data distribution that allows the model to rely on batch statistics. DisCoPatch uses the VAE’s suboptimal outputs (generated and reconstructed) as negative samples to train the discriminator, thereby improving its ability to delineate the boundary between in-distribution samples and covariate shifts. By tightening this boundary, DisCoPatch achieves state-of-the-art results in public OOD detection benchmarks. The proposed model not only excels in detecting covariate shifts, achieving 95.5% AUROC on ImageNet-1K(-C) but also outperforms all prior methods on public Near-OOD (95.0%) benchmarks. With a compact model size of 25MB, it achieves high OOD detection performance at notably lower latency than existing methods, making it an efficient and practical solution for real-world OOD detection applications. The code is publicly available.

💡 Research Summary

DisCoPatch introduces a novel unsupervised framework for out‑of‑distribution (OOD) detection that leverages the intrinsic bias of Batch Normalization (BN) toward batch statistics. The authors observe that, in adversarially trained discriminators, clean (in‑distribution) and adversarial (synthetic) samples originate from two distinct domains, each characterized by its own mean and variance in BN layers. To exploit this “two‑domain hypothesis,” DisCoPatch processes images as collections of patches and constructs each training and inference batch exclusively from patches of a single image. This guarantees that the BN statistics computed during inference correspond to a single underlying distribution, allowing the discriminator to rely on these statistics for reliable OOD scoring.

The architecture combines an adversarial Variational Autoencoder (VAE) with a discriminator. The VAE is trained with the standard ELBO loss while its decoder generates two types of negative samples: (i) fully synthetic patches sampled from the latent prior, and (ii) reconstructions of real patches. Both are fed to the discriminator as OOD examples, encouraging it to separate not only high‑frequency artifacts typical of GAN outputs but also low‑frequency degradations (e.g., blur) that appear in covariate shifts. During training, the BN momentum is set to one, forcing the discriminator to use only the current batch statistics rather than running averages; the same setting is kept at test time. Consequently, only the discriminator is needed for inference, making the model lightweight (≈25 MB) and fast.

Extensive experiments on ImageNet‑1K (and its corrupted variant ImageNet‑1K‑C) and several Near‑OOD benchmarks (e.g., NINCO, Sketch) demonstrate state‑of‑the‑art performance: 95.5 % AUROC on covariate‑shift detection and 95.0 % AUROC on Near‑OOD, surpassing prior methods. The approach also yields competitive results on far‑OOD tasks while offering substantially lower latency than generative‑based baselines such as GANs, Normalizing Flows, or Diffusion Models.

Key contributions include: (1) a rigorous analysis of BN’s domain‑specific statistics and their exploitation for OOD detection, (2) the DisCoPatch framework that integrates patch‑based batch construction with adversarial VAE training, and (3) a compact, high‑performance OOD detector suitable for real‑time applications. The code is publicly released, facilitating reproducibility and practical deployment.