Out-of-Distribution Detection from Small Training Sets using Bayesian Neural Network Classifiers

Out-of-Distribution (OOD) detection is critical to AI reliability and safety, yet in many practical settings, only a limited amount of training data is available. Bayesian Neural Networks (BNNs) are a promising class of model on which to base OOD detection, because they explicitly represent epistemic (i.e. model) uncertainty. In the small training data regime, BNNs are especially valuable because they can incorporate prior model information. We introduce a new family of Bayesian posthoc OOD scores based on expected logit vectors, and compare 5 Bayesian and 4 deterministic posthoc OOD scores. Experiments on MNIST and CIFAR-10 In-Distributions, with 5000 training samples or less, show that the Bayesian methods outperform corresponding deterministic methods.

💡 Research Summary

**
Out‑of‑Distribution (OOD) detection is a cornerstone of reliable and safe AI, yet most existing works assume abundant labeled data for training. In many real‑world domains—medical imaging, industrial defect inspection, fraud detection, remote sensing—only a few thousand labeled examples are available, which makes OOD detection especially challenging because conventional deep classifiers tend to over‑fit and produce over‑confident predictions on unseen data. This paper investigates whether Bayesian Neural Networks (BNNs), which explicitly model epistemic uncertainty through a posterior distribution over weights, can alleviate this problem and provide superior OOD detection when training data are scarce.

The authors first review four widely used deterministic post‑hoc OOD scores: Softmax Entropy (SE), Maximum Logit (ML), k‑Nearest‑Neighbour distance in logit space (k‑NN), and a newly introduced class‑conditioned k‑NN (kNN+). The deterministic scores operate on a single forward pass of a standard neural network, using either the softmax probabilities or the raw logit vector. The paper then extends each of these scores to a Bayesian setting by replacing the point‑estimate logits with the expected logit vector (\hat{z}(x)=\mathbb{E}_{q(\omega)}