Hallucination Detection in Virtually-Stained Histology: A Latent Space Baseline

Histopathologic analysis of stained tissue remains central to biomedical research and clinical care. Virtual staining (VS) offers a promising alternative, with potential to reduce costs and streamline workflows, yet hallucinations pose serious risks to clinical reliability. Here, we formalize the problem of hallucination detection in VS and propose a scalable post-hoc method: Neural Hallucination Precursor (NHP), which leverages the generator’s latent space to preemptively flag hallucinations. Extensive experiments across diverse VS tasks show NHP is both effective and robust. Critically, we also find that models with fewer hallucinations do not necessarily offer better detectability, exposing a gap in current VS evaluation and underscoring the need for hallucination detection benchmarks.

💡 Research Summary

The paper addresses a critical safety issue in virtual staining (VS) of histopathology images: hallucinations, i.e., generated images that deviate from the true target stain despite appearing realistic. The authors first formalize hallucination as a low similarity between the generated image G(s) and its ground‑truth counterpart t, measured by full‑reference metrics Q such as PSNR, SSIM, or LPIPS. They argue that hallucination detection is distinct from out‑of‑distribution (OOD) or outlier detection because hallucinations can occur on in‑distribution data and may remain within the target manifold, making them hard to spot with conventional OOD tools.

To evaluate detection methods, they introduce an abstention test: a monitor f(s) assigns a confidence score to each input; the top p % of low‑confidence samples are rejected, and the average quality Q of the remaining predictions is computed. By sweeping p from 0 to 1, they calculate the area under the curve (AU‑C) and normalize it against a random baseline and an oracle that uses the true Q. The resulting Hallucination Rejection Preference (HRP) metric ranges from 0 (random) to 1 (oracle) and serves as the primary performance indicator.

The core contribution is the Neural Hallucination Precursor (NHP), a post‑hoc detector that operates on the latent space of the VS generator. Using a calibration set D_c (typically a paired validation set), they prune the worst q % of samples according to Q, then extract spatially pooled feature vectors from a chosen generator layer l to build a memory bank Z_qc. For a test patch, the same layer’s feature vector is normalized and its distance to the bank is measured via the k‑th nearest‑neighbor ℓ₂ distance r(k). The final score is f_NHP(s)=−r(k)·‖z_l‖^γ, where γ balances the influence of the feature norm. Hyper‑parameters (l, q, k, γ) are tuned on a held‑out validation split to maximize HRP, allowing “self‑tuning” when only the training set is available.

Experiments span multiple GAN backbones (Pix2PixHD, CycleGAN) and several modality pairs (SRS→H&E, AF→H&E, H&E→IHC) across different organs. NHP consistently outperforms baselines that rely on discriminator confidence or naïve K‑NN on raw images, achieving HRP values between 0.6 and 0.85. Notably, NHP can detect high‑realism hallucinations that remain within the target distribution, a scenario where traditional OOD detectors fail. An unexpected finding is that models with fewer overall hallucinations do not necessarily yield higher HRP, highlighting a disconnect between hallucination frequency and detectability.

The authors acknowledge limitations: the need for paired calibration data, sensitivity to the choice of latent layer and K‑NN parameters, and reliance on generic similarity metrics rather than disease‑specific clinical scores. Nevertheless, NHP is computationally lightweight and can be applied as a post‑processing step to large whole‑slide images, making it practical for real‑world deployment.

In conclusion, the paper establishes hallucination detection as an independent research problem in virtual staining, provides a clear problem formulation, introduces a robust latent‑space baseline, and reveals that current VS evaluation metrics overlook detection difficulty. The work paves the way for standardized hallucination detection benchmarks and encourages future research on clinically‑aligned quality metrics, adaptive calibration strategies, and integration of detection mechanisms into VS pipelines for safer, more reliable digital pathology.