Antithetic Noise in Diffusion Models

We systematically study antithetic initial noise in diffusion models, discovering that pairing each noise sample with its negation consistently produces strong negative correlation. This universal phenomenon holds across datasets, model architectures, conditional and unconditional sampling, and even other generative models such as VAEs and Normalizing Flows. To explain it, we combine experiments and theory and propose a \textit{symmetry conjecture} that the learned score function is approximately affine antisymmetric (odd symmetry up to a constant shift), supported by empirical evidence. This negative correlation leads to substantially more reliable uncertainty quantification with up to $90%$ narrower confidence intervals. We demonstrate these gains on tasks including estimating pixel-wise statistics and evaluating diffusion inverse solvers. We also provide extensions with randomized quasi-Monte Carlo noise designs for uncertainty quantification, and explore additional applications of the antithetic noise design to improve image editing and generation diversity. Our framework is training-free, model-agnostic, and adds no runtime overhead. Code is available at https://github.com/jjia131/Antithetic-Noise-in-Diffusion-Models-page.

💡 Research Summary

This paper investigates a surprisingly simple yet universal property of diffusion models: pairing each random Gaussian seed z with its negation −z (antithetic sampling) yields generated samples that are strongly negatively correlated. The authors systematically evaluate this phenomenon across a wide spectrum of settings—including unconditional and conditional diffusion models, various architectures (UNet‑based DDIM, transformer‑based DiT, consistency‑distilled models), different datasets (LSUN‑Church, CelebA‑HQ, CIFAR‑10, ImageNet, LAION‑2B), and even non‑diffusion generative models such as VAEs and Normalizing Flows. In every case, the Pearson correlation between the two outputs lies between –0.5 and –0.9, far stronger than the near‑zero correlation observed for two independent seeds. Centralizing the pixel values (subtracting the mean across the pair) further amplifies the negative correlation, indicating that the effect is not merely driven by shared global structures.

To explain this, the authors propose a “symmetry conjecture”: at each denoising timestep t, the learned score network εₜ(x) is approximately affine‑antisymmetric, i.e., εₜ(x)+εₜ(–x)≈2cₜ for some constant vector cₜ. They prove a lemma showing that Corr(f(Z), f(–Z))=–1 implies f is affine‑antisymmetric at the origin, and they empirically demonstrate that the correlation of εₜ evaluated on antithetic pairs stays close to –1 throughout the diffusion trajectory. Because the deterministic DDIM update can be written as a linear combination Fₜ(x)=aₜx+bₜεₜ(x), the affine‑antisymmetry of εₜ guarantees that Fₜ(–x)≈–Fₜ(x)+2bₜcₜ, preserving the strong negative correlation at every step and culminating in highly anti‑correlated final images.

The paper leverages this property for uncertainty quantification. By treating the antithetic pair as control variates, the variance of any Monte‑Carlo estimator based on diffusion samples can be dramatically reduced. Experiments show up to a 90 % reduction in confidence‑interval width and more than a 100‑fold reduction in computational cost for tasks such as pixel‑wise statistic estimation and evaluation of diffusion‑based inverse solvers. Importantly, this improvement requires no retraining, no architectural changes, and incurs zero runtime overhead.

Beyond basic antithetic pairing, the authors explore randomized quasi‑Monte‑Carlo (RQMC) designs. Using low‑discrepancy Sobol sequences to generate antithetic seeds further reduces variance, marking the first application of QMC techniques to diffusion sampling. The combined approach also enhances image diversity: because z and –z lie on opposite poles of the high‑dimensional Gaussian sphere, the two reverse‑diffusion trajectories explore distant regions of the data manifold, yielding more varied outputs without sacrificing perceptual quality (as measured by SSIM and LPIPS).

Additional applications include plug‑and‑play improvements to image editing pipelines, where antithetic sampling stabilizes intermediate latent manipulations, and extensions to stochastic samplers (DDPM) where the effect persists, albeit with slightly weaker correlation. The authors acknowledge limitations: the symmetry conjecture remains empirically motivated without a full theoretical proof; the impact on perceptual quality under extreme conditioning (e.g., long text prompts) needs further study; and human‑subject evaluations are absent.

In summary, the work uncovers a model‑agnostic, training‑free technique that exploits the inherent antisymmetry of diffusion score networks. Antithetic noise provides a cheap, universally applicable tool for variance reduction, more reliable uncertainty quantification, and enhanced sample diversity, opening new avenues for both theoretical analysis and practical deployment of diffusion‑based generative models.