DiffBreak: Is Diffusion-Based Purification Robust?

Diffusion-based purification (DBP) has become a cornerstone defense against adversarial examples (AEs), regarded as robust due to its use of diffusion models (DMs) that project AEs onto the natural data manifold. We refute this core claim, theoretically proving that gradient-based attacks effectively target the DM rather than the classifier, causing DBP’s outputs to align with adversarial distributions. This prompts a reassessment of DBP’s robustness, attributing it to two critical flaws: incorrect gradients and inappropriate evaluation protocols that test only a single random purification of the AE. We show that with proper accounting for stochasticity and resubmission risk, DBP collapses. To support this, we introduce DiffBreak, the first reliable toolkit for differentiation through DBP, eliminating gradient flaws that previously further inflated robustness estimates. We also analyze the current defense scheme used for DBP where classification relies on a single purification, pinpointing its inherent invalidity. We provide a statistically grounded majority-vote (MV) alternative that aggregates predictions across multiple purified copies, showing partial but meaningful robustness gain. We then propose a novel adaptation of an optimization method against deepfake watermarking, crafting systemic perturbations that defeat DBP even under MV, challenging DBP’s viability.

💡 Research Summary

This paper critically re‑examines diffusion‑based purification (DBP), a defense that leverages diffusion models (DMs) to “project” adversarial examples (AEs) onto the natural data manifold. The authors first expose a fundamental theoretical flaw: DBP’s robustness claim rests on the assumption that the score model sθ accurately approximates the true score of the marginal distributions at every diffusion step. Since sθ is itself a learned neural network, it is vulnerable to adversarial manipulation. The authors prove (Theorem 3.1) that standard gradient‑based adaptive attacks, when back‑propagated through the entire DBP pipeline, do not merely bypass the purifier—they implicitly target sθ, reshaping its parameters so that the reverse diffusion trajectory generates samples from an adversarial distribution. In other words, the attack optimizes the purifier itself, turning DBP from a defensive projection into an active component of the attack.

Building on this insight, the paper identifies practical issues in prior work: many implementations use checkpointing to reduce memory consumption, but this breaks the computational graph and yields incorrect gradients. To fix this, the authors introduce DiffGrad, a module that records intermediate diffusion states and reconstructs the exact backward dependencies, enabling precise gradient computation without prohibitive memory overhead. DiffGrad is packaged in the open‑source DiffBreak toolkit, which also provides utilities for various attack strategies.

The authors then critique the evaluation protocol commonly used for DBP. Because the reverse diffusion process is stochastic, measuring robustness on a single purified sample per input severely under‑estimates the true misclassification probability. They propose a statistically sound multi‑sample majority‑vote (MV) protocol: generate multiple purified copies of the same input, classify each, and take the majority label as the final decision. While MV modestly improves reported robustness, it still leaves DBP vulnerable.

To demonstrate the limits of MV, the paper adapts a low‑frequency (LF) attack originally designed for deep‑fake watermark removal. The LF attack crafts perturbations that are smooth and affect many pixels simultaneously, thereby influencing a large fraction of the stochastic diffusion paths. Experiments on CIFAR‑10 and ImageNet show that LF attacks defeat DBP even under MV, reducing effective robustness to near zero. Moreover, when the authors replace the flawed gradients of prior attacks with the correct DiffGrad gradients, strong attacks such as AutoAttack dramatically outperform earlier reported results, collapsing DBP’s claimed robustness from >70% to below 17%.

In summary, the contributions are: (1) a theoretical proof that adaptive attacks target the score model, invalidating DBP’s core robustness argument; (2) identification and correction of gradient implementation bugs via DiffGrad; (3) a rigorous MV evaluation protocol; (4) a novel LF attack that breaks DBP even under MV; and (5) the DiffBreak toolkit that integrates all these components. The findings suggest that DBP, as currently practiced, does not provide reliable defense against adaptive adversaries, and future work must either harden the score model itself or devise fundamentally different purification strategies.