Adversarial Wear and Tear: Exploiting Natural Damage for Generating Physical-World Adversarial Examples

The presence of adversarial examples in the physical world poses significant challenges to the deployment of Deep Neural Networks in safety-critical applications such as autonomous driving. Most existing methods for crafting physical-world adversarial examples are ad-hoc, relying on temporary modifications like shadows, laser beams, or stickers that are tailored to specific scenarios. In this paper, we introduce a new class of physical-world adversarial examples, AdvWT, which draws inspiration from the naturally occurring phenomenon of wear and tear', an inherent property of physical objects. Unlike manually crafted perturbations, wear and tear’ emerges organically over time due to environmental degradation, as seen in the gradual deterioration of outdoor signboards. To achieve this, AdvWT follows a two-step approach. First, a GAN-based, unsupervised image-to-image translation network is employed to model these naturally occurring damages, particularly in the context of outdoor signboards. The translation network encodes the characteristics of damaged signs into a latent `damage style code’. In the second step, we introduce adversarial perturbations into the style code, strategically optimizing its transformation process. This manipulation subtly alters the damage style representation, guiding the network to generate adversarial images where the appearance of damages remains perceptually realistic, while simultaneously ensuring their effectiveness in misleading neural networks. Through comprehensive experiments on two traffic sign datasets, we show that AdvWT effectively misleads DNNs in both digital and physical domains. AdvWT achieves an effective attack success rate, greater robustness, and a more natural appearance compared to existing physical-world adversarial examples. Additionally, integrating AdvWT into training enhances a model’s generalizability to real-world damaged signs.

💡 Research Summary

**
The paper introduces AdvWT (Adversarial Wear and Tear), a novel physical‑world adversarial attack that exploits naturally occurring degradation on outdoor traffic signs. Unlike prior physical attacks that rely on artificial perturbations such as laser beams, shadows, or conspicuous stickers, AdvWT leverages the gradual, persistent, and diverse patterns of wear and tear—cracks, fading paint, rust, dirt accumulation—to create adversarial examples that are visually indistinguishable from genuine damage.

The method proceeds in two stages. First, a StarGAN‑v2 based unsupervised image‑to‑image translation framework learns a latent “damage style” space from paired clean and naturally damaged sign images. The architecture comprises a style encoder, a noise‑to‑style mapping network, a generator that injects style via Adaptive Instance Normalization, and a discriminator. Training optimizes a weighted sum of seven losses: style reconstruction, diversity sensitivity, cycle‑consistency, adversarial, domain‑disentanglement, KL‑regularization, and content preservation. This yields high‑fidelity translations that preserve the semantic identity of the sign while realistically simulating degradation.

In the second stage, the learned damage style code is perturbed with a small adversarial offset δ. In a white‑box setting, δ is optimized using gradient‑based PGD‑like updates on the target classifier’s loss; in black‑box scenarios, surrogate models or random search are employed to find effective δ. The final adversarial image x_adv = G(x, s_d + δ) retains the appearance of natural wear yet causes the classifier to misclassify.

Experiments on the German Traffic Sign Recognition Benchmark (GTSRB) and the Belgian Traffic Sign Dataset (BTSD) demonstrate strong performance. In digital simulations, AdvWT achieves over 85 % attack success and reduces top‑1 accuracy by more than 20 %. In physical tests—printing the generated signs, mounting them on a vehicle, and capturing images under varying illumination, distance, and viewing angles—success rates exceed 71 %, outperforming prior physical attacks (e.g., AdvLaser, AdvShadow, AdvCam). Human perceptual studies confirm that AdvWT’s outputs are significantly more natural‑looking than existing methods. Transferability tests show that adversarial examples crafted against one model (ResNet‑50) succeed against other architectures (MobileNet‑V2, EfficientNet‑B0) with >60 % success.

Beyond attack evaluation, the authors explore a defensive angle: augmenting training data with AdvWT‑generated damaged signs improves recognition accuracy on real‑world damaged signs by roughly 4 %, indicating that exposure to realistic degradation can enhance model robustness.

Key contributions are: (1) introducing natural wear and tear as a stealthy, persistent adversarial vector; (2) formulating damage‑aware style learning and latent‑space adversarial manipulation; (3) delivering a high‑quality GAN‑based damage simulator; (4) achieving high attack success and robustness in both digital and physical domains; and (5) demonstrating that the same synthetic damage can be leveraged for adversarial training, thereby strengthening defenses.

AdvWT highlights a new class of long‑lasting, hard‑to‑detect threats in safety‑critical systems such as autonomous driving, and underscores the importance of incorporating realistic environmental degradation into both robustness testing and defense design. Future work may extend the approach to other object categories (e.g., vehicle bodies, building facades) and develop real‑time detection or mitigation strategies for naturally degrading adversarial cues.