Dynamic Deception: When Pedestrians Team Up to Fool Autonomous Cars

Many adversarial attacks on autonomous-driving perception models fail to cause system-level failures once deployed in a full driving stack. The main reason for such ineffectiveness is that once deployed in a system (e.g., within a simulator), attacks tend to be spatially or temporally short-lived, due to the vehicle’s dynamics, hence rarely influencing the vehicle behaviour. In this paper, we address both limitations by introducing a system-level attack in which multiple dynamic elements (e.g., two pedestrians) carry adversarial patches (e.g., on cloths) and jointly amplify their effect through coordination and motion. We evaluate our attacks in the CARLA simulator using a state-of-the-art autonomous driving agent. At the system level, single-pedestrian attacks fail in all runs (out of 10), while dynamic collusion by two pedestrians induces full vehicle stops in up to 50% of runs, with static collusion yielding no successful attack at all. These results show that system-level failures arise only when adversarial signals persist over time and are amplified through coordinated actors, exposing a gap between model-level robustness and end-to-end safety.

💡 Research Summary

The paper “Dynamic Deception: When Pedestrians Team Up to Fool Autonomous Cars” addresses a critical gap between model‑level adversarial attacks and system‑level failures in autonomous driving. While many prior works demonstrate that perception networks can be fooled by carefully crafted perturbations, those attacks often do not translate into unsafe vehicle behavior once the perception output is fused over time, filtered, and fed into a control stack. The authors propose a novel system‑level threat model that leverages multiple dynamic actors—specifically two pedestrians wearing adversarial patches on their clothing—to create a persistent, amplified adversarial signal.

The core idea is two‑fold. First, because a single T‑shirt can only host a small patch, the attack’s visual impact is limited. To overcome this, the authors design a “collusion” strategy: two pedestrians each wear a partial pattern that, from the ego‑vehicle’s camera viewpoint, aligns to form a larger effective patch. The base image for the patch is a red camellia flower, chosen for its color similarity to a stop sign, and an adversarial stop‑sign perturbation is embedded within it. This design maintains stealthiness for human observers while being interpreted by the vehicle’s object detector as a stop sign.

Second, the authors emphasize dynamics. Static collusion (pedestrians standing still) only yields a brief, transient detection that the downstream control logic can ignore. By contrast, the pedestrians walk in front of the vehicle, maintaining the aligned patch within the field of view for an extended period. This sustained exposure forces the perception module to repeatedly output a high‑confidence stop‑sign detection, which the vehicle’s decision‑making layer ultimately treats as a legitimate traffic sign, triggering a full stop.

Experiments are conducted in the CARLA simulator using a state‑of‑the‑art autonomous driving agent (Transfuser++). Four scenarios are evaluated: single static, single dynamic, multi‑static collusion, and multi‑dynamic collusion, each repeated ten times. Results show that single‑pedestrian attacks (both static and dynamic) and multi‑static collusion never cause the vehicle to stop. In contrast, multi‑dynamic collusion succeeds in 2–5 out of 10 runs, demonstrating that persistence and coordinated motion are essential for turning a model‑level misclassification into a system‑level safety breach.

The paper’s contributions are: (1) introducing a stealthy system‑level attack that uses pedestrians as carriers of adversarial patches; (2) proposing collusive patches to overcome size constraints; (3) showing that dynamic motion of the carriers dramatically increases attack efficacy; and (4) providing extensive simulation‑based validation against a modern autonomous driving stack.

These findings have important implications for defense. Existing defenses largely focus on detecting anomalous static objects or single‑frame perturbations. The demonstrated attack exploits temporal continuity and multi‑object coordination, which are not well‑covered by current detection or mitigation strategies. Future work should therefore explore defenses that monitor long‑term consistency across frames, analyze inter‑object relationships, and detect suspicious coordinated behavior among dynamic agents. Additionally, real‑world validation—examining how printed patches behave under varying lighting, weather, and viewpoint conditions—will be crucial to assess the practical risk of such attacks.