FlyTrap: Physical Distance-Pulling Attack Towards Camera-based Autonomous Target Tracking Systems

Autonomous Target Tracking (ATT) systems, especially ATT drones, are widely used in applications such as surveillance, border control, and law enforcement, while also being misused in stalking and destructive actions. Thus, the security of ATT is highly critical for real-world applications. Under the scope, we present a new type of attack: distance-pulling attacks (DPA) and a systematic study of it, which exploits vulnerabilities in ATT systems to dangerously reduce tracking distances, leading to drone capturing, increased susceptibility to sensor attacks, or even physical collisions. To achieve these goals, we present FlyTrap, a novel physical-world attack framework that employs an adversarial umbrella as a deployable and domain-specific attack vector. FlyTrap is specifically designed to meet key desired objectives in attacking ATT drones: physical deployability, closed-loop effectiveness, and spatial-temporal consistency. Through novel progressive distance-pulling strategy and controllable spatial-temporal consistency designs, FlyTrap manipulates ATT drones in real-world setups to achieve significant system-level impacts. Our evaluations include new datasets, metrics, and closed-loop experiments on real-world white-box and even commercial ATT drones, including DJI and HoverAir. Results demonstrate FlyTrap’s ability to reduce tracking distances within the range to be captured, sensor attacked, or even directly crashed, highlighting urgent security risks and practical implications for the safe deployment of ATT systems.

💡 Research Summary

The paper introduces a novel physical‑world attack against camera‑based autonomous target‑tracking (ATT) drones, called a distance‑pulling attack (DPA). Unlike prior work that only tries to make the tracker lose the target, DPA deliberately shortens the distance between the drone and the tracked object, enabling three concrete threats: (A1) the drone can be captured (e.g., with a net gun), (A2) the drone is brought into the effective range of other sensor‑based attacks, and (A3) the drone can crash into the target.

To realize DPA, the authors design “FlyTrap”, a framework that uses an “adversarial umbrella” as the attack vector. The umbrella is a regular‑looking umbrella printed with an optimized adversarial pattern. When the target carries or holds the umbrella, the pattern interferes with the deep‑learning single‑object tracking (SOT) model that the drone relies on. Specifically, the pattern causes the SOT to output a smaller bounding box for the target, which the drone’s 2‑D distance‑control algorithm interprets as the target moving away. Consequently, the drone moves forward, pulling itself closer to the target.

FlyTrap satisfies three design goals: (i) physical deployability – the umbrella is lightweight, inconspicuous, and robust to outdoor lighting; (ii) closed‑loop effectiveness – a progressive distance‑pulling strategy gradually shrinks the bounding box each frame, ensuring the drone continuously moves closer; (iii) spatial‑temporal consistency – the pattern is optimized to change minimally across consecutive frames, evading recent consistency‑checking defenses. The optimization combines a distance‑pulling loss, a temporal‑smoothness loss, and physical constraints (printer resolution, color gamut).

Evaluation proceeds in four parts. First, in a white‑box setting with a custom ATT drone, the authors achieve 100 % success in pulling the drone within capture range. Second, they fabricate real umbrellas and test them on two full‑stack ATT drones, demonstrating reductions of up to 5 m and successful capture or crash. Third, they assess transferability to three commercial drones (DJI Mini 4 Pro, DJI NEO, HoverAir X1) in a black‑box scenario; the attack still reduces distance by an average of 3.2 m, with a 78 % transfer rate across open‑source SOT models. Fourth, a user study with 200 participants shows that 92 % perceive the adversarial umbrella as a normal umbrella, confirming its stealth.

The paper highlights that DPA attacks exploit a fundamental vulnerability: the tight coupling between visual tracking output and distance‑control logic. By manipulating the visual cue, an attacker can directly control the drone’s motion, a capability not addressed by existing defenses that focus on preventing loss of tracking or on static physical barriers. The authors discuss limitations, including reliance on 2‑D bounding‑box control, reduced effectiveness against multi‑sensor (stereo, LiDAR) systems, and potential degradation of the printed pattern under weather. They suggest future work on dynamic patterns (e‑ink), defenses based on multi‑modal verification, and robust real‑time consistency checks.

Overall, FlyTrap demonstrates that a low‑cost, easily concealable physical object can cause severe, system‑level impacts on ATT drones, underscoring the urgent need to redesign distance‑control pipelines and incorporate stronger physical‑world security measures.