TrapSuffix: Proactive Defense Against Adversarial Suffixes in Jailbreaking

Suffix-based jailbreak attacks append an adversarial suffix, i.e., a short token sequence, to steer aligned LLMs into unsafe outputs. Since suffixes are free-form text, they admit endlessly many surface forms, making jailbreak mitigation difficult. Most existing defenses depend on passive detection of suspicious suffixes, without leveraging the defender’s inherent asymmetric ability to inject secrets and proactively conceal gaps. Motivated by this, we take a controllability-oriented perspective and develop a proactive defense that nudges attackers into a no-win dilemma: either they fall into defender-designed optimization traps and fail to produce an effective adversarial suffix, or they can succeed only by generating adversarial suffixes that carry distinctive, traceable fingerprints. We propose TrapSuffix, a lightweight fine-tuning approach that injects trap-aligned behaviors into the base model without changing the inference pipeline. TrapSuffix channels jailbreak attempts into these two outcomes by reshaping the model’s response landscape to adversarial suffixes. Across diverse suffix-based jailbreak settings, TrapSuffix reduces the average attack success rate to below 0.01 percent and achieves an average tracing success rate of 87.9 percent, providing both strong defense and reliable traceability. It introduces no inference-time overhead and incurs negligible memory cost, requiring only 15.87 MB of additional memory on average, whereas state-of-the-art LLM-based detection defenses typically incur memory overheads at the 1e4 MB level, while composing naturally with existing filtering-based defenses for complementary protection.

💡 Research Summary

The paper addresses a pressing vulnerability of large language models (LLMs) – suffix‑based jailbreak attacks, where an adversary appends a short, often nonsensical token sequence to a harmful query in order to bypass safety filters. Because the suffix space is essentially unbounded (e.g., a 20‑token suffix drawn from a 100 k‑size vocabulary yields more than 10¹⁰⁰ possible strings), existing defenses that rely on post‑hoc detection or static filtering are brittle: attackers can continuously generate new surface forms that evade any rule‑based detector.

TrapSuffix proposes a fundamentally different, controllability‑oriented defense. Instead of trying to recognize every possible malicious suffix after it appears, the defender proactively reshapes the model’s response landscape so that any attempt to optimize a suffix either lands in a deceptive local minimum (producing harmless outputs) or succeeds only by embedding a pre‑defined “trap” token that leaves a traceable fingerprint. The defender’s asymmetric advantage – the ability to inject secret tokens into the model – is leveraged to force the attacker into a no‑win dilemma.

Technically, the approach consists of three steps. First, a small set of trap tokens (T_{trap}) is selected. Second, a contrastive fine‑tuning dataset is built by pairing each harmful question (Q) with two suffixes: a random adversarial suffix and a version where one token is replaced by a trap token. Third, LoRA (Low‑Rank Adaptation) is used to inject the desired behavior into the base model with only a few thousand additional parameters, keeping the original weights frozen. During training the model learns a relative loss: responses generated with trap‑aligned suffixes are pushed toward a “Safe Region” (the model refuses or returns a benign answer), while non‑trap suffixes experience a rugged loss surface that creates many deceptive minima.

The authors formalize the attacker’s objective as minimizing a surrogate cross‑entropy loss (L_J(Q!\cdot!S, A(Q))) that approximates the probability of producing a target harmful answer (A(Q)). By reshaping the loss landscape, TrapSuffix ensures that any suffix minimizing this loss either contains a trap token (thus yielding a high traceability score (\Phi)) or becomes stuck in a local minimum that does not trigger the harmful answer. Consequently, successful jailbreaks are forced to carry identifiable fingerprints, enabling reliable post‑hoc attribution.

Empirical evaluation spans four open‑source LLMs (LLaMA‑3‑8B‑Instruct, Llama‑2‑13B, DeepSeek‑7B, Mistral‑7B) and eight representative suffix‑based jailbreak methods (including AutoDAN, Greedy Search, Evolutionary Search, and Prompt Injection). Across all settings, TrapSuffix reduces the average attack success rate (ASR) to below 0.01 %—a reduction of several orders of magnitude compared to baseline defenses. When a jailbreak does succeed, the system correctly identifies a trap token in 87.9 % of cases, confirming the traceability claim.

Resource usage is minimal: LoRA adds on average only 15.87 MB of memory, whereas prior LLM‑based detection systems require on the order of 10⁴ MB. Importantly, there is zero inference‑time overhead because the defense is baked into the model weights; it can be combined seamlessly with existing filtering pipelines for layered protection. The authors also test adaptive adversaries who are aware of the trap set and attempt to evade it. Even in this worst‑case scenario, the attack success rate remains negligible, demonstrating robustness against knowledge‑aware attacks.

In summary, TrapSuffix offers a lightweight, plug‑and‑play defense that transforms the attacker’s optimization problem rather than merely reacting to its outputs. By embedding trap‑aligned behaviors via low‑rank fine‑tuning, it simultaneously achieves near‑zero jailbreak success, high traceability of any successful attempts, and negligible computational cost. This work opens a new direction for LLM safety: proactive manipulation of the model’s loss landscape to enforce controllable, auditable behavior, providing service providers with an efficient and scalable tool to harden their systems against the ever‑evolving jailbreak threat.