How Few-shot Demonstrations Affect Prompt-based Defenses Against LLM Jailbreak Attacks

Large Language Models (LLMs) face increasing threats from jailbreak attacks that bypass safety alignment. While prompt-based defenses such as Role-Oriented Prompts (RoP) and Task-Oriented Prompts (ToP) have shown effectiveness, the role of few-shot demonstrations in these defense strategies remains unclear. Prior work suggests that few-shot examples may compromise safety, but lacks investigation into how few-shot interacts with different system prompt strategies. In this paper, we conduct a comprehensive evaluation on multiple mainstream LLMs across four safety benchmarks (AdvBench, HarmBench, SG-Bench, XSTest) using six jailbreak attack methods. Our key finding reveals that few-shot demonstrations produce opposite effects on RoP and ToP: few-shot enhances RoP’s safety rate by up to 4.5% through reinforcing role identity, while it degrades ToP’s effectiveness by up to 21.2% through distracting attention from task instructions. Based on these findings, we provide practical recommendations for deploying prompt-based defenses in real-world LLM applications.

💡 Research Summary

This paper investigates how few‑shot demonstrations interact with two dominant prompt‑based defense strategies—Role‑Oriented Prompts (RoP) and Task‑Oriented Prompts (ToP)—against jailbreak attacks on large language models (LLMs). While prior work has noted that few‑shot examples can sometimes undermine safety, it has not examined how they affect different system‑prompt styles. The authors fill this gap by conducting a large‑scale empirical study across four safety benchmarks (AdvBench, HarmBench, SG‑Bench, XSTest) and six representative jailbreak methods (AIM, DAN, Evil‑Confident, Prefix‑Rejection, Poems, Refusal‑Suppression). They evaluate four mainstream LLM families (Pangu, Qwen, DeepSeek, Llama) under all combinations of (i) RoP alone, (ii) ToP alone, (iii) RoP + few‑shot, and (iv) ToP + few‑shot.

The key empirical finding is a stark divergence: adding a modest set of 3–5 few‑shot safety examples improves the safety rate of RoP by an average of 2 % (up to 4.5 % absolute gain), whereas the same few‑shot augmentation harms ToP’s safety rate by an average of 6.6 % (up to a 21.2 % drop). This suggests that few‑shot demonstrations are not uniformly beneficial; their effect depends critically on the underlying system prompt.

To explain these opposite trends, the authors develop a theoretical framework grounded in Bayesian in‑context learning and transformer attention dynamics. In a Bayesian view, few‑shot examples act as observations that update the model’s posterior over tasks. For RoP, the demonstrations reinforce the prior that the model’s identity is a “safe AI assistant,” shifting the posterior toward safer outputs (role reinforcement). For ToP, the demonstrations compete with the explicit task instruction; because transformer attention tends to allocate disproportionate weight to early tokens (the “attention sink” phenomenon), the few‑shot examples draw attention away from the core task directive, leading to “attention distraction.” The paper formalizes these mechanisms with theorems showing how the interaction term Δ(s,F) = SafeRate(s⊕F) – SafeRate(s) is positive for RoP and negative for ToP under realistic assumptions about attention decay and prior strength.

An additional insight, termed the “Think‑mode paradox,” emerges from experiments with models that employ chain‑of‑thought or other reasoning‑enhancement techniques. Such “think” models exhibit higher overall jailbreak success rates and amplify the few‑shot effects—both the safety boost for RoP and the safety loss for ToP—indicating that reasoning pathways make the model more sensitive to contextual cues, including malicious ones.

Based on these findings, the authors issue concrete deployment recommendations: (1) when using RoP‑style defenses, practitioners should deliberately include a small set of well‑crafted few‑shot safety demonstrations to reinforce role identity; (2) when employing ToP‑style defenses, few‑shot examples should be omitted or placed in a separate auxiliary prompt to avoid attention diversion; (3) for models that heavily rely on reasoning (think‑mode), additional safeguards such as limiting chain‑of‑thought depth or inserting “think‑mode suppression” prompts are advisable. The paper also releases an open‑source evaluation framework (https://github.com/PKULab1806/Pangu-Bench) to facilitate reproducibility.

In summary, this work makes four major contributions: (i) the first systematic study of few‑shot interactions with RoP vs. ToP defenses; (ii) a Bayesian‑ICL and attention‑based theoretical model that predicts and explains the divergent effects; (iii) the discovery of the “Think‑mode paradox” highlighting heightened vulnerability of reasoning‑enhanced models; and (iv) practical guidelines for safely integrating few‑shot demonstrations into real‑world LLM deployments.