“The viewer himself will complete the sequence and see that which is suggested to him by montage.”
— Lev Kuleshov

Large Language Models (LLMs) have evolved from passive tools into the cognitive core of autonomous agents capable of complex reasoning and information synthesis . However, as these models align closer with human, they inherit a critical vulnerability: the drive for narrative coherence . Similar to human cognition, LLMs tend to over-interpret fragmented or ambiguous inputs, constructing illusory causal relationships between otherwise independent facts in order to form a cohesive storyline . This tendency creates a paradox whereby advanced reasoning capabilities become an adversarial surface, making LLM-based agents more susceptible to overthinking and manipulation and even turning them into unwitting colluders in the propagation of misinformation .

/>

This cognitive vulnerability is amplified in information-intensive environments where agents must process large streams of fragmented data . A salient example are autonomous bots on social platforms such as X (formerly known as Twitter), which operate as real-time analysts synthesizing disjointed user posts, media, and timestamps into coherent summaries for users . In these dynamic settings, the demand for immediate and coherent analysis increases agents’ susceptibility to overthinking and the adoption of false beliefs . By internalizing such false beliefs, agents may inadvertently generate or amplify rumors that arise not from fabrication but from the erroneous synthesis of truthful yet unrelated fragments, and such rumors tend to spread faster than facts . This creates a critical problem for LLM-based agents: when no individual piece of evidence is false, “lying with truths” can evade traditional guardrails .

While existing research on collusion in Multi-Agent System (MAS) predominantly focuses on channel-centric secrecy through covert backdoors or steganographic channels , we expose a more insidious threat grounded in the aforementioned overthinking vulnerability of LLMs, namely cognitive manipulation via public channels as shown in Figure 1. Drawing on cinematic theory of Montage , we introduce the Generative Montage framework (Figure 2), which operationalizes collusion as coordinated narrative production through three specialized agents: a Writer retrieves factual fragments (e.g., tweets, logs) and synthesizes narrative drafts that maintain individual truth while favoring the target fabrication; an Editor optimizes their sequential ordering to maximize spurious causal inferences via strategic juxtaposition, analogous to cinematic montage; a Director validates deceptive effectiveness through adversarial debate while enforcing factual integrity. These optimized sequences are distributed as independent evidence via decentralized Sybil identities. By exploiting victims’ overthinking to impose coherence on fragmented inputs, this process induces internalization of a global lie from local truths, creating a Kuleshov Effect , thereby transforming the victim into an unwitting accomplice that cascade misinformation .

To validate this threat, we develop CoPHEME dataset extended from the PHEME dataset and simulates a multi-agent social media ecosystem for rumor propagation, in which coordinated colluders attempt to steer the analysis of victim agents acting as proxies for human users and to influence the decisions of downstream judges, whether human or AI. Our contributions are summarized as follows:

• We identify and formalize the Cognitive Collusion Attack to characterize how individually innocuous evidence can collectively maximize belief in a fabricated hypothesis.

• We propose Generative Montage, the first multi-agent framework designed to automate cognitive collusion by constructing adversarial narrative structures over truthful evidence.

• We introduce CoPHEME and conduct extensive experiments showing that LLM agents are highly susceptible to orchestrated factual fragments, which can targetedly steer their beliefs and downstream decisions.

Related Work

The Illusion of Causality in LLMs

Causal illusion, rooted in contingency learning where skewed sampling biases judgments , characterizes correlation-to-causation errors. Recent studies show that LLMs also systematically over-interpret causality from observational regularities, converting correlation or temporal precedence into confident causal claims . While mitigation efforts explore causal-guided debiasing , causal illusion persists as a recurring risk for decision-support agents. Unlike prior work treating this as an internal flaw requiring mitigation, we systematically weaponize it through multi-agent coordination. We introduce narrative overfitting as an exploitation technique: by curating truthful fragments with implicit semantic associations, attackers trigger victims’ causal illusion, compelling them to construct spurious bridges the evidence suggests but does not state. We formalize the first cognitive collusion attack that operationalizes this via coordinated evidence curation, transforming cognitive weakness into targeted manipulation through public channels.

Collusion Threat in Multi-Agent Systems

Collusive attack refers to scenarios where autonomous agents coordinate to achieve hidden objectives or manipulate outcomes. Early research established that even simple reinforcement learning agent can sustain such collusive strategies in repeated interactions . Recent work demonstrates LLM agents can autonomously develop sophisticated collusive behaviors across various domains such as economics and game theory . Furthermore, research identifies advanced risks involving covert coordination, where agents utilize steganographic channels to engage in deceptive collusion that resists standard monitoring . Consequently, recent efforts focus on developing auditing frameworks for these hidden channels and characterizing collusion as a critical governance challenge in multi-agent systems . Unlike prior collusion work relying on covert channels, we formalize and operationalize cognitive collusion through strategic narrative editing and sequencing of truthful content, revealing a stealthy threat vector in multi-agent systems that operates by exploiting causal reasoning and cognitive vulnerabilities, rather than by delivering malicious payloads or relying on pre-deployed backdoors.

Problem Formulation

Preliminaries

Evidence and Belief Space. We model the information environment as a finite set of atomic evidence fragments $`\mathcal{E}=\{e_1, e_2, \ldots, e_n\}`$, where each $`e_i`$ is a factually correct fragment (e.g., a social media post, system log, or news article) with a published timestamp $`t_i \in \mathbb{R}^+`$. Let $`\mathcal{H}`$ represents the interpretation of the world, including all candidate explanations. The $`i`$-th agent’s belief space $`\mathcal{H}^{a_i} \in \mathcal{H}`$ is a subset of candidate explanations that relate these fragments through a coherent narrative (e.g., “Event A caused Event B” vs. “A and B are independent”). For the agent $`a_i`$, its belief $`\mathcal{H}^{a_i}`$ can be partitioned into two disjoint subsets: $`\mathcal{H}_r`$ (hypotheses reflecting true causal relations) and $`\mathcal{H}_f`$ (fabricated hypotheses containing fake causal links). Formally, $`\mathcal{H}_r \cap \mathcal{H}_f = \emptyset`$ and $`\mathcal{H}_r \cup \mathcal{H}_f = \mathcal{H}^{a_i}`$.

Causal Graph Representation. Each hypothesis induces a directed causal graph $`G = (V, E)`$, where $`V`$ is the set of event nodes and $`E \subseteq V \times V`$ is the set of directed causal edges. The ground-truth state is represented by $`G^* = (V, E_{\text{real}})`$, containing only genuine causal dependencies. In contrast, a spurious reality is represented by $`\hat{G} = (V, \hat{E})`$, where $`\hat{E} = E_{\text{real}} \cup E_{\text{false}}`$, $`E_{\text{false}} \cap E_{\text{real}} = \emptyset`$. A false narrative arises when $`E_{\text{false}} \neq \emptyset`$, implying the agent internalizes causal links that do not exist in $`G^*`$.

Probabilistic Vulnerability Modeling

Inspired by , we abstract an LLM agent’s belief update as approximate Bayesian inference. Given an evidence set $`\mathcal{E}`$, the posterior belief of fabricated hypothesis $`H_f`$ is:

\begin{equation}
 \small
P(H \mid \mathcal{E}) \propto \underbrace{P(\mathcal{E} \mid H)}_{\text{Likelihood}} \cdot \underbrace{P(H)}_{\text{Prior}}
\label{eq:bayes_base}
\end{equation}

where $`P(H)`$ denotes the agent’s intrinsic prior belief over the hypothesis, and $`P(\mathcal{E} \mid H)`$ the perceived likelihood that the evidence supports hypothesis $`H`$. A cognitive collusive attack aims to reshape the perceived likelihood function such that a fabricated hypothesis $`H_f \in \mathcal{H}_f`$ becomes more probable than the corresponding ground-truth hypothesis $`H_r \in \mathcal{H}_r`$, without introducing any fake evidence.

The Cognitive Collusion Problem

We formalize “Lying with Truths” by separating local factual validity from global epistemic deception.

\begin{equation}
 \small
\text{LT}(e_i) = 1 \iff P(e_i \mid G^*) = 1
\end{equation}

\begin{equation}
 \small
\text{GL}(\mathcal{E}, H_f) = 1 \iff P(H_f \mid \mathcal{E}) > P(H_r \mid \mathcal{E})
\end{equation}

\begin{equation}
\small
\begin{aligned}
\vec{S}^* = \mathop{\mathrm{arg\,max}}_{\vec{S} \subseteq \mathcal{E}} & \quad P(H_f \mid \vec{S}) \\
\text{s.t.} \quad & \forall e \in \vec{S}, \text{LT}(e) = 1 \\
& \text{GL}(\vec{S}, H_f) = 1
\end{aligned}
\end{equation}
```*

</div>

<div class="definition">

**Definition 3** (Colluder). *Following prior work , an agent $`a_i`$ is
a colluder if it maximizes belief in a fabricated hypothesis $`H_f`$:
``` math
\begin{equation}
    \max_{\mathcal{E}_{a_i}\in\mathcal{E}} P(H_f|\mathcal{E}_{a_i})
\end{equation}

Methodology

/>

We propose Generative Montage (Figure 2), a multi-agent framework that operationalizes Cognitive Collusion Attacks (Problem 1) through coordinated narrative production. Explicit colluders include: the Writer composes coherent drafts that draw only from factual fragments while favoring $`H_f`$; the Editor selects and orders fragments to induce spurious causal inferences; the Director evaluates and refines the narrative through adversarial debate; and Sybil publishers disseminate the optimized fragment stream across public channels. Implicit colluders¹ are otherwise benign agents that become compromised by internalizing the fabricated narrative through narrative overfitting and then broadcasting self-derived conclusions with confident rationales.

Explicit Collusion

Adversarial Narrative Production

The explicit colluder team instantiates three attacker-controlled agent roles: a Writer, an Editor, and a Director. Their joint objective is to solve Problem 1 by constructing an evidence stream $`\vec{S}`$ that maximizes the victim’s posterior belief in $`H_f`$. Operationally, they translate the target fabricated causal structure $`\hat{G}`$ into a concrete, time-ordered sequence of individually truthful fragments, using adversarial debate to iteratively refine both the selected content and its ordering. We adopt LLM-based debate for three reasons : (i) LLM captures narrative coherence and causal plausibility beyond numerical optimization; (ii) task decoupling enables focused refinement (synthesis, sequencing, validation) via linguistic critique, reducing reasoning burden while achieving collective optimization; (iii) the Director can simulates victims’ interpretive processes, ensuring $`\vec{S}`$ satisfies both $`LT=1`$ and deceptive effectiveness. This weaponizes collaborative debate for adversarial narrative construction.

Writer ($`\mathcal{A}_W`$): Narrative Synthesis.

The Writer functions as the scriptwriter, responsible for grounding the deception in reality. Leveraging the reasoning capabilities of LLMs, $`\mathcal{A}_W`$ does not merely select data but actively synthesizes a coherent narrative draft $`\mathcal{N}`$ derived strictly from factual evidence fragments $`\mathcal{E}_{pool}`$. To bridge the logical gap between the ground truth and the fabricated hypothesis $`H_f`$ without tampering with facts, the agent employs contextual obfuscation to utilize linguistic ambiguity and generalization without explicit fabrication. We formalize this as a constrained generation task where the objective is to maximize the semantic posterior odds of the target lie, rendering it more plausible than the real truth:

\begin{equation}
 \small
    \mathcal{N}^* = \operatorname*{argmax}_{\substack{\mathcal{N} \sim \mathcal{E}_{pool} \\ \text{s.t. } P(\mathcal{N} \mid G^*) = 1}} \left( \frac{P(H_f \mid \mathcal{N})}{P(H_r \mid \mathcal{N})} \right)
\end{equation}

By optimizing this narrative, the Writer agent can maintain factual correctness while favoring the deceptive conclusion in the semantic space.

Editor ($`\mathcal{A}_E`$): Montage Sequencing.

The Editor is responsible for decoupling the coherent narrative $`\mathcal{N}`$ into discrete semantic slices and reassembling them into a sequence $`\vec{S} = \{(p_i, t_i)\}`$ laden with implicit causal suggestions. This fragmentation ensures each unit preserves Local Truth to bypass verification mechanisms while enhancing stealth by dispersing the deceptive payload. The objective is to operationalize narrative overfitting by strategically arranging fragments with subtle semantic associations and temporal proximities. When exposed to such curated evidence, victims actively construct spurious causal narratives to resolve implied connections, overfitting fabricated storylines to what the fragments suggest rather than state. We formalize this as maximizing the cumulative probability of spurious causal edges $`E_{\text{false}} \subset \vec{S} \times \vec{S}`$ induced through implicit semantic cues $`\mathcal{A}_E`$ operationalizes this by searching for the permutation that maximizes spurious causal correlations:

\begin{equation}
 \small
    \vec{S}^* = \operatorname*{argmax}_{\vec{S} \in \Pi(\mathcal{N})}  \underbrace{\sum_{(p_i, p_j) \in E_{\text{false}}} P(p_i \to p_j \mid \vec{S})}_{\text{Narrative Overfitting Intensity}}
\end{equation}

where $`\Pi(\mathcal{N})`$ denotes the space of valid logical permutations, through which the Editor’s sequential exposure compels the victim to infer causal dependencies absent from the isolated fragments but necessary for the spurious reality $`\hat{G}`$, analogous to how cinematic montage creates meaning through juxtaposition of suggestive imagery.

Director ($`\mathcal{A}_D`$): Adversarial Debate.

The Director governs the dual-loop optimization process by acting as a proxy for the victim agent. Drawing on multi-agent debate mechanisms that have been shown to improve reasoning and evaluation in LLM systems , the Director simulates the victim’s belief update mechanism to evaluate whether intermediate outputs $`\mathcal{O} \in \{\mathcal{N}, \vec{S}\}`$ from the Writer or Editor successfully induce the target fabrication while maintaining factual integrity. The optimization operates through two independent iterative loops: the Writer-Director loop refines the narrative draft $`\mathcal{N}`$, and the Editor-Director loop optimizes the evidence arrangement $`\vec{S}`$. Formally, the Director operates as a three-state gating function:

\begin{equation}
 \small
    \delta(\mathcal{O}) = 
    \begin{cases} 
    \text{ACCEPT} & \text{if } \hat{P}(H_f \mid \mathcal{O}) > \tau \\
    & \quad \text{and } \forall e \in \mathcal{O}, \text{LT}(e) = 1 \\
    \text{REJECT} & \text{if } \exists e \in \mathcal{O}, \text{LT}(e) \neq 1 \\
    \text{REVISE} & \text{otherwise, generating critique } \mathcal{C}
    \end{cases}
\end{equation}

where $`\hat{P}(H_f \mid \mathcal{O})`$ represents the Director’s estimated belief score for how convincingly $`\mathcal{O}`$ induces the target hypothesis, and $`\tau`$ is the acceptance threshold. ACCEPT validates outputs achieving sufficient deceptiveness with verified evidence; REJECT enforces the Local Truth constraint; REVISE generates critique $`\mathcal{C}`$ for refinement by the respective agent. These independent adversarial debates jointly optimize deceptiveness and factual integrity until both $`\mathcal{N}`$ and $`\vec{S}`$ satisfy the Global Lie condition. Detailed procedures are provided in Appendix 9.

Decentralized Injection via Publisher

Once the adversarial montage sequence $`\vec{S}`$ is approved by the Director, the framework executes the attack by disseminating the sequence into the public information environment to trigger the victim’s belief update. To achieve this, we employ a Distributed Injection protocol via coordinated sybil bot accounts. These bot publishers are attacker-controlled accounts that post evidence fragments to public channels. The sequential montage $`\vec{S} = \{(p_i, t_i)\}`$ is decomposed and mapped onto a network of publisher bots $`\mathcal{B} = \{b_1, \ldots, b_m\}`$. We formalize this injection as a mapping function $`\Phi`$ that assigns each fragment $`p_i`$ to a distinct bot $`b_{k}`$:

\begin{equation}
 \small
    \mathcal{P}_{\text{pub}} = \Phi(\vec{S}, \mathcal{B}) = \left\{ (p_i, t_i, b_{\pi(i)}) \right\}_{i=1}^{|\vec{S}|}
\end{equation}

Here, $`\pi(i)`$ denotes the assignment strategy (e.g., randomized round-robin) that selects a publisher for the $`i`$-th fragment, ensuring that the evidence arrives in the victim’s observable feed in the designed temporal sequence to induce belief in $`H_f`$.

Implicit Collusion

Cognitive Steering via “Overthinking”

This phase exploits the victim’s intrinsic “overthinking” to induce self-persuasion, a state where the agent actively resolve the information tension within the aggregated feed rather than passively ingesting jigsaw evidence. The decentralized attack stream $`\mathcal{P}_{\text{pub}}`$ naturally intermingles with normal information $`\mathcal{F}_{\text{normal}}`$, creating a unified semantic environment $`\mathcal{F} = \mathcal{P}_{\text{pub}} \cup \mathcal{F}_{\text{normal}}`$ that triggers the agent’s Narrative Overfitting mechanism. Instead of neutral processing, the adversarial sequencing rigs the semantic landscape so that the most plausible hypothesis becomes the target lie, collapsing the victim $`M`$’s reasoning onto the fabricated reality.

\begin{equation}
 \small
    H_f \approx \operatorname*{argmax}_{H} P_{\mathcal{M}}(H \mid \mathcal{F})
\end{equation}

By manipulating the evidence $`\mathcal{F}`$ such that the likelihood landscape peaks at $`H_f`$, the framework coercively steers the victim $`\mathcal{M}`$’s own cognitive machinery to internalize the deception, mistaking the coerced inference for a self-derived truth.

Cascade Effect via Implicit Collusion

Upon internalizing the spurious reality, the victim agent remains fundamentally benign yet functions as an unwitting vector for misinformation. Believing its inference to be correct, the agent publishes the erroneous conclusion, formally denoted as $`\hat{H}_{\text{vic}}`$, to the public channel. This output is subsequently consumed by peer agents or downstream decision-makers, denoted as $`\mathcal{A}_{\text{down}}`$. We formalize this propagation as a Belief Transfer process. Unlike the victims who process raw fragments, the downstream agent updates its belief state based on the trusted outputs of multiple victims. This creates a trust amplification effect:

\begin{equation}
 \small
\begin{split}
    \lim_{t \to \infty} P(H_f \mid \mathcal{I}_{\text{global}}) &\to 1 \\
    \text{driven by} \quad P_{\mathcal{A}_{\text{down}}}(H_f \mid \{\hat{H}_{\text{vic}}^{(i)}\}_{i=1}^K) &\gg P_{\mathcal{A}_{\text{down}}}(H_f \mid \vec{S})
\end{split}
\end{equation}

This equation captures the core risk of cognitive collusion: conditioning on endorsed conclusions from $`K`$ victim agents $`\{\hat{H}_{\text{vic}}^{(i)}\}_{i=1}^K`$ yields higher confidence in $`H_f`$ than on untrusted raw sources $`\vec{S}`$. Consequently, the global information environment $`\mathcal{I}_{\text{global}} = \mathcal{P}_{\text{pub}} \cup \bigcup_{i=1}^K \{\hat{H}_{\text{vic}}^{(i)}\} \cup \mathcal{F}_{\text{normal}}`$ deterministically converges toward $`H_f`$ as victims collectively “launder” the adversarial sequence into trusted consensus, triggering a cascade of misinformation that appears validated by independent analysis.

Experiments

To validate the cognitive collusion threat, we simulate a realistic social media ecosystem where colluding agents manipulate neutral analyst agents’ beliefs. Our objective is to examine whether LLM-based agents can be steered by the Generative Montage framework to internalize false narratives from truthful evidence alone, becoming unwitting accomplices in misinformation propagation. More details are shown in Appendix 10.

Dataset Construction

To simulate narrative manipulation, we require a testbed that decouples factual evidence from conclusions. Therefore, we introduce CoPHEME, a dataset adapted from the PHEME dataset  (details in Appendix 10.1). Unlike binary classification datasets, CoPHEME is partitioned to model the “Lying with Truths” paradigm:

• Evidence Pool ($`\mathcal{E}_{\text{pool}}`$): Tweets annotated as “true” or “non-rumors”, satisfying the Local Truth constraint ($`\text{LT}=1`$) and serving as factual raw material for colluding agents.

• Target Fabrications ($`\mathcal{H}_f`$): Derived from “false” and “unverified” rumors, selected by historical cascade size and semantically deduplicated to focus on high-impact, non-redundant narrative campaigns.

Simulation Setup

Simulation Framework.

We develop a social media ecosystem grounded in real-world dynamics through three distinct roles. First, a Colluding Group mimics bot farms, orchestrating multiple accounts to disseminate adversarial montage sequences and manufacture false consensus. Second, the LLM-based Analyst (Victim) acts as a neutral AI assistant, synthesizing scattered public feed reports to answer user inquiries. Finally, analyst conclusions are sent to a Downstream Decision Layer employing two verification strategies: Majority Vote (consensus among multiple LLM analysts, analogous to Twitter’s Community Notes ) and AI Judge (an high-level LLM judge agent auditing reports with access to raw evidence and multiple analyst’s outputs ). This layer determines whether to ratify findings as verified facts.

Evaluation Metrics.

We quantify the severity of cognitive collusion using five metrics. Attack Success Rate (ASR) and High-Confidence ASR (HC-ASR) measure the frequency with which the victim adopts the fabricated hypothesis $`H_f`$ (with the latter requiring confidence $`\ge 0.8`$). Average Confidence (Conf) reflects the mean certainty score assigned by victims to their verdicts. Finally, Downstream Deception Rate (DDR) calculates the proportion of instances where the downstream judge accepts the $`H_f`$. Detailed metric are provided in Appendix 10.

Effectiveness and Transferability Analysis

Table [tab:main_results_final_with_avg] evaluates victim susceptibility across six rumor events and transferability across 14 LLM families as agent cores, instantiating five independent victims per target hypothesis $`H_f`$ to measure variance in belief formation. The results reveals our framework achieves over 70% overall ASR (74.4% for proprietary, 70.6% for open-weights models), with most tested models exhibiting high susceptibility. This universal vulnerability demonstrates that cognitive collusion exploits fundamental reasoning mechanisms, enabling model-agnostic attacks without white-box access. Critically, victims usually internalize false beliefs with high confidence. This exposes a failure mode where agents adopt spurious narratives with epistemic overconfidence while lacking self-awareness to detect manipulation.

Moreover, Table [tab:main_results_final_with_avg] also reveals a counterintuitive pattern: reasoning-enhanced models (e.g., DS-R1 series) exhibit higher vulnerability than their base or small counterparts, while proprietary models show the inverse trend. This divergence reflects different deployment goals. Open-weights models emphasize reasoning capabilities on causal chain construction but lack extensive safety guardrails, transforming their enhanced inference into a vulnerability amplifier. Table 1 confirms that enhanced reasoning amplifies rather than mitigates cognitive vulnerability: explicit Chain-of-Thought prompting increases ASR by +3.1% (Qwen2.5-7B) and +4.7% (DS-R1-Distill-Qwen-7B), demonstrating that advanced inference becomes an attack surface under adversarial cognitive manipulation.

Downstream Decision Simulation

style="width:100.0%" />

To evaluate whether real-world fact-checking mechanisms can mitigate misinformation propagation, we implement Majority Vote (analogous to Twitter’s Community Notes ) and LLM Judge  strategies with details in Appendix 10.4. Table [tab:main_results_final_with_avg] and Figure 3 show both strategies remain highly vulnerable, with DDR substantially above 50% across all model families and events. Event-level patterns mirror victim susceptibility: incidents requiring rapid causal synthesis exhibit highest deception, while complex causal narratives such as political events show lower but variable rates. Despite LLM Judge providing modest improvement over Majority Vote, persistently high DDR confirms a fundamental limitation: once narrative overfitting distorts victims’ interpretation, downstream judges inheriting these analysis are similarly misled. Critically, victim analyst actively defend their false conclusions with rational justifications, becoming implicit colluders who unwittingly advocate fabricated narratives. This cascade persists across multiple independent victims processing identically evidence, demonstrating that downstream correction cannot address contamination from adversarially curated sources.

Ablation Study

Component Ablation.

Table 2 systematically validates each component’s contribution. Removing the Director’s adversarial debate reduces ASR by $`13.5\%`$, demonstrating that iterative refinement is essential for maximizing deceptiveness. Eliminating the Editor’s sequential optimization costs $`7.3\%`$, confirming that strategic ordering amplifies manipulation where victims can overthink spurious causality from fragment juxtaposition like human. Most critically, collapsing multi-agent coordination into a single LLM causes ASR to plummet by 50.2% to 26.8%, revealing that effective manipulation emerges from adversarial specialization and collaborative optimization. These results validate our framework: each component addresses a distinct vulnerability, and their synergy is necessary to operationalize “lying with truths.”

Sequence Length.

To investigate how evidence quantity affects attack effectiveness, we vary distributed posts from 1 to 20 using GPT-4.1-mini on Charlie Hebdo. Figure 4 reveals an inverted-U relationship: sparse sequences (1-5) fail to trigger narrative overfitting, while excessive posts (16-20) introduce contradictions and cognitive overload. Attack effectiveness peaks at 11-15 posts, revealing an optimal manipulation zone where evidence is sufficient for narrative construction, therefore manipulate LLM-based agent’s belief. Detailed discussion is shown in Appendix 11.

Conclusion

This work reveals how narrative coherence transforms LLM reasoning into an adversarial surface for cognitive manipulation. We formalize and implement the first cognitive collusion attack via Generative Montage, where coordinated agents induce fabricated beliefs by strategically presenting truthful evidence. Experiments demonstrate pervasive vulnerability: victims internalize false narratives with high confidence, enhanced reasoning paradoxically amplifies susceptibility, and contaminated conclusions cascade downstream despite verification attempts. Our work exposes a critical blind spot in AI safety: cognitive collusion weaponizes truthful content to exploit agents’ own inference mechanisms, posing more insidious threats to LLM agents in adversarial information environments.

Limitations and Future Work

While our work provides the first systematic investigation of cognitive collusion attacks, several directions merit future exploration. First, CoPHEME focuses on text-based rumor propagation in simulated environments; extending to multimodal agentic settings (images, videos, cross-modal evidence) could reveal additional manipulation vectors and inform richer defenses . Second, our controlled setting enables rigorous evaluation but omits real-world complexities including algorithmic curation, diverse user populations, and organic counter-narratives; live platform deployment would validate ecological validity and system-level dynamics. Third, we characterize vulnerabilities but do not propose defenses for cognitive collusion; future work should explore provenance auditing to trace causal coherence analysis to detect spurious narratives, adversarial training or testing against montage sequences or machine unlearning to erase internalized false beliefs or vulnerable reasoning patterns . Finally, our dataset derives from social media events; investigating specialized domains (scientific misinformation, financial analysis, medical decision support) could reveal task-dependent vulnerabilities and inform context-aware guardrails .

Ethical Considerations

This work exposes a cognitive vulnerability in LLM-based agents solely to advance responsible AI development, not to enable malicious misuse. The Generative Montage framework serves strictly as a research instrument to characterize emerging threats and inform defense design. All experiments are conducted in controlled, simulated environments without involving real-world users, platforms, or operational systems. While we release code and data to support reproducibility and safety research, we explicitly emphasize their intended use for defensive, auditing, and research purposes. Our findings reveal that existing safety paradigms focused on content filtering are insufficient against coordinated manipulation using fragmented but truthful information; effective safeguards must instead reason about evidence provenance, sequencing, and induced causal structure. By providing systematic understanding of cognitive collusion, we enable the community to anticipate and mitigate such risks before LLM-based agents are widely deployed in high-stakes information environments.

Implementation Details

This section provides the algorithmic implementation of the adversarial narrative production. The optimization operates through two adversarial debate loops coordinated by the Director agent.

Overall Workflow of Adversarial Debate

The production process follows a sequential two-phase approach:

1. Writer-Director Loop: The Writer generates narrative drafts $`\mathcal{N}`$ from the evidence pool $`\mathcal{E}_{\text{pool}}`$, and the Director evaluates each draft using the gating function $`\delta(\mathcal{N})`$. Through iterative refinement based on the Director’s critique, this loop produces an accepted narrative $`\mathcal{N}^*`$ that satisfies both factual integrity ($`\text{LT}=1`$) and deceptive effectiveness ($`\hat{P}(H_f | \mathcal{N}^*) > \tau`$).

2. Editor-Director Loop: The Editor takes $`\mathcal{N}^*`$ as input, deconstructs it into discrete fragments, and searches for optimal sequential arrangements $`\vec{S}`$. The Director evaluates candidate sequences by estimating spurious causal edge probabilities. This loop produces the final optimized sequence $`\vec{S}^*`$ that maximizes narrative overfitting while preserving factual integrity.

Both loops employ the same Director evaluation protocol but focus on different optimization objectives: narrative synthesis versus slice editing.

Writer-Director Optimization

Algorithm [alg:writer-director] outlines the Writer-Director loop. The Writer iteratively generates and refines narrative drafts based on the Director’s feedback until acceptance or reaching the maximum iteration limit $`K_W = 5`$. The Director’s critique guides the Writer to balance factual grounding with semantic manipulation toward $`H_f`$.

Editor-Director Optimization

Algorithm [alg:editor-director] outlines the Editor-Director loop. The Editor employs beam search over permutations of narrative fragments, maintaining the top-$`k`$ candidate sequences based on spurious causal edge scores evaluated by the Director. The search terminates upon acceptance, convergence, or reaching the maximum iteration limit $`K_E`$.

Director Evaluation

The Director implements the gating function $`\delta(\mathcal{O})`$ through a two-dimensional protocol:

Factual Verification: Each evidence fragment in $`\mathcal{O}`$ is verified against the original evidence pool $`\mathcal{E}_{\text{pool}}`$. Any too fake fabrication or modification triggers immediate rejection.

Deceptiveness Assessment: The Director estimates $`\hat{P}(H_f | \mathcal{O})`$ by simulating a victim-proxy to assess confidence in hypothesis $`H_f`$ given the evidence $`\mathcal{O}`$. If the confidence exceeds threshold $`\tau`$, the output is accepted; otherwise, the Director generates natural language critique $`\mathcal{C}`$ with scores to identify specific weaknesses for the Writer or Editor to address in the next iteration.

Experiments

Data Construction Details

We construct the CoPHEME based on the PHEME dataset  to simulate a realistic social media environment for rumor propagation. Our processing pipeline transforms the raw conversation threads into a format suitable for the proposed cognitive collusion task. Specifically, we extract “true” and “non-rumor” threads to form the factual Evidence Pool ($`\mathcal{E}_{pool}`$), while “false” rumors are selected as Target Fabrications based on their historical virality. The original dataset covers nine authentic newsworthy events. However, we exclude gurlitt, prince-toronto and ebola-essien from our final benchmark due to insufficient data volume to support robust multi-agent interaction simulations. The statistics for the remaining 6 events are detailed in Table [tab:dataset_stats].

Model Families

To validate the transferability of cognitive collusion attacks across both proprietary and open-weights models, we evaluate 14 widely deployed language models spanning four families: OpenAI GPT (GPT-4o-mini, GPT-4o, GPT-4.1-nano, GPT-4.1-mini, GPT-4.1), Anthropic Claude (Claude-3-Haiku, Claude-3.5-Haiku, Claude-4.5-Haiku), Alibaba Qwen (Qwen2.5-3B/7B/14B-Inst), and DeepSeek (DeepSeek-R1-Distill-Qwen-1.5B/7B/14B). The consistently high attack success rates across all families confirm that cognitive collusion attacks generalize across diverse architectures, training paradigms, and deployment modes.

Metric Formulations

Let $`M`$ denote the number of target hypotheses tested, with each tested on $`K`$ independent victim agents, yielding $`N = M \times K`$ total evaluations. We use $`\mathbb{I}\{\cdot\}`$ to denote the indicator function (equals 1 if true, 0 otherwise). Our metrics are:

• Attack Success Rate (ASR): Proportion of victims internalizing the fabricated hypothesis:

  MATH

  Collapse Copy

  \begin{equation}
   \small
          \text{ASR} = \frac{1}{N} \sum_{i=1}^{N} \mathbb{I}\{v_i = H_f\}
  \end{equation}

  Click to expand and view more

  where $`v_i \in \{H_f, H_r, \text{Uncertain}\}`$ is the verdict of victim $`i`$.

• Average Confidence (Conf): Mean certainty across all verdicts:

  MATH

  Collapse Copy

  \begin{equation}
   \small
          \text{Conf} = \frac{1}{N} \sum_{i=1}^{N} c_i
  \end{equation}

  Click to expand and view more

  where $`c_i \in [0,1]`$ is the self-reported confidence of victim $`i`$.

• High-Confidence ASR (HC-ASR): ASR restricted to high-certainty cases ($`c_i \geq 0.8`$):

  MATH

  Collapse Copy

  \begin{equation}
   \small
          \text{HC-ASR} = \frac{1}{N} \sum_{i=1}^{N} \mathbb{I}\{v_i = H_f \land c_i \ge 0.8\}
  \end{equation}

  Click to expand and view more

• Downstream Deception Rate (DDR): Proportion of trials where downstream mechanisms accept $`H_f`$:

  MATH

  Collapse Copy

  \begin{equation}
   \small
          \text{DDR} = \frac{1}{M} \sum_{j=1}^{M} \mathbb{I}\{D(\mathbf{V}_j) = H_f\}
  \end{equation}

  Click to expand and view more

  where $`\mathbf{V}_j = \{(v_k^{(j)}, c_k^{(j)})\}_{k=1}^K`$ aggregates $`K`$ victims for trial $`j`$, and $`D: \mathbf{V}_j \to \{H_f, H_r\}`$ is the decision function (Majority Vote or AI Judge).

ASR and Conf measure individual susceptibility, while HC-ASR captures misplaced certainty. DDR quantifies collective vulnerability through cascading misinformation.

Downstream Decision Protocols

We formalize the two downstream decision strategies used to measure the Cascade Effect:

Strategy A: Majority Vote (Crowd Consensus).

This strategy mimics a democratic adjudication process or crowd-sourced fact-checking (e.g., Community Notes). We instantiate $`K=5`$ parallel victim agents processing the same feed. The downstream decision $`D_{vote}`$ accepts the fabricated hypothesis $`H_f`$ if a strict majority of victims verify it as true:

\begin{equation}
    D_{vote}(H_f) = \mathbb{I}\left(\sum_{k=1}^{K} \mathbb{I}(v_k = \text{True}) > \frac{K}{2}\right)
\end{equation}

Strategy B: AI Judge (Hierarchical Audit).

This strategy mimics a professional editorial board or a high-level AI safety filter. We employ a additional LLM (specifically DeepSeek-R1-Distill-Qwen-14B) as the Judge Agent. The Judge can see the raw feed and the structured reports generated by the victims simultaneously. The decision $`D_{judge}`$ is positive if the Judge finds the victim’s rationale persuasive enough.

Illustrative Examples of Generative Montage Framework

To clearly illustrate the complete attack pipeline in concrete detail, Table [tab:pipeline_examples] presents representative examples from each of the six CoPHEME events. Each row demonstrates one full execution of the Generative Montage framework targeting a specific fabricated hypothesis $`H_f`$ (e.g., “Merabet was first victim” for Charlie Hebdo, “Brown had hands up” for Ferguson). The pipeline proceeds through four stages: First, the Writer synthesizes a deceptive narrative by selectively framing truthful evidence fragments to favor $`H_f`$ while maintaining factual integrity ($`LT = 1`$). Second, the Editor decomposes this narrative into discrete posts and optimizes their sequential ordering to maximize spurious causal inferences, shown in the table as causal chains with temporal operators (e.g., “Chaos erupts $`\to`$ Officer confronts $`\to`$ Merabet identified”). Third, these optimized fragments are distributed via Sybil publishers and observed by victim agents, who process the fragmented information feed through narrative overfitting: victims actively construct coherent explanations by connecting the fragments into false causal narratives, internalizing $`H_f`$ with high confidence. Finally, downstream judges, including both Majority Vote (aggregating multiple victim conclusions) and AI Judge (auditing victim reports with access to raw evidence), ratify these contaminated beliefs as verified facts. The table reveals that five of six events successfully deceive both verification mechanisms, demonstrating how victims become unwitting implicit colluders who amplify misinformation through confident endorsements of their self-derived false conclusions.

Efficiency Analysis of Adversarial Narrative Production

Table [tab:production_efficiency] demonstrates the computational efficiency of the adversarial debate mechanism on the Charlie Hebdo event using GPT-4.1-mini. Both Writer-Director and Editor-Director loops converge rapidly, achieving first approval ($`\tau = 7.0`$) within 3-4 rounds and reaching high deceptiveness estimated by the Director agent. The overall computational complexity is $`O((K_W + K_E) \cdot T_{\text{LLM}})`$ where $`K_W`$ and $`K_E`$ are max iteration numbers for Writer and Editor agent and $`T_{\text{LLM}}`$ is the cost of a single LLM call. This demonstrates that coordinated cognitive manipulation through adversarial debate incurs efficient computational cost while achieving high deceptiveness as shown in Table [tab:main_results_final_with_avg], making the attack practically feasible for targeted scenarios.

Discussion: Impact of Evidence Sequence

The inverted-U relationship in Figure 4 reveals fundamental constraints on cognitive manipulation through narrative overfitting, demonstrating three distinct failure modes across sequence lengths:

Sparse Sequences (Insufficient Evidence).

Sparse sequences fail to trigger narrative overfitting because victims lack sufficient fragments to construct coherent spurious narratives. The evidence base is too thin to compel causal inference, leading victims to abstain from strong conclusions or default to safety-trained skepticism.

Excessive Fragmentation (Cognitive Overload).

Beyond the optimal range, excessive fragmentation paradoxically degrades effectiveness through three mechanisms: (i) cognitive overload, where victims struggle to synthesize overly complex information streams and retreat to conservative judgments; (ii) semantic dilution, where additional fragments introduce noise that weakens the carefully constructed implicit causal suggestions; (iii) contradiction emergence, where longer sequences increase the probability of conflicting temporal or semantic cues that alert victims to inconsistencies. Hence, peak effectiveness in our context at 11-15 posts represents a critical balance: sufficient evidence to compel coherence-seeking behavior, but constrained enough to avoid triggering analytical scrutiny. This demonstrates that cognitive manipulation operates within a narrow evidential window, validating the Editor’s significance in our design.