Communication Enhances LLMs' Stability in Strategic Thinking

Large Language Models (LLMs) often exhibit pronounced context-dependent variability that undermines predictable multi-agent behavior in tasks requiring strategic thinking. Focusing on models that range from 7 to 9 billion parameters in size engaged in a ten-round repeated Prisoner’s Dilemma, we evaluate whether short, costless pre-play messages emulating the cheap-talk paradigm affect strategic stability. Our analysis uses simulation-level bootstrap resampling and nonparametric inference to compare cooperation trajectories fitted with LOWESS regression across both the messaging and the no-messaging condition. We demonstrate consistent reductions in trajectory noise across a majority of the model-context pairings being studied. The stabilizing effect persists across multiple prompt variants and decoding regimes, though its magnitude depends on model choice and contextual framing, with models displaying higher baseline volatility gaining the most. While communication rarely produces harmful instability, we document a few context-specific exceptions and identify the limited domains in which communication harms stability. These findings position cheap-talk style communication as a low-cost, practical tool for improving the predictability and reliability of strategic behavior in multi-agent LLM systems.

💡 Research Summary

Background and Motivation
As large language models (LLMs) are increasingly deployed as autonomous or delegated agents, their behavior in multi‑agent settings must be reliable. Prior work has identified several sources of output variability—sampling temperature, hardware nondeterminism, and especially prompt or context changes—that can cause strategic instability in games where small stochastic fluctuations lead to dramatically different outcomes. The authors frame this problem as “strategic instability” and propose a minimal‑cost intervention: cheap‑talk, i.e., cost‑free, non‑binding pre‑play messages exchanged before each decision.

Experimental Design
Four LLMs in the 7‑9 billion‑parameter range were selected (exact model names are not disclosed). Each model was evaluated across six contextual framings: five socially‑oriented frames (cooperation, competition, environmental, business, and a neutral baseline) plus an additional neutral condition. For each model‑frame pair, 100 independent simulations of a ten‑round repeated Prisoner’s Dilemma (IPD) were run, yielding roughly 200 agent‑level trajectories per condition. In the “messaging” treatment, agents exchanged a single free‑form sentence before each round; they were explicitly told that messages are costless, non‑binding, and should be interpreted cautiously. In the control condition, agents acted solely on the observed play history.

Stability Metric
For each condition, the round‑wise average cooperation rate (a 10‑point time series) was smoothed using locally weighted scatterplot smoothing (LOWESS) with a bandwidth of 0.4. The root‑mean‑square error (RMSE) between the observed averages and the LOWESS fit quantified deviation from a smooth trajectory: lower RMSE indicates higher strategic stability. To assess the effect of messaging, the authors performed a bootstrap at the simulation level (10,000 resamples), recomputing the RMSE for both treatments each time and recording the difference (no‑messaging − messaging). A 95 % confidence interval that excluded zero was taken as evidence of a significant effect. To control for multiple comparisons across model‑frame combinations, a binomial test was also applied.

Key Findings

1. Overall Effect – In 18 of the 24 model‑frame combinations (75 %), pre‑play messaging produced a statistically significant reduction in RMSE, meaning cooperation trajectories became smoother and more predictable.
2. Model‑Dependent Magnitude – The stabilizing impact was strongest for the more volatile smaller models (e.g., 7 B), while the largest 9 B models showed little or no significant change, suggesting that baseline instability drives the benefit.
3. Contextual Influence – Although the effect persisted across all prompt variants, the “cooperation‑emphasis” framing yielded the largest stability gains, whereas the “business competition” frame generated a few exceptions where messaging actually increased RMSE (notably for one 8 B model).
4. Temperature Robustness – Re‑running the entire suite with temperature set to 0 (deterministic sampling) reproduced the same pattern, indicating that the observed variability stems more from internal context sensitivity than from stochastic sampling.
5. Network Constraints – Additional experiments that simulated communication constraints (e.g., limited bandwidth) showed that the messaging benefit remained, but its magnitude continued to be driven primarily by model choice rather than by the framing alone.

Limitations and Future Work

• Horizon Length – The ten‑round horizon is relatively short; longer repeated games could reveal whether the stability boost endures over extended interactions.
• Message Content – Because messages were unrestricted free‑form sentences, the study could not quantify the informational content, truthfulness, or strategic exaggeration of the messages. Future work should manipulate message fidelity to see how “lying” or “over‑communicating” affects stability.
• Model Scope – The analysis is limited to 7‑9 B models. Extending to both much larger models (>30 B) and highly distilled lightweight models would test the generality of the cheap‑talk mechanism.
• Real‑World Constraints – Real multi‑agent deployments involve latency, token limits, and security policies. Incorporating these practical constraints will be essential before deploying cheap‑talk as a reliability layer in production systems.

Conclusion
The paper provides compelling empirical evidence that a simple, cost‑free pre‑play communication step can markedly improve the predictability of strategic behavior in LLM‑driven multi‑agent environments. The effect is most pronounced for models that are intrinsically volatile, and it holds across a variety of contextual framings and decoding regimes. By treating strategic stability as a foundational substrate rather than an after‑thought, the authors demonstrate that cheap‑talk can serve as a practical, low‑overhead tool for enhancing the reliability of future agentic AI systems.