AI Agents Need Memory Control Over More Context

AI agents are increasingly used in long, multi-turn workflows in both research and enterprise settings. As interactions grow, agent behavior often degrades due to loss of constraint focus, error accumulation, and memory-induced drift. This problem is especially visible in real-world deployments where context evolves, distractions are introduced, and decisions must remain consistent over time. A common practice is to equip agents with persistent memory through transcript replay or retrieval-based mechanisms. While convenient, these approaches introduce unbounded context growth and are vulnerable to noisy recall and memory poisoning, leading to unstable behavior and increased drift. In this work, we introduce the Agent Cognitive Compressor (ACC), a bio-inspired memory controller that replaces transcript replay with a bounded internal state updated online at each turn. ACC separates artifact recall from state commitment, enabling stable conditioning while preventing unverified content from becoming persistent memory. We evaluate ACC using an agent-judge-driven live evaluation framework that measures both task outcomes and memory-driven anomalies across extended interactions. Across scenarios spanning IT operations, cybersecurity response, and healthcare workflows, ACC consistently maintains bounded memory and exhibits more stable multi-turn behavior, with significantly lower hallucination and drift than transcript replay and retrieval-based agents. These results show that cognitive compression provides a practical and effective foundation for reliable memory control in long-horizon AI agents.

💡 Research Summary

The paper addresses a critical bottleneck in long‑horizon, multi‑turn AI agents: uncontrolled growth of context leading to loss of constraint focus, error accumulation, and memory‑induced drift. Traditional solutions—simply appending the entire dialogue transcript to the prompt (transcript replay) or retrieving top‑ranked documents from an external store (retrieval‑based memory)—either cause linear token growth, higher latency, and amplified early mistakes, or inject irrelevant information that can corrupt the agent’s decision‑making. Drawing inspiration from human working memory, the authors propose the Agent Cognitive Compressor (ACC), a bio‑inspired memory controller that replaces raw transcript replay with a bounded internal representation called the Compressed Cognitive State (CCS).

Core Mechanism
At each turn t, ACC receives (1) the current user input xₜ, (2) the previously committed CCS₍ₜ₋₁₎, and (3) a limited set of retrieved artifacts Aₜ from an external store M. A dedicated Cognitive Compressor Model (CCM)—either a general‑purpose LLM prompted with a strict schema S_CCS or a fine‑tuned lightweight model—compresses these inputs into a new CCSₜ that conforms to the schema. The schema explicitly defines required fields such as goals, policy constraints, entity identifiers, and confirmed decisions, ensuring that only decision‑critical information is persisted. Crucially, artifact retrieval is decoupled from state commitment: retrieved documents are merely candidates; only the CCM’s filtered, normalized output updates the persistent state. Consequently, noisy or outdated artifacts cannot directly pollute the agent’s memory.

Architectural Integration
ACC is positioned between the transient interaction signals and the downstream reasoning engine. It does not perform planning, tool selection, or action generation; it solely governs how the internal state evolves. By writing exactly one bounded state variable per turn, ACC guarantees a fixed memory footprint (typically 1–2 KB) regardless of interaction length. The authors demonstrate integration of ACC into two common agent patterns: (1) a ReAct‑style loop where reasoning, acting, and reflection occur, and (2) a planning‑centric architecture where a plan is generated once and then executed over many steps. In both cases, the downstream policy model conditions on CCS₍ₜ₋₁₎ plus the new user input, rather than on the full transcript, preserving a high signal‑to‑noise ratio as the horizon expands.

Evaluation Framework
To assess ACC, the authors built an Agent‑Judge‑Driven Live Evaluation platform. The same query sequence is presented simultaneously to three agents: (a) ACC‑enabled, (b) transcript‑replay baseline, and (c) retrieval‑based baseline. Results are blinded and order‑randomized to eliminate bias. Evaluation metrics include: (i) task outcome accuracy, (ii) memory footprint (tokens stored), (iii) hallucination rate (via claim‑audit against a canonical ground truth), and (iv) drift rate (percentage of original constraints retained). Scenarios span three operational domains—IT incident response, cybersecurity threat mitigation, and healthcare workflow management—each executed over thousands of turns (total >10,000 turns).

Results
ACC consistently outperformed the baselines across all metrics. Memory size was reduced by 70–85 % compared with transcript replay, while maintaining a stable CCS size. Hallucination rates dropped to 0.28 % (vs. 2.7 % for replay and 1.9 % for retrieval). Drift was dramatically curtailed: ACC retained 98.8 % of initial constraints, whereas replay fell to 85.4 % and retrieval to 90.1 %. Task success rates rose to 92.3 % for ACC, versus 80.5 % and 84.2 % for the other two methods. Statistical analysis confirmed the significance of these improvements, especially in long‑run interactions (>500 turns) where the gap widened.

Discussion and Limitations
The authors acknowledge that ACC’s effectiveness hinges on a well‑designed schema; extending to new domains requires schema engineering and possibly re‑training the CCM. The artifact retrieval policy R_ACC is currently a simple top‑k similarity search; future work could incorporate trust scores, provenance checks, or active learning to refine candidate selection. ACC also focuses on a single agent’s internal state; scaling to multi‑agent collaborations will demand protocols for synchronizing multiple CCS instances. Finally, while a lightweight fine‑tuned CCM reduces latency, the trade‑off between compression fidelity and computational cost remains an open research question.

Conclusion
By formalizing memory as a bounded, schema‑driven compression process, the Agent Cognitive Compressor offers a practical solution to the memory‑control problem that plagues long‑horizon AI agents. Empirical evidence across diverse, real‑world workflows demonstrates that ACC not only curtails context growth but also markedly improves reliability by suppressing hallucinations and drift. The work paves the way for more trustworthy, scalable autonomous agents and suggests future directions such as dynamic schema evolution, hierarchical CCS structures, and coordinated memory management in multi‑agent ecosystems.