Towards Continuous Intelligence Growth: Self-Training, Continual Learning, and Dual-Scale Memory in SuperIntelliAgent

We introduce SuperIntelliAgent, an agentic learning framework that couples a trainable small diffusion model (the learner) with a frozen large language model (the verifier) to enable continual intelligence growth through self-supervised interaction. Unlike conventional supervised fine-tuning, SuperIntelliAgent learns autonomously without annotation: the learner generates candidate outputs, the verifier evaluates them through step-by-step reasoning, and their interaction produces chosen/rejected pairs for Direct Preference Optimization (DPO). This converts each input into a pseudo-training signal for continual improvement. The framework integrates dual-scale memory: short-term in-context memory that preserves reasoning traces across refinement cycles, and long-term memory that consolidates acquired knowledge through lightweight on-the-fly fine-tuning. A replay buffer retains samples that show verifiable progress and replays them as auxiliary supervision, reinforcing recent learning while forming adaptive curricula. SuperIntelliAgent is infrastructure-agnostic and can be plugged into existing agentic frameworks while turning ordinary inference loops into a lifelong optimization process. We posit that pairing a trainable learner with a reasoning-capable verifier forms a minimal reliable unit of growing intelligence, as paired feedback and partial-history replay yield richer learning curricula and stronger preference alignment. With a small number of automatically generated DPO pairs, the learner improves across all benchmarks, indicating that this mechanism provides a promising direction for continual intelligence accumulation and real-world deployment.

💡 Research Summary

SuperIntelliAgent proposes a minimal yet powerful architecture for lifelong learning in autonomous agents. The system pairs a lightweight, trainable diffusion model (the learner) with a frozen large language model (the verifier). For each input, the learner generates multiple candidate outputs. The verifier, equipped with chain‑of‑thought reasoning, evaluates each candidate step‑by‑step and produces a binary preference: “chosen” or “rejected.” These preference pairs are fed directly into a Direct Preference Optimization (DPO) loss, instantly updating the learner’s parameters. Consequently, every inference step becomes a self‑generated training signal, eliminating the need for human‑annotated data.

Two complementary memory mechanisms enable both short‑term continuity and long‑term knowledge accumulation. Short‑term, in‑context memory stores the verifier’s reasoning traces and injects them into subsequent prompts, preserving the logical flow across refinement cycles. This allows the verifier to reason about its own previous judgments, creating a coherent curriculum for the learner. Long‑term memory consolidates verified progress by performing lightweight on‑the‑fly fine‑tuning of the learner on selected samples. The authors adopt a meta‑learning style update that makes incremental adjustments as new data arrive, ensuring that knowledge is accumulated rather than overwritten.

A replay buffer further stabilizes learning. Only samples that demonstrate verifiable improvement—measured by the verifier’s score margin, reasoning depth, and consistency—are stored. During later training iterations these samples are replayed as auxiliary supervision, reinforcing recent gains while providing an adaptive curriculum that naturally balances novelty and reinforcement. The buffer’s selection criteria are designed to avoid catastrophic forgetting and to prioritize high‑impact learning signals.

Empirical evaluation shows that even with a few dozen automatically generated DPO pairs, the learner improves across a suite of NLP, code‑generation, and reasoning benchmarks. Compared to conventional fine‑tuning pipelines, SuperIntelliAgent exhibits a monotonic performance increase over time, confirming that the learner‑verifier loop can continuously harvest useful training data from its own operation. The framework is deliberately infrastructure‑agnostic: it can be plugged into existing agentic systems such as LangChain, AutoGPT, or ReAct without altering their external APIs. By simply wrapping the inference loop with the learner‑verifier interaction, developers obtain a lifelong optimization process for free.

The paper highlights two broader research implications. First, a frozen, reasoning‑capable LLM can serve as a reliable teacher, providing high‑quality feedback without any human labeling, thereby enabling truly self‑supervised growth. Second, the combination of short‑term in‑context memory and long‑term fine‑tuning demonstrates that agents can simultaneously maintain reasoning continuity and accumulate durable knowledge. Future work may explore extending the verifier to multimodal domains, optimizing memory management for large‑scale deployments, and testing the architecture in more complex environments such as robotics or simulation. In sum, SuperIntelliAgent establishes the learner‑verifier pair as a minimal reliable unit for continuous intelligence expansion, offering a promising pathway toward scalable, real‑world AI agents that improve autonomously over their operational lifetime.