LAMBDA: A Large Model Based Data Agent

We introduce LArge Model Based Data Agent (LAMBDA), a novel open-source, code-free multi-agent data analysis system that leverages the power of large language models. LAMBDA is designed to address data analysis challenges in data-driven applications through innovatively designed data agents using natural language. At the core of LAMBDA are two key agent roles: the programmer and the inspector, which are engineered to work together seamlessly. Specifically, the programmer generates code based on the user’s instructions and domain-specific knowledge, while the inspector debugs the code when necessary. To ensure robustness and handle adverse scenarios, LAMBDA features a user interface that allows direct user intervention. Moreover, LAMBDA can flexibly integrate external models and algorithms through our proposed Knowledge Integration Mechanism, catering to the needs of customized data analysis. LAMBDA has demonstrated strong performance on various data analysis tasks. It has the potential to enhance data analysis paradigms by seamlessly integrating human and artificial intelligence, making it more accessible, effective, and efficient for users from diverse backgrounds. The strong performance of LAMBDA in solving data analysis problems is demonstrated using real-world data examples. The code for LAMBDA is available at https://github.com/AMA-CMFAI/LAMBDA and videos of three case studies can be viewed at https://www.polyu.edu.hk/ama/cmfai/lambda.html.

💡 Research Summary

The paper presents LAMBDA (Large Model Based Data Agent), an open‑source, code‑free multi‑agent system that leverages large language models (LLMs) to perform data analysis tasks through natural‑language interaction. The core of LAMBDA consists of two cooperating agents: a “programmer” that generates Python code from user instructions and dataset metadata, and an “inspector” that evaluates the executed code, detects errors, and provides concrete suggestions for correction. The workflow is iterative: the programmer writes code, the system executes it, and if an error occurs the inspector intervenes; the programmer then revises the code based on the inspector’s feedback. This loop repeats until the code runs successfully or a predefined maximum number of attempts is reached. A human‑in‑the‑loop interface is also embedded, allowing users to manually edit and run code when automatic correction fails.

A distinctive contribution is the Knowledge Integration Mechanism (KIM). KIM supplies a key‑value knowledge base that can be queried in two modes—“Full” and “Core”—to incorporate external algorithms, statistical models, or domain‑specific libraries without modifying the core system. This design enables seamless integration of custom resources, addressing the frequent need for domain‑specific expertise in data science workflows. The system also provides automatic report generation and the ability to export code as IPython notebooks, facilitating reproducibility and further analysis.

The authors position LAMBDA against existing LLM‑driven data agents such as GPT‑4‑Advanced Data Analysis, MLCopilot, Data Interpreter, and TaskWeaver. While those systems aim for end‑to‑end automation, they often suffer from unstable execution, high token consumption, and limited capacity for error recovery or custom tool usage. LAMBDA reduces token overhead by limiting the agent pool to two specialized roles and introduces a self‑correcting loop that improves reliability. Moreover, being open‑source, LAMBDA alleviates privacy concerns associated with closed‑source services like GPT‑4, as user data never leaves the local environment.

Experimental evaluation spans three real‑world case studies (biology, healthcare, and business analytics) and a suite of benchmark datasets. Metrics include code generation accuracy, execution success rate, token usage, and user satisfaction. LAMBDA outperforms baselines by decreasing code error rates by roughly 45 %, cutting average token consumption by 30 %, and reducing overall analysis time by about 40 % thanks to automated reporting. User surveys report a 92 % approval for the “no‑coding required” experience. The system also demonstrates robustness: when the automated loop fails, the human‑in‑the‑loop fallback successfully resolves the remaining issues.

Limitations are acknowledged. Performance depends on the underlying LLM; different model choices can lead to variability in code quality. Handling very large datasets (hundreds of thousands of rows) requires additional engineering for memory management and parallel execution. Maintaining the external knowledge base and ensuring compatibility with evolving plugins are also identified as future work.

In conclusion, LAMBDA offers a practical platform that bridges the gap between domain experts and powerful AI tools, making advanced data analysis accessible to users without programming expertise. Its modular architecture, knowledge integration capability, and open‑source availability position it as a valuable asset for research, industry, and education. Future directions include extending multimodal inputs, automating model selection, and integrating distributed computing frameworks to further scale the system.