AISSISTANT: Human-AI Collaborative Review and Perspective Research Workflows in Data Science

High-quality scientific review and perspective papers require substantial time and effort, limiting researchers’ ability to synthesize emerging knowledge. While Large Language Models (LLMs) leverage AI Scientists for scientific workflows, existing frameworks focus primarily on autonomous workflows with very limited human intervention. We introduce AIssistant, the first open-source agentic framework for Human–AI collaborative generation of scientific perspectives and review research in data science. AIssistant employs specialized LLM-driven agents augmented with external scholarly tools and allows human intervention throughout the workflow. The framework consists of two main multi-agent systems: Research Workflow with seven agents and a Paper Writing Workflow with eight agents. We conducted a comprehensive evaluation with both human expert reviewers and LLM-based assessment following NeurIPS standards. Our experiments show that OpenAI o1 achieves the highest quality scores on chain-of-thought prompting with augmented Literature Search tools. We also conducted a Human–AI interaction survey with results showing a 65.7% time savings. We believe that our work establishes a baseline for Human–AI collaborative scientific workflow for review and perspective research in data science, demonstrating that agent-augmented pipelines substantially reduce effort while maintaining research integrity through strategic human oversight.

💡 Research Summary

The paper introduces AIssistant, an open‑source, agentic framework designed to enable human‑AI collaborative generation of scientific review and perspective papers in data science. Unlike prior AI Scientist systems that emphasize fully autonomous pipelines, AIssistant embeds a Human‑in‑the‑Loop (HITL) architecture throughout both a Research Workflow (seven specialized agents) and a Paper‑Writing Workflow (eight specialized agents). Each agent is defined as a function that takes user input, a system prompt, a set of external tools, and the current set of assets (intermediate outputs) to produce new outputs and updated assets. This formalism allows transparent tracking of contributions and easy insertion of human feedback at any stage.

The Research Workflow proceeds through Ideation, Research Question formulation, Related Literature retrieval, Method design, Implementation, Result generation, and Analysis. The Literature Retrieval agent leverages external scholarly APIs such as Semantic Scholar and ORKG ASK, presenting the human user with selectable check‑boxes to curate relevant papers, thereby reducing hallucinations and improving citation relevance. The Paper‑Writing Workflow sequentially generates Title, Abstract, Introduction, Related Work, Method & Implementation, Results & Discussion, Conclusion, and finally a LaTeX‑Refine agent that applies chain‑of‑thought prompting to enforce coherence, logical flow, and formatting while preserving a separate bibliography to avoid fabricated references.

The authors conducted a comprehensive evaluation on 48 generated papers (24 review, 24 perspective) created by two PhD‑level data‑science researchers using AIssistant. They tested two LLM back‑ends—OpenAI o1 and GPT‑4o‑mini—under three prompting regimes (Zero‑Shot, Few‑Shot, Chain‑of‑Thought) and with/without literature‑search tools. Human experts and an LLM reviewer (GPT‑5) assessed each manuscript according to NeurIPS‑style criteria: clarity, originality, technical soundness, significance, reproducibility, limitations, and ethical considerations. Results show that OpenAI o1 paired with chain‑of‑thought prompting and active literature‑search tools achieved the highest weighted average scores (2.79–2.82), outperforming both GPT‑4o‑mini and the same model without tool assistance. GPT‑4o‑mini demonstrated markedly lower cost (≈ $0.002 per paper) but also lower quality, highlighting a trade‑off between expense and performance.

A human‑computer interaction survey reported an average 65.7 % reduction in time spent compared with a fully manual process. Participants noted that the framework allowed them to focus on high‑level creative reasoning while the agents handled repetitive drafting, literature curation, and formatting tasks. Hardware tests on two CPU‑only configurations (i7‑165H and i5‑1145G7) showed negligible performance differences, confirming that the system does not require GPU acceleration.

In summary, AIssistant establishes a baseline for Human‑AI collaborative scientific writing in data science. By combining specialized LLM agents, external scholarly tools, and systematic human oversight, it achieves substantial efficiency gains without sacrificing research integrity. The framework’s modular design, cost‑effective operation, and strong empirical validation suggest it could become a standard component of future scholarly workflow automation.