Towards AI as Colleagues: Multi-Agent System Improves Structured Ideation Processes

Most AI systems today are designed to manage tasks and execute predefined steps. This makes them effective for process coordination but limited in their ability to engage in joint problem-solving with humans or contribute new ideas. We introduce MultiColleagues, a multi-agent conversational system that shows how AI agents can act as colleagues by conversing with each other, sharing new ideas, and actively involving users in collaborative ideation processes. In a within-subjects study with 20 participants, we compared MultiColleagues to a single-agent baseline. Results show that MultiColleagues fostered stronger perceived social presence, and participants rated their outcomes as higher in quality and novelty, with more elaboration during ideation. These findings demonstrate the potential of AI agents to move beyond process partners toward colleagues that share intent, strengthen group dynamics, and collaborate with humans to advance ideas.

💡 Research Summary

The paper presents MultiColleagues, a multi‑agent conversational system designed to move artificial intelligence from a tool or “copilot” role toward that of a genuine collaborative colleague in structured ideation. Building on recent advances in large language models (LLMs) and multi‑agent frameworks such as AutoGen and Crew AI, the authors argue that true human‑AI teamwork requires agents that can adopt distinct personas, engage in turn‑based dialogue, and be overseen by a human facilitator.

MultiColleagues is guided by three design goals: (DG1) support fluid transitions between divergent (Explore) and convergent (Focus) phases of brainstorming; (DG2) expose users to multiple, complementary perspectives by assigning each agent a specific persona (e.g., technical expert, ethicist, UX designer); and (DG3) preserve transparent human control through an on‑demand facilitator interface. The system workflow consists of (1) persona selection and ranking, (2) dynamic turn‑selection where the next agent is chosen based on the current conversational context, (3) generation of the agent’s response, and (4) user actions (reply, continue, or call the facilitator). All interactions are logged for later analysis.

To evaluate the approach, the authors conducted a within‑subjects study with 20 participants who performed a creative design task (“trustworthy mood‑aware karaoke in autonomous vehicles”) under two conditions: MultiColleagues and a state‑of‑the‑art single‑agent baseline (essentially a ChatGPT‑style brainstorming tool). Each participant experienced both conditions in counterbalanced order, spending about 30 minutes per condition. Quantitative measures included perceived social presence, expert‑rated idea quality and novelty, and the number of elaboration turns. Qualitative data were collected via post‑session interviews.

Results show that the multi‑agent condition significantly outperformed the single‑agent baseline. Perceived social presence increased by an average of 0.6 points on a 5‑point Likert scale, indicating that participants felt the AI agents behaved more like teammates. Idea quality and novelty scores rose by 0.6 and 0.7 points respectively, and the number of elaboration turns grew by 23 %, suggesting deeper exploration. Interview excerpts highlighted that participants appreciated the “conversation among AI colleagues,” felt that “different viewpoints broadened my thinking,” and valued the ability to intervene through the facilitator when the discussion drifted.

The authors interpret these findings as evidence that (1) multi‑agent personas enhance the social dimension of human‑AI collaboration, fostering a sense of partnership rather than mere assistance; (2) exposing users to diverse perspectives mitigates the homogeneity often observed with single‑LLM interactions and leads to more novel, higher‑quality ideas; and (3) a human‑centric orchestration layer mitigates over‑automation concerns while preserving user agency. Design implications include the importance of dynamic turn‑taking to reduce cognitive load, the need for well‑crafted personas that convey expertise credibly, and the necessity of an intuitive facilitator UI that allows users to pause, redirect, or reset the dialogue at any point.

In conclusion, MultiColleagues demonstrates that AI agents can function as collaborative colleagues in structured creative tasks, delivering measurable gains in social presence, idea quality, and depth of ideation. The work opens avenues for future research on scaling the approach to larger teams, longer‑term collaborations, and cross‑domain applications, as well as exploring meta‑collaboration among AI agents themselves.