Position: The Real Barrier to LLM Agent Usability is Agentic ROI

Large Language Model (LLM) agents represent a promising shift in human-AI interaction, moving beyond passive prompt-response systems to autonomous agents capable of reasoning, planning, and goal-directed action. While LLM agents are technically capable of performing a broad range of tasks, not all of these capabilities translate into meaningful usability. This position paper argues that the central question for LLM agent usability is no longer whether a task can be automated, but whether it delivers sufficient Agentic Return on Investment (Agentic ROI). Agentic ROI reframes evaluation from raw performance to a holistic, utility-driven perspective, guiding when, where, and for whom LLM agents should be deployed. Despite widespread application in high-ROI tasks like coding and scientific research, we identify a critical usability gap in mass-market, everyday applications. To address this, we propose a zigzag developmental trajectory: first scaling up to improve information gain and time savings, then scaling down to reduce cost. We present a strategic roadmap across these phases to make LLM agents truly usable, accessible, and scalable in real-world applications.

💡 Research Summary

The paper argues that the true obstacle to the widespread usability of Large Language Model (LLM) agents is not whether a task can be automated, but whether the deployment delivers sufficient “Agentic Return on Investment” (Agentic ROI). The authors introduce Agentic ROI as a composite metric that multiplies Information Gain (the improvement in output quality over a human baseline) by Time Savings (the reduction in human hours spent on a task) and divides the product by Cost (monetary expenses such as token usage, API fees, and compute). This formulation ensures that an agent that harms quality yields zero ROI regardless of time saved, and it normalizes the value across different domains and user profiles.

To validate the metric, the authors surveyed 34 participants (14 AI practitioners, 20 end‑users) across five representative domains: coding, scientific research, office work, e‑commerce, and personal assistance. Participants rated perceived quality improvement, minutes saved per task, and overall usability on 10‑point scales; cost was estimated from subscription fees divided by assumed task volume. After normalizing the scores to a 0‑1 range, the authors computed Agentic ROI for each domain. The results show that coding and research exhibit the highest ROI, driven by large baseline human effort (high T₀) and substantial quality gains (high Q_agent). In contrast, office, e‑commerce, and personal assistance display low ROI because the baseline tasks are already highly optimized for speed, and the overhead of prompting and clarification often exceeds the time saved. A strong linear correlation (r = 0.95) between ROI and reported usability confirms that ROI is a reliable predictor of perceived usefulness.

The analysis uncovers a critical usability gap: high‑demand consumer domains have low Agentic ROI, explaining why mass‑market adoption of LLM agents lags behind specialized, high‑ROI fields. Users tolerate latency but are deterred by the cognitive load of prompt engineering, which erodes time‑saving benefits. Moreover, the authors observe that ROI is highly context‑dependent; tasks that are trivial in a normal setting (e.g., ordering coffee) can become high‑ROI when the user’s hands are occupied, suggesting that dynamic context awareness could dramatically boost ROI in everyday scenarios.

To address these findings, the paper proposes a “zigzag” development trajectory: an initial scaling‑up phase that expands model size and compute to maximize information gain and time savings, followed by a scaling‑down phase that introduces smaller, more efficient variants to cut costs while preserving most of the performance gains. This pattern mirrors the evolution of OpenAI’s model series (o1 → o3 → o4) and is presented as a roadmap for LLM agents. The roadmap recommends:

1. Focus early development on high‑ROI domains to push the frontier of quality and speed.
2. Invest in prompt‑automation tools, multimodal interaction, and lightweight model families for low‑ROI domains to reduce user effort and monetary cost.
3. Personalize the baseline quality (Q₀) and baseline time (T₀) per user, allowing the ROI metric to adapt to varying expertise, disabilities, or digital literacy.

By optimizing Agentic ROI rather than raw performance alone, the authors argue that LLM agents can become truly usable, accessible, and economically viable for the broader public. The paper concludes that a holistic, ROI‑driven perspective is essential for guiding future research, product design, and deployment strategies in the rapidly evolving field of autonomous AI agents.