Scaling Small Agents Through Strategy Auctions

Small language models are increasingly viewed as a promising, cost-effective approach to agentic AI, with proponents claiming they are sufficiently capable for agentic workflows. However, while smaller agents can closely match larger ones on simple tasks, it remains unclear how their performance scales with task complexity, when large models become necessary, and how to better leverage small agents for long-horizon workloads. In this work, we empirically show that small agents’ performance fails to scale with task complexity on deep search and coding tasks, and we introduce Strategy Auctions for Workload Efficiency (SALE), an agent framework inspired by freelancer marketplaces. In SALE, agents bid with short strategic plans, which are scored by a systematic cost-value mechanism and refined via a shared auction memory, enabling per-task routing and continual self-improvement without training a separate router or running all models to completion. Across deep search and coding tasks of varying complexity, SALE reduces reliance on the largest agent by 53%, lowers overall cost by 35%, and consistently improves upon the largest agent’s pass@1 with only a negligible overhead beyond executing the final trace. In contrast, established routers that rely on task descriptions either underperform the largest agent or fail to reduce cost – often both – underscoring their poor fit for agentic workflows. These results suggest that while small agents may be insufficient for complex workloads, they can be effectively “scaled up” through coordinated task allocation and test-time self-improvement. More broadly, they motivate a systems-level view of agentic AI in which performance gains come less from ever-larger individual models and more from market-inspired coordination mechanisms that organize heterogeneous agents into efficient, adaptive ecosystems.

💡 Research Summary

The paper investigates the limits of small language‑model‑based agents in complex, long‑horizon tasks and proposes a novel test‑time routing framework called Strategy Auctions for Workload Efficiency (SALE). Using the Qwen‑3 family (4B, 8B, 14B, 32B parameters), the authors evaluate two representative domains—deep search (requiring extensive reasoning, tool use, and information synthesis) and coding (requiring multi‑step planning, debugging, and test‑driven refinement). Task complexity is quantified by human solution time, yielding five logarithmically spaced bins from sub‑second to one‑hour tasks, and a new benchmark HST‑Bench containing 753 tasks.

Empirical results show that while the smallest 4B agent reaches about 87 % of the 32B model’s pass@1 on the easiest tasks, its relative performance collapses to roughly 21 % on the most complex tasks. This confirms that small agents can match larger ones on simple problems but fail to scale with task difficulty, suggesting that model size must be treated as a per‑task decision rather than a global replacement strategy.

To address this, SALE draws inspiration from freelance marketplaces. For each task, every candidate agent submits a concise strategic plan (the “bid”). Bids are evaluated using a cost‑value function (C – V), where C estimates the token cost of executing the plan and V predicts the expected utility (e.g., pass@1). The provisional winner is the bid minimizing C – V. Crucially, before final execution, agents cheaper than the provisional winner can refine their strategies by consulting a shared auction memory that stores past bids, outcomes, and refinements. This memory‑driven refinement can overturn the provisional ranking, effectively allowing smaller, cheaper agents to improve their chances over time—a process analogous to freelancers upskilling through repeated gigs.

SALE differs from traditional routing approaches. Non‑predictive routing runs all candidate models to completion, which is infeasible for long‑horizon agents because trace lengths can reach millions of tokens. Predictive routing trains a separate routing model, incurring additional training cost and often degrading as task difficulty rises. In contrast, SALE requires only short plan generation at routing time, incurs negligible extra inference beyond the final execution trace, and continuously adapts via auction feedback without any offline training.

Across both domains, SALE achieves a 53 % reduction in reliance on the largest 32B model and a 35 % reduction in total token cost, while improving pass@1 by +3.5 % on deep search and +2.7 % on coding. The overhead beyond the final trace is minimal. Moreover, as the auction memory grows, the selection frequency of the 4B agent rises, demonstrating that the system progressively shifts workload toward cheaper models. Established routers either underperform the largest model or fail to lower cost, underscoring their poor fit for agentic workflows where task inputs are weak predictors of downstream success.

The paper’s contributions are: (1) a systematic study of how task complexity widens the performance gap between small and large agents on realistic workloads; (2) the HST‑Bench benchmark linking human solution time to task difficulty; (3) the SALE framework that unifies per‑task routing with test‑time self‑improvement via strategic bidding and shared memory; (4) empirical evidence that SALE outperforms any single agent and existing routers on the performance‑cost Pareto frontier; and (5) a broader argument that market‑inspired coordination mechanisms, rather than ever larger individual models, may drive the next wave of agentic AI systems.

In summary, while small agents alone cannot handle highly complex tasks, the authors demonstrate that a marketplace‑style auction combined with memory‑driven plan refinement can effectively “scale up” these agents, reducing dependence on large models, cutting inference cost, and even improving accuracy. This work points toward a systems‑level perspective where heterogeneous agents cooperate through dynamic, economic mechanisms, opening a promising direction for building efficient, adaptive AI ecosystems.