ASA: Training-Free Representation Engineering for Tool-Calling Agents

Adapting LLM agents to domain-specific tool calling remains notably brittle under evolving interfaces. Prompt and schema engineering is easy to deploy but often fragile under distribution shift and strict parsers, while continual parameter-efficient fine-tuning improves reliability at the cost of training, maintenance, and potential forgetting. We identify a critical Lazy Agent failure mode where tool necessity is nearly perfectly decodable from mid-layer activations, yet the model remains conservative in entering tool mode, revealing a representation-behavior gap. We propose Activation Steering Adapter (ASA), a training-free, inference-time controller that performs a single-shot mid-layer intervention and targets tool domains via a router-conditioned mixture of steering vectors with a probe-guided signed gate to amplify true intent while suppressing spurious triggers. On MTU-Bench with Qwen2.5-1.5B, ASA improves strict tool-use F1 from 0.18 to 0.50 while reducing the false positive rate from 0.15 to 0.05, using only about 20KB of portable assets and no weight updates.

💡 Research Summary

The paper tackles a practical problem in large language model (LLM) agents: reliably invoking external tools when required, despite frequent changes in tool sets, API signatures, and interaction protocols. Existing approaches fall into two camps. Prompt‑and‑schema engineering is easy to deploy but fragile under distribution shift and strict parsers, while parameter‑efficient fine‑tuning (e.g., LoRA, QLoRA) improves in‑domain success but incurs ongoing training, deployment, and forgetting costs. The authors identify a “Lazy Agent” failure mode: linear probes on mid‑layer activations (layer ≈ 18) can decode tool‑necessity with >99 % AUC, yet the model’s generation conservatively stays in plain‑text mode in >80 % of those cases. This reveals a representation‑behavior gap—latent intent exists in the hidden state but does not cross the discrete decision threshold required for a tool call.

To bridge this gap, they propose the Activation Steering Adapter (ASA), a training‑free, inference‑time controller that intervenes once on a selected hidden layer. ASA consists of three lightweight components:

1. Steering vectors – a global intent direction (µ_pos − µ_neg) and per‑domain offsets, computed from class‑conditional means of the hidden state and L2‑normalized.
2. Router – a tiny linear classifier that maps standardized hidden representations to a domain label, enabling domain‑specific steering.
3. Probe – a per‑domain sigmoid regressor that estimates the probability p(x) that a tool call is appropriate.

During inference, the router selects a domain ˆd, the probe yields p(x), and a mixture‑of‑vectors (MoV) is formed as ˆv_ˆd + β·ˆv_global. A signed gate, Gate(h) = sign(p(x) − 0.5), determines whether to add (+α·MoV) or subtract (‑α·MoV) from the hidden state. This single‑shot perturbation is then propagated forward; no model weights are altered. The hyperparameters α (overall strength) and β (global vs. domain contribution) are tuned on a small validation split.

The authors introduce MTU‑Bench, a rigorously constructed benchmark that enforces deterministic triggering, strict parsing, and argument format validation across four domains (code, math, search, translation). Experiments with Qwen2.5‑1.5B (and also 0.5B and 8B variants) show that ASA raises strict tool‑use F1 from 0.18 to 0.50 and reduces the false‑positive rate from 0.15 to 0.05. The entire controller occupies roughly 20 KB, orders of magnitude smaller than LoRA adapters, and adds only 1–2 ms latency. Ablation studies confirm that (i) mixing a modest global component (β≈0.2–0.5) yields the best trade‑off, (ii) router accuracy above 90 % is sufficient for stable performance, and (iii) the signed gate effectively suppresses spurious triggers without harming recall.

The paper contributes three main insights: (1) the existence of a representation‑behavior gap in tool‑calling LLM agents, (2) a practical, training‑free method (ASA) that aligns latent intent with discrete execution behavior, and (3) a new benchmark for evaluating strict tool‑use under evolving schemas. Limitations include reliance on a single hidden layer and linear router/probe models; future work may explore multi‑layer interventions, non‑linear routing, and long‑term stability in production settings. Overall, ASA offers a lightweight, maintainable solution for deploying robust tool‑calling agents in dynamic environments.