Bridging the Perception Gap: A Lightweight Coarse-to-Fine Architecture for Edge Audio Systems

Deploying Audio-Language Models (Audio-LLMs) on edge infrastructure exposes a persistent tension between perception depth and computational efficiency. Lightweight local models tend to produce passive perception - generic summaries that miss the subtle evidence required for multi-step audio reasoning - while indiscriminate cloud offloading incurs unacceptable latency, bandwidth cost, and privacy risk. We propose CoFi-Agent (Tool-Augmented Coarse-to-Fine Agent), a hybrid architecture targeting edge servers and gateways. It performs fast local perception and triggers conditional forensic refinement only when uncertainty is detected. CoFi-Agent runs an initial single-pass on a local 7B Audio-LLM, then a cloud controller gates difficult cases and issues lightweight plans for on-device tools such as temporal re-listening and local ASR. On the MMAR benchmark, CoFi-Agent improves accuracy from 27.20% to 53.60%, while achieving a better accuracy-efficiency trade-off than an always-on investigation pipeline. Overall, CoFi-Agent bridges the perception gap via tool-enabled, conditional edge-cloud collaboration under practical system constraints.

💡 Research Summary

The paper tackles a fundamental dilemma in deploying audio‑language models (Audio‑LLMs) on edge devices: deep, accurate perception often demands heavy computation, while edge constraints (latency, bandwidth, privacy) limit the resources that can be devoted to each query. Lightweight local models typically produce “passive perception” – a single‑pass, generic summary that lacks the fine‑grained evidence needed for multi‑step reasoning. In contrast, always‑offloading to the cloud or running an exhaustive tool pipeline (e.g., repeated listening, automatic speech recognition) can achieve higher accuracy but incurs unacceptable latency, bandwidth usage, and privacy exposure.

CoFi‑Agent (Tool‑Augmented Coarse‑to‑Fine Agent) is introduced as a hybrid, conditional architecture that keeps raw audio on‑premise and only invokes expensive cloud reasoning when the local model signals uncertainty. The system consists of three logical stages:

1. Edge Coarse Perception (Stage 0) – A 7‑billion‑parameter Qwen2‑Audio‑Instruct model runs in FP16 on the edge GPU. It processes the audio‑question pair once, producing an initial answer (p₀) and a short rationale (s₀). This step is extremely fast (≈0.15 s) and serves as a “fast path” for easy queries.

2. Adaptive Confidence Gate – A lightweight prompt‑based classifier G, hosted in the cloud, examines (s₀, Q, p₀) for hedging language, missing evidence, or logical inconsistency. It outputs a binary flag u∈{0,1}. If u = 0, the system returns p₀ immediately; if u = 1, the query is escalated to the refinement pipeline. In experiments, about 62 % of MMAR samples trigger escalation.

3. Conditional Refinement (Stages 1‑2)
   Stage 1 – Cloud‑Guided Planning: The cloud controller selects a region‑of‑interest (ROI) in the audio using a hybrid segmentation strategy (energy‑based event detection plus fixed sliding windows). Only timestamps are sent to the edge, preserving privacy.
   Stage 2 – On‑Device Tool Execution: Two lightweight tools may be invoked on the selected ROI:
   – Temporal Re‑listening – the local Audio‑LLM re‑processes the ROI with a focused sub‑question, yielding refined acoustic evidence (e_audio).
   – On‑Device ASR – Whisper‑small transcribes speech in the ROI, producing textual evidence (T_text). The system can run either tool alone or both, depending on the plan.
   Finally, the cloud reasoner (GPT‑4o) integrates (s₀, e_audio, T_text, Q) to generate the final verdict (y_final).

System Implementation – The edge node is emulated on an NVIDIA Quadro RTX 6000 (24 GB) with a single GPU budget. All models stay resident in memory to avoid loading overhead. The cloud controller uses OpenAI GPT‑4o with temperature = 0 for deterministic outputs. Network round‑trip time to a US‑East data center averages 15 ms (p50) and 45 ms (p95).

Evaluation – The authors benchmark on MMAR (1,000 audio‑question pairs) measuring (i) accuracy and (ii) average wall‑clock latency (seconds per sample, batch = 1). Results:

The adaptive version with ASR achieves the best trade‑off: it more than doubles the baseline accuracy while keeping average latency under 10 s, substantially better than the always‑on pipeline that spends extra time on every sample. A detailed latency breakdown shows that network RTT, cloud gating, tool execution, and final cloud generation dominate the cost, while the edge coarse pass remains negligible.

Tool‑Usage Distribution – Under adaptive gating, 38.2 % of queries finish on the fast path (no tools). Of the remaining 61.8 % that are escalated, 27.4 % use only re‑listening, 19.7 % only ASR, and 14.7 % both. This demonstrates that many hard cases can be resolved with a single lightweight tool, avoiding the overhead of a full always‑on pipeline.

Case Studies – Three illustrative MMAR samples show how CoFi‑Agent corrects baseline failures:

1. A conversational implication question where the baseline mis‑interprets tone; ASR reveals divergent phrasing, enabling the cloud reasoner to detect the meaning shift.
2. A keyword verification in Mandarin; ASR transcription disproves the presence of “Ni Hao,” correcting the answer.
3. A Wi‑Fi password riddle; full dialogue transcription lets the reasoner parse the joke and extract the hidden password string.

Failure Modes – The authors identify three dominant error sources still limiting performance: (1) ASR degradation under extreme low‑SNR, (2) missed brief events in long recordings due to the heuristic segment proposer, and (3) queries that require external knowledge not present in the audio. They suggest future work on noise‑robust ASR, learnable segmentation, and multimodal knowledge integration.

Conclusion – CoFi‑Agent demonstrates that a conditional, coarse‑to‑fine design can bridge the perception gap on edge audio systems. By keeping raw waveforms local, transmitting only compact evidence, and invoking cloud tools only when uncertainty is detected, the system respects privacy, bandwidth, and latency constraints while nearly doubling accuracy (27.20 % → 53.60 %). The paper points to next steps: learnable gating mechanisms, joint training of the segment proposer with the reasoning module, and extending the paradigm to bandwidth‑constrained edge video scenarios. Ultimately, the work argues that future edge AI must intelligently decide “when to think fast locally and when to think slow in the cloud.”