Accelerating Structured Chain-of-Thought in Autonomous Vehicles

Chain-of-Thought (CoT) reasoning enhances the decision-making capabilities of vision-language-action models in autonomous driving, but its autoregressive nature introduces significant inference latency, making it impractical for real-time applications. To address this, we introduce FastDriveCoT, a novel parallel decoding method that accelerates template-structured CoT. Our approach decomposes the reasoning process into a dependency graph of distinct sub-tasks, such as identifying critical objects and summarizing traffic rules, some of which can be generated in parallel. By generating multiple independent reasoning steps concurrently within a single forward pass, we significantly reduce the number of sequential computations. Experiments demonstrate a 3-4$\times$ speedup in CoT generation and a substantial reduction in end-to-end latency across various model architectures, all while preserving the original downstream task improvements brought by incorporating CoT reasoning.

💡 Research Summary

The paper tackles a critical bottleneck in modern vision‑language‑action (VLA) systems for autonomous driving: the latency introduced by chain‑of‑thought (CoT) prompting. While CoT improves reasoning and downstream decision quality by forcing the model to articulate intermediate steps (environment description, object identification, rule summarization, etc.), its autoregressive nature forces the model to generate hundreds of tokens sequentially. In a domain where control loops run at 10 Hz or faster, such overhead is unacceptable.

To solve this, the authors propose FastDriveCoT, a parallel decoding framework that restructures CoT into a structured template and a dependency graph. The template breaks the reasoning process into explicit fields (weather, road condition, lane configuration, critical objects, traffic signs, traffic lights, rule summary, interaction summary, overall decision, etc.). Some fields are multi‑instance (e.g., up to three lane time‑ranges, up to four critical objects). For these, a two‑stage process is introduced: an enumeration stage that lists the instances, followed by an elaboration stage that describes each instance. This design enables the detailed descriptions of multiple instances to be generated simultaneously.

The core technical contribution is a directed acyclic graph (DAG) that encodes the precedence constraints among fields. Independent fields (e.g., weather and road condition) have no incoming edges and can be decoded in parallel, while dependent fields (e.g., traffic‑rule summary) wait until their prerequisites are finished. The authors devise a dynamic‑programming‑based scheduler (Algorithm 1) that maintains a “ready set” of nodes with zero remaining dependencies. In each forward pass, the model generates one token for every field in the ready set, updates the KV‑cache jointly, and removes completed nodes, thereby exposing new ready nodes. The algorithm is provably optimal: the number of forward passes equals the length of the critical path (the longest dependency chain), which is the theoretical lower bound for any schedule.

Implementation details address the practical challenges of parallel token generation: attention masks are constructed per field, position IDs are reset within each field, and padding is used to keep tensor shapes uniform. By sharing the KV‑cache across fields in the same pass, memory traffic is dramatically reduced compared to naïvely generating each field independently.

Experiments span several large language models (LLaMA‑2, DeepSeek‑v3, etc.) integrated into VLA pipelines for both meta‑action prediction and trajectory generation on datasets such as nuScenes and the CARLA Leaderboard. Results show a 3.1×–4.1× speedup in CoT token generation and a 30 %+ reduction in overall end‑to‑end latency, while preserving the accuracy gains originally reported for CoT‑augmented policies. The speedup is consistent across both purely autoregressive and “transfusion” (dual‑system) architectures, confirming the method’s robustness.

The paper also situates FastDriveCoT within related work. It distinguishes itself from prior parallel decoding research that focuses on mathematics or coding tasks, where the number of independent sub‑tasks is limited. In autonomous driving, the reasoning pattern is highly regular and domain‑specific, providing a natural avenue for extensive parallelism. The authors compare against task‑decomposition approaches (Least‑to‑Most, Tree‑of‑Thoughts) and token‑level speculative decoding, highlighting that FastDriveCoT achieves higher parallelism without sacrificing reasoning fidelity.

Limitations are acknowledged: the current template uses fixed slots (three lane ranges, four objects), requiring “N/A” placeholders when fewer instances exist. Dynamic slot allocation and automatic graph construction are left for future work, as is extending the approach to other robotic domains such as manipulation.

In summary, FastDriveCoT demonstrates that structured, graph‑guided parallel decoding can make LLM‑based reasoning viable for real‑time autonomous driving. By minimizing the number of forward passes while retaining the expressive power of CoT, the method bridges the gap between high‑quality language reasoning and the stringent latency constraints of safety‑critical systems. This work paves the way for broader adoption of large‑model reasoning in robotics, where similar template‑graph paradigms could unlock efficient, interpretable decision‑making across diverse tasks.