ForSim: Stepwise Forward Simulation for Traffic Policy Fine-Tuning

As the foundation of closed-loop training and evaluation in autonomous driving, traffic simulation still faces two fundamental challenges: covariate shift introduced by open-loop imitation learning and limited capacity to reflect the multimodal behaviors observed in real-world traffic. Although recent frameworks such as RIFT have partially addressed these issues through group-relative optimization, their forward simulation procedures remain largely non-reactive, leading to unrealistic agent interactions within the virtual domain and ultimately limiting simulation fidelity. To address these issues, we propose ForSim, a stepwise closed-loop forward simulation paradigm. At each virtual timestep, the traffic agent propagates the virtual candidate trajectory that best spatiotemporally matches the reference trajectory through physically grounded motion dynamics, thereby preserving multimodal behavioral diversity while ensuring intra-modality consistency. Other agents are updated with stepwise predictions, yielding coherent and interaction-aware evolution. When incorporated into the RIFT traffic simulation framework, ForSim operates in conjunction with group-relative optimization to fine-tune traffic policy. Extensive experiments confirm that this integration consistently improves safety while maintaining efficiency, realism, and comfort. These results underscore the importance of modeling closed-loop multimodal interactions within forward simulation and enhance the fidelity and reliability of traffic simulation for autonomous driving. Project Page: https://currychen77.github.io/ForSim/

💡 Research Summary

The paper addresses two long‑standing challenges in traffic simulation for autonomous driving: covariate shift caused by open‑loop imitation learning and the inability to faithfully reproduce the multimodal behaviors observed in real traffic. While the recent RIFT framework mitigates covariate shift through group‑relative optimization, its forward simulation remains largely non‑reactive—only the first step is closed‑loop, and subsequent steps are rolled out in an open‑loop fashion, which leads to unrealistic agent interactions and limits fidelity.

To overcome these limitations, the authors propose ForSim, a stepwise closed‑loop forward simulation paradigm. At each virtual timestep, every traffic agent selects the candidate trajectory that best aligns spatiotemporally with a reference trajectory and propagates it using a PID controller coupled with a kinematic bicycle model. This ensures physical plausibility while preserving intra‑modality consistency. Three rollout strategies are examined: Max‑Likelihood (always picks the highest‑confidence candidate, quickly collapsing multimodality), Mode‑Consistent (fixes the initial mode but suffers from temporal misalignment), and the newly introduced Trajectory‑Aligned rollout, which keeps the initial reference trajectory fixed and dynamically selects the candidate minimizing average displacement error after temporal alignment. The Trajectory‑Aligned approach maintains multimodal diversity and physical coherence throughout the rollout.

For other agents, the paper compares Constant‑Action (open‑loop action propagation), Single‑Prediction (single open‑loop trajectory), and Stepwise Prediction (closed‑loop, interaction‑aware propagation). The Stepwise Prediction rollout updates other agents’ predictions at every virtual step, enabling truly interactive simulations.

Integrating ForSim with RIFT and evaluating on the nuPlan dataset within the CARLA simulator, the authors demonstrate a substantial reduction in collision rates and improved safety metrics while preserving efficiency, realism, and comfort. Notably, in scenarios with multiple plausible maneuvers (e.g., intersection turn choices), ForSim maintains distinct rollouts for each mode, preventing policy collapse to a single dominant behavior. This richer multimodal feedback enhances group‑relative optimization, leading to more reliable policy fine‑tuning.

In summary, ForSim contributes a physically grounded, stepwise closed‑loop simulation that simultaneously guarantees multimodal fidelity, intra‑modality consistency, and interaction awareness. The work paves the way for higher‑fidelity traffic simulation and more robust autonomous driving policy development, with future extensions envisioned for complex urban environments, real‑time policy updates, and sim‑to‑real domain adaptation.