Scenario-Transferable Semantic Graph Reasoning for Interaction-Aware Probabilistic Prediction

💡 Research Summary

**
The paper tackles the fundamental problem of predicting the future behaviors of traffic participants for autonomous vehicles, with a particular focus on achieving zero‑shot transferability across diverse driving scenarios. Existing works typically rely on high‑definition (HD) map images or vectorized scene contexts as inputs, which often contain irrelevant or distracting information that can degrade forecasting performance in certain situations. To overcome this limitation, the authors introduce a novel “generic representation” that fuses semantic information with domain knowledge (traffic rules, road topology constraints).

The proposed pipeline first separates static and dynamic aspects of the environment. Static information (road geometry, lane markings, traffic signs) is transformed into a semantic description, while domain knowledge is applied as a hard attention mechanism to filter out agents that cannot influence the target vehicle. Dynamic information (positions, velocities, accelerations of surrounding vehicles) is linked to high‑level “semantic goals” such as “cut in front of the blue car” or “stop behind the stop line,” mirroring the way human drivers think.

These processed elements are then encoded into two types of graphs: a two‑dimensional Semantic Graph (2D‑SG) that captures spatial relationships among static entities and current dynamic agents, and a three‑dimensional Semantic Graph (3D‑SG) that adds a temporal dimension to represent each semantic goal together with its anticipated end state (goal location and arrival time). Unlike conventional graph‑based approaches where each node corresponds to a single agent, here each node embodies a semantic goal, implicitly aggregating the context of multiple agents.

The core prediction engine, called the Semantic Graph Network (SGN), leverages the inductive biases of Graph Neural Networks (GNNs). SGN performs multi‑layer message passing and attention across the 2D‑SG to encode spatial interactions, then propagates this information into the 3D‑SG to reason about spatio‑temporal structures. By distinguishing intra‑relations (within a goal) from inter‑relations (between different goals), the network learns appropriate weighting schemes for complex, hierarchical interactions.

The authors provide a theoretical analysis showing that SGN possesses greater expressive power than standard GNNs, primarily because the semantic‑goal‑centric node design enables the model to capture higher‑order dependencies without requiring an excessive number of layers. Empirically, the method is evaluated on large‑scale real‑world datasets such as Argoverse and nuScenes, covering highways, intersections, and roundabouts. Across all benchmarks, SGN outperforms state‑of‑the‑art baselines (CNN‑based, LSTM‑based, and previous GNN‑based predictors) in terms of accuracy metrics like min‑ADE/min‑FDE.

Crucially, the paper demonstrates zero‑shot transferability: a model trained on a limited set of domains (e.g., only highway data) retains high performance when tested on unseen domains (e.g., dense urban intersections) without any fine‑tuning. This robustness is attributed to the generic, semantics‑driven representation that abstracts away scenario‑specific details while preserving essential relational information.

In summary, the work contributes three major advances: (1) a systematic method for extracting generic, semantics‑based static and dynamic representations using domain knowledge; (2) a unified prediction framework that formulates both inputs and outputs as semantic graphs, enabling explicit modeling of high‑level driving intentions; and (3) a specialized graph reasoning network (SGN) that achieves superior prediction accuracy and demonstrates strong zero‑shot generalization. These contributions collectively push the field toward more adaptable and reliable behavior prediction systems for autonomous driving.