DD-MDN: Human Trajectory Forecasting with Diffusion-Based Dual Mixture Density Networks and Uncertainty Self-Calibration

Human Trajectory Forecasting (HTF) predicts future human movements from past trajectories and environmental context, with applications in Autonomous Driving, Smart Surveillance, and Human-Robot Interaction. While prior work has focused on accuracy, social interaction modeling, and diversity, little attention has been paid to uncertainty modeling, calibration, and forecasts from short observation periods, which are crucial for downstream tasks such as path planning and collision avoidance. We propose DD-MDN, an end-to-end probabilistic HTF model that combines high positional accuracy, calibrated uncertainty, and robustness to short observations. Using a few-shot denoising diffusion backbone and a dual mixture density network, our method learns self-calibrated residence areas and probability-ranked anchor paths, from which diverse trajectory hypotheses are derived, without predefined anchors or endpoints. Experiments on the ETH/UCY, SDD, inD, and IMPTC datasets demonstrate state-of-the-art accuracy, robustness at short observation intervals, and reliable uncertainty modeling. The code is available at: https://github.com/kav-institute/ddmdn.

💡 Research Summary

DD‑MDN (Diffusion‑based Dual Mixture Density Network) addresses three critical gaps in human trajectory forecasting (HTF): high positional accuracy, reliable uncertainty estimation, and robustness to very short observation windows. The authors combine a few‑shot denoising diffusion backbone with a dual‑Mixture Density Network (MDN) that simultaneously learns two complementary Gaussian mixture representations.

The first representation, Θ_step, predicts a per‑time‑step Gaussian mixture (M components) for each future horizon step. This yields calibrated, per‑timestep confidence regions (e.g., 68 % and 95 % ellipses) directly from the model without post‑hoc processing. The second representation, Θ_anchor, stacks the same means and covariances across all future steps to form a single high‑dimensional Gaussian mixture in the 2·T_fut trajectory space. Each mixture component now corresponds to a coherent “anchor trajectory” together with its temporal covariance, providing realistic, time‑consistent path hypotheses.

Training uses the Negative Log‑Likelihood (NLL) loss, which is a strictly proper scoring rule. For unimodal Gaussians NLL decomposes into a Mahalanobis distance term (penalizing mean error) and a log‑determinant term (penalizing over‑confident covariances), naturally balancing calibration and sharpness. In the multimodal case the mixture weights are soft‑maxed, and components far from the ground‑truth contribute little to the loss, encouraging specialization and self‑calibration of each mode.

To avoid unnecessary complexity, the model employs dynamic mode pruning. An epoch‑dependent threshold δ(e) and temperature η(e) control a sigmoid gate G_m(e) that gradually suppresses low‑weight components. Only modes with gated weight above the threshold remain active, and their weights are renormalized, allowing the effective number of mixture components M* to adapt to each scene.

A key novelty is the diffusion process applied in parameter space rather than pixel space. The diffusion backbone corrupts the Gaussian parameters (means, covariances, and mixture weights) with known Markov noise and learns to denoise them. This creates a generative prior over the manifold of valid distribution parameters, enforcing global temporal coherence that standard MDNs lack.

During inference, Θ_step and Θ_anchor are fed into an affine re‑parameterization sampler that draws K discrete trajectory hypotheses. Each hypothesis inherits the per‑timestep uncertainty from Θ_step and the coherent anchor path from Θ_anchor, and the hypotheses are ranked by their learned probabilities. Consequently, downstream planners receive not only diverse future paths but also calibrated likelihoods for each path.

Extensive experiments on four benchmarks—ETH/UCY, Stanford Drone Dataset (SDD), inD, and IMPTC—show that DD‑MDN achieves state‑of‑the‑art ADE/FDE scores while dramatically reducing Expected Calibration Error (ECE) and NLL compared to both deterministic diffusion models (e.g., LED, SingularTrajectory) and existing probabilistic approaches (e.g., Social‑STAGE, TUTR). Moreover, when the observation window is reduced to as little as 0.5 seconds, performance degradation is minimal, demonstrating the model’s robustness to short‑term inputs.

In summary, DD‑MDN introduces a principled way to fuse diffusion‑based representation learning with dual Gaussian mixture modeling, delivering high‑accuracy, diverse, and self‑calibrated trajectory forecasts. Its ability to provide reliable uncertainty estimates and to operate effectively with minimal observation data makes it especially suitable for safety‑critical applications such as autonomous driving, human‑robot collaboration, and smart surveillance.