CacheFlow: Fast Human Motion Prediction by Cached Normalizing Flow

Many density estimation techniques for 3D human motion prediction require a significant amount of inference time, often exceeding the duration of the predicted time horizon. To address the need for faster density estimation for 3D human motion prediction, we introduce a novel flow-based method for human motion prediction called CacheFlow. Unlike previous conditional generative models that suffer from poor time efficiency, CacheFlow takes advantage of an unconditional flow-based generative model that transforms a Gaussian mixture into the density of future motions. The results of the computation of the flow-based generative model can be precomputed and cached. Then, for conditional prediction, we seek a mapping from historical trajectories to samples in the Gaussian mixture. This mapping can be done by a much more lightweight model, thus saving significant computation overhead compared to a typical conditional flow model. In such a two-stage fashion and by caching results from the slow flow model computation, we build our CacheFlow without loss of prediction accuracy and model expressiveness. This inference process is completed in approximately one millisecond, making it 4 times faster than previous VAE methods and 30 times faster than previous diffusion-based methods on standard benchmarks such as Human3.6M and AMASS datasets. Furthermore, our method demonstrates improved density estimation accuracy and comparable prediction accuracy to a SOTA method on Human3.6M. Our code and models are available at https://github.com/meaten/CacheFlow.

💡 Research Summary

CacheFlow addresses the critical need for fast, accurate density estimation in 3D human motion prediction, a task essential for safety‑critical applications such as autonomous driving and human‑robot interaction. Traditional stochastic approaches—GANs, VAEs, and diffusion models—either lack explicit probability density modeling or suffer from prohibitive inference latency. While conditional normalizing flows (NFs) and continuous normalizing flows (CNFs) provide exact likelihoods, they still require a full forward pass of the flow network at test time, which is computationally expensive for high‑dimensional motion data.

The core contribution of CacheFlow is a two‑stage architecture that decouples an unconditional flow model from the conditional component. First, an unconditional CNF fθ is trained to map a simple base distribution (standard Gaussian) to the latent space of future motions. This model is independent of any observed past trajectory, allowing its forward pass to be executed offline. The outputs—latent vectors zₖ, corresponding motion representations xₖ, and the Jacobian determinants |det Jfθ(zₖ)|—are stored as a large cache of K triplets.

During inference, a lightweight conditional base distribution qϕ(z|c) is estimated from the observed past sequence c using a small neural network (e.g., an MLP or a shallow transformer). This network predicts the parameters of a Gaussian mixture that serves as an informative prior over the latent space. The inference pipeline then simply selects or samples latent codes z from the cached set according to the probabilities given by qϕ. The final future‑motion density is reconstructed analytically as

p(x|c) = qϕ(z|c) · |det Jfθ(z)|⁻¹,

and the motion sequence x is obtained by passing the selected z through the pre‑computed flow (or directly reading the cached xₖ). Because the heavy flow computation has been pre‑cached, the only online operations are the evaluation of qϕ and a memory lookup, resulting in an inference time of roughly 1 ms—four times faster than state‑of‑the‑art VAEs and thirty times faster than diffusion‑based models.

Training combines two complementary objectives. Flow matching is employed to train the CNF without costly ODE integration: a vector field vθ(zₜ) is directly optimized to match a ground‑truth field derived from straight‑line interpolations between sampled z₀ and z₁. Simultaneously, the conditional base network is trained with a maximum‑likelihood loss to ensure that qϕ(z|c) assigns high probability to latent codes that correspond to the true future motion given the past. This dual‑loss scheme preserves the expressive power of the unconditional flow while enabling a highly efficient conditional mapping.

Extensive experiments on the Human3.6M and AMASS benchmarks demonstrate that CacheFlow matches or exceeds the prediction accuracy of the best existing methods (e.g., MPJPE around 41 mm on Human3.6M) while achieving substantially higher log‑likelihood scores than both VAEs and diffusion models, indicating superior density estimation. Diversity metrics show comparable or slightly better sample variety relative to VAEs. The method also dramatically reduces computational overhead, making it suitable for real‑time deployment on embedded platforms.

Limitations include the memory footprint of the cached triplets, which scales linearly with the number of latent samples K and the dimensionality of the latent space. The authors discuss potential compression strategies and adaptive caching for future work. Additionally, the conditional base network’s capacity may limit performance on extremely long or highly complex histories; incorporating attention‑based encoders could mitigate this.

In summary, CacheFlow introduces a novel caching paradigm for normalizing‑flow‑based motion prediction, delivering millisecond‑scale inference without sacrificing accuracy or expressive density modeling. Its design is readily extensible to other high‑dimensional sequential domains such as hand‑gesture forecasting or vehicle trajectory prediction, and it opens avenues for integrating explicit probabilistic guarantees into real‑time autonomous systems.