Benchmarking Long Roll-outs of Auto-regressive Neural Operators for the Compressible Navier-Stokes Equations with Conserved Quantity Correction

Deep learning has been proposed as an efficient alternative for the numerical approximation of PDE solutions, offering fast, iterative simulation of PDEs through the approximation of solution operators. However, deep learning solutions have struggle to perform well over long prediction durations due to the accumulation of auto-regressive error, which is compounded by the inability of models to conserve physical quantities. In this work, we present conserved quantity correction, a model-agnostic technique for incorporation physical conservation criteria within deep learning models. Our results demonstrate consistent improvement in the long-term stability of auto-regressive neural operator models, regardless of the model architecture. Furthermore, we analyze the performance of neural operators from the spectral domain, highlighting significant limitations of present architectures. These results highlight the need for future work to consider architectures that place specific emphasis on high frequency components, which are integral to the understanding and modeling of turbulent flows.

💡 Research Summary

This paper addresses two fundamental challenges that arise when using auto‑regressive neural operators (AR‑NOs) to simulate the compressible Navier‑Stokes equations over long time horizons: (1) the accumulation of prediction errors inherent to the recursive rollout, and (2) the violation of physical conservation laws (mass, momentum, energy) by the learned models. While prior work has demonstrated impressive short‑term performance of neural operators such as the Fourier Neural Operator (FNO) and physics‑informed variants (PINO), those studies typically focus only on mass conservation and evaluate on relatively short rollouts or on datasets with high viscosity that are dynamically simple. Consequently, long‑term stability remains a major obstacle for realistic turbulent flow prediction.

The authors propose a model‑agnostic “Conserved Quantity Correction” (CQC) framework that can be applied as a post‑processing step to any neural operator. CQC distinguishes between scalar conserved fields (e.g., density, energy density) that are non‑negative and vector conserved fields (e.g., momentum components) that may change sign. For scalar fields, the predicted output is rescaled so that its discrete ℓ₁‑norm (a proxy for the integral over the domain) matches that of the previous timestep. This magnitude correction guarantees non‑negativity and exact conservation of the total scalar quantity. For vector fields, a shift correction is applied: the difference between the previous total integral and the predicted integral is added uniformly across the domain, preserving the sign distribution while enforcing zero net change (or a prescribed net change). Both corrections are inexpensive, differentiable, and require only the previous state, making them suitable for auto‑regressive training.

To evaluate CQC, the study implements two representative neural operators: the original FNO and the more recent DPOT (Fourier‑Attention based operator). Both architectures are modified to accept a minimal temporal history (one or two previous timesteps) rather than the ten‑step histories used in earlier works, thereby improving training efficiency and allowing the models to focus on the most dynamically informative early frames. Additional practical measures include internal normalization (as in Huang et al.) and clipping of predicted values to a small positive floor (1e‑8) to avoid unphysical negatives.

Experiments are conducted on the 2‑D compressible Navier‑Stokes benchmark from PDEBench, specifically the inviscid case (viscosities set to 1e‑8) with Mach numbers 0.1 and 1.0. These datasets are deliberately chosen for their highly dynamic behavior, which challenges models that might otherwise learn near‑identity mappings on more viscous, slowly evolving flows. The authors perform rollouts extending to several hundred timesteps, comparing baseline (no correction) and CQC‑enhanced versions in terms of relative L₂ error, conservation error, and spectral energy distribution.

Key findings include:

1. Conservation Accuracy – CQC reduces total conserved‑quantity drift to near machine precision. Momentum correction, which was absent in earlier work, yields a three‑fold reduction in net momentum error compared with mass‑only correction.
2. Long‑Term Error Growth – Over long rollouts, the relative L₂ error of CQC‑augmented models is 20‑35 % lower than that of the uncorrected baselines. The error growth curve is markedly flatter, indicating improved stability.
3. Spectral Fidelity – High‑frequency Fourier modes (above ~0.3 π/Δx) retain significantly more energy when CQC is applied, suggesting that the correction mitigates the tendency of neural operators to dampen small‑scale features over time.
4. Architecture‑Independence – Both FNO and DPOT benefit similarly from CQC, with DPOT showing a modest edge in absolute accuracy but no qualitative difference in stability gains.

A spectral analysis reveals a systemic limitation of current operator designs: they under‑represent high‑frequency components, which are crucial for capturing turbulent cascades. The authors argue that future architectures should incorporate explicit high‑frequency emphasis mechanisms (e.g., multi‑scale pyramids, dedicated high‑pass pathways, or physics‑based constraints on the spectral tail).

In summary, the paper introduces a simple yet powerful conserved‑quantity correction that can be grafted onto any neural operator without redesigning the core architecture. By enforcing exact conservation of mass, momentum, and energy at each rollout step, the method dramatically improves long‑term predictive stability and spectral fidelity for compressible flow simulations. The work provides a clear pathway for integrating physical invariants into data‑driven PDE solvers and sets the stage for next‑generation neural operators that are both accurate and physically trustworthy over extended simulation horizons.