Diffusion models recently developed for generative AI tasks can produce high-quality samples while still maintaining diversity among samples to promote mode coverage, providing a promising path for learning stochastic closure models. Compared to other types of generative AI models, such as GANs and VAEs, the sampling speed is known as a key disadvantage of diffusion models. By systematically comparing transport-based generative models on a numerical example of 2D Kolmogorov flows, we show that flow matching in a lower-dimensional latent space is suited for fast sampling of stochastic closure models, enabling single-step sampling that is up to two orders of magnitude faster than iterative diffusion-based approaches. To control the latent space distortion and thus ensure the physical fidelity of the sampled closure term, we compare the implicit regularization offered by a joint training scheme against two explicit regularizers: metric-preserving (MP) and geometry-aware (GA) constraints. Besides offering a faster sampling speed, both explicitly and implicitly regularized latent spaces inherit the key topological information from the lower-dimensional manifold of the original complex dynamical system, which enables the learning of stochastic closure models without demanding a huge amount of training data.
Complex dynamical systems, such as turbulent flows [1] or solid mechanics [2] in engineering applications and physical processes in the Earth system [3], are often featured by interactions across vast and continuous scales of space and time. The computational cost of fully resolving every scale in a Direct Numerical Simulation (DNS) is often prohibitive [4] for real-world science and engineering problems, and practical numerical simulations need to rely on closure models to approximate the impact of unresolved, small-scale dynamics on the numerically resolved coarse-grained variables. Most existing methods, e.g., RANS or LES closures for modeling turbulence, rely on a deterministic assumption, which only approximately holds if the unresolved scales achieve equilibrium in a time scale much faster than the one that those resolved scales evolve with. However, such a separation between resolved and unresolved scales may not exist for certain problems where the unresolved scales are far from equilibrium, motivating recent studies of going beyond the deterministic closures and exploring stochastic modeling approaches [5].
Stochastic modeling has been explored for complex dynamical systems such as turbulence, for several decades [6,7], leading to the development of stochastic models for some complex features of turbulent flows, e.g., intermittency [8] and back scattering [9]. Starting around the millennium, a substantial amount of research about stochastic models was explored for geophysical flows [10][11][12], with an excellent review of stochastic modeling for weather and climate presented by [13]. In the meantime, stochastic modeling techniques such as random matrices were also explored in solid mechanics [14] to account for the model uncertainties. More recently, mesoscale stochastic approaches were explored in the modeling of many complex systems, such as metallic foams [15] and cellular interactions [16]. From a broader perspective, stochasticity naturally shows up in reduced-order modeling techniques such as Mori-Zwanzig formalism, which demonstrates that when fast-evolving variables are integrated out of a system, their influence on the slow variables manifests as both a modified deterministic force and essential memory (non-Markovian) and stochastic noise terms [17,18]. In practice, stochastic parameterizations have been shown to sharpen mean predictions, restore physical multi-modal variability, and reproduce the heavy-tailed statistics of extreme events across a wide range of applications [13,[19][20][21][22][23][24]. However, developing and calibrating stochastic closures present their own significant challenges [25][26][27][28], which often pose a more complicated model structure than classical deterministic closures and thus underscore the need for both a larger amount of data and a more sophisticated calibration procedure. This need can be addressed by the growing field of scientific machine learning (SciML), which seeks to augment or replace traditional scientific modeling pipelines with machine learning techniques [29][30][31][32]. Broadly, SciML efforts in dynamical systems modeling follow two main thrusts. The first thrust aims to create data-driven surrogates that approximate the system’s evolution from data, effectively replacing traditional physics-based models, e.g., via system identification [33][34][35][36] or operator learning [37][38][39]. The second thrust [40][41][42][43][44][45][46], as the focus of this work, uses machine-learning-based models not to replace the traditional physics-based solver but to augment it. This is the goal of data-driven closure modeling, which retains the well-established physical solver for the resolved scales and uses a learned model for the contributions from unresolved ones. It is worth noting that many research works (e.g., [40,41]) in the second thrust adopted a deterministic form of the machinelearning-based models, while the recent advances in generative AI techniques opened up the possibilities of systematically constructing and calibrating data-driven stochastic closure models [44].
Among the recent developments of generative AI techniques, three key paradigms, all united under a general transport-based framework, have emerged as compelling solutions:
• Score-based Diffusion Models transform data into a simple prior distribution (typically Gaussian noise) through a fixed forward SDE and then learn to reverse this process with a learned score function. This approach has been successfully applied to stochastic closure modeling [44] and excels at capturing rich, non-Gaussian posteriors, but their highly curved transport paths necessitate slow, iterative sampling with hundreds of solver steps to maintain fidelity [47][48][49]. The extension to conditional diffusion models has been explored by various computational mechanics problems [44,[50][51][52][53][54].
• Flow Matching replaces the stochastic noising path with a simpler, often linear, interpola
This content is AI-processed based on open access ArXiv data.