Scalable physics-informed deep generative model for solving forward and inverse stochastic differential equations

Physics-informed deep learning approaches have been developed to solve forward and inverse stochastic differential equation (SDE) problems with high-dimensional stochastic space. However, the existing deep learning models have difficulties solving SDEs with high-dimensional spatial space. In the present study, we propose a scalable physics-informed deep generative model (sPI-GeM), which is capable of solving SDE problems with both high-dimensional stochastic and spatial space. The sPI-GeM consists of two deep learning models, i.e., (1) physics-informed basis networks (PI-BasisNet), which are used to learn the basis functions as well as the coefficients given data on a certain stochastic process or random field, and (2) physics-informed deep generative model (PI-GeM), which learns the distribution over the coefficients obtained from the PI-BasisNet. The new samples for the learned stochastic process can then be obtained using the inner product between the output of the generator and the basis functions from the trained PI-BasisNet. The sPI-GeM addresses the scalability in the spatial space in a similar way as in the widely used dimensionality reduction technique, i.e., principal component analysis (PCA). A series of numerical experiments, including approximation of Gaussian and non-Gaussian stochastic processes, forward and inverse SDE problems, are performed to demonstrate the accuracy of the proposed model. Furthermore, we also show the scalability of the sPI-GeM in both the stochastic and spatial space using an example of a forward SDE problem with 38- and 20-dimension stochastic and spatial space, respectively.

💡 Research Summary

This paper addresses a critical bottleneck in solving stochastic differential equations (SDEs) that involve both high‑dimensional random inputs and high‑dimensional physical (spatial or spatio‑temporal) domains. While recent physics‑informed deep learning methods—such as physics‑informed neural networks (PINNs) and physics‑informed generative models (PI‑GeMs, PI‑GANs, PI‑VAEs)—have demonstrated scalability in the stochastic dimension, they struggle when the spatial dimension exceeds two, because the loss functions (e.g., MMD, KL, Wasserstein‑1) require evaluating distances over thousands of discretization points, leading to prohibitive computational cost.

To overcome this, the authors propose a scalable physics‑informed deep generative model (sPI‑GeM) that consists of two complementary components:

1. Physics‑informed Basis Network (PI‑BasisNet).

   • Architecture: Two subnetworks, (NN_C(U;\theta_C)) and (NN_B(x;\theta_B)). The former maps stochastic information (U) (e.g., snapshots of forcing terms or boundary data) to a vector of coefficients (\psi = (\psi_1,\dots,\psi_p)^T). The latter maps spatial coordinates (x) to a set of deterministic basis functions (\phi = (\phi_1,\dots,\phi_p)^T).
   • Output: The solution approximation is expressed as an inner product (u_{NN}(U,x)=\sum_{i=1}^p \psi_i(U),\phi_i(x)). Automatic differentiation embeds the differential operator and boundary conditions directly into the network, yielding a physics‑informed loss.
   • Training loss: (L_a = \omega_{\text{data}} L_{\text{data}} + \omega_{\text{eq}} L_{\text{eq}}), where (L_{\text{data}}) penalizes mismatches at sensor locations and boundary points, and (L_{\text{eq}}) penalizes the PDE residual evaluated via automatic differentiation.

2. Physics‑informed Generative Model (PI‑GeM).

   • Goal: Learn the probability distribution of the coefficient vector (\psi) rather than the full high‑dimensional field.
   • Architecture: A Wasserstein GAN with gradient penalty (WGAN‑GP). The generator (G(\xi;\theta_G)) takes a latent Gaussian vector (\xi) and outputs a synthetic coefficient vector approximating (\psi). The discriminator (D(\cdot;\theta_D)) distinguishes real coefficients (obtained from the trained PI‑BasisNet) from generated ones.
   • Losses: Standard WGAN‑GP objectives, where the generator minimizes (-\mathbb{E}_\xi