Recent advances in scientific machine learning (SciML) have enabled neural operators (NOs) to serve as powerful surrogates for modeling the dynamic evolution of physical systems governed by partial differential equations (PDEs). While existing approaches focus primarily on learning simulations from the target PDE, they often overlook more fundamental physical principles underlying these equations. Inspired by how numerical solvers are compatible with simulations of different settings of PDEs, we propose a multiphysics training framework that jointly learns from both the original PDEs and their simplified basic forms. Our framework enhances data efficiency, reduces predictive errors, and improves out-of-distribution (OOD) generalization, particularly in scenarios involving shifts of physical parameters and synthetic-to-real transfer. Our method is architecture-agnostic and demonstrates consistent improvements in normalized root mean square error (nRMSE) across a wide range of 1D/2D/3D PDE problems. Through extensive experiments, we show that explicit incorporation of fundamental physics knowledge significantly strengthens the generalization ability of neural operators. We will release models and codes at https://sites.google.com/view/sciml-fundemental-pde.
Recent advances in scientific machine learning (SciML) have broadened traditional machine learning (ML) for modeling physical systems, using deep neural networks (DNNs) especially neural operators (NOs) (Li et al., 2021a;Pathak et al., 2022;Lam et al., 2023;Bi et al., 2023) as fast, accurate surrogates for solving partial differential equations (PDEs) (Raissi et al., 2019;Edwards, 2022;Kochkov et al., 2021;Pathak et al., 2022). However, compared to numerical methods, a key disadvantage of recent data-driven SciML models is their limited integration of fundamental physical knowledge.
Numerical solvers, though tailored to specific PDEs or discretizations, inherently preserve physical laws (e.g., conservation, symmetry), ensuring consistent and plausible simulations across diverse conditions (physical parameters, boundaries, geometries, etc.) (Ketcheson et al., 2012;Hansen et al., 2023;Mouli et al., 2024;Holl & Thuerey, 2024). In contrast, data-driven models, despite learning across PDE types (e.g., via multiphysics pretraining in SciML foundation models (McCabe et al., 2023;Hao et al., 2024)), remain sensitive to training distributions, degrading under distribution shifts (Subramanian et al., 2023;Benitez et al., 2024) and requiring large, diverse datasets. This fragility is worsened by the absence of rigorous verification: Unlike classical solvers, SciML models are rarely evaluated against decomposed PDE components. This gap introduces three major challenges: 1) High data demands: Without physics priors, neural operators require large, diverse datasets for high precision, as seen in recent SciML foundation models (Hao et al., 2024;McCabe et al., 2023) which focus on generalization without addressing training data efficiency. 2) Physical inconsistency: Lacking inductive biases, these models may violate conservation laws or produce unphysical outputs, particularly in long-term rollout predictions. 3) Poor generalization: Neural PDE solvers often struggle with unseen simulation settings and requires retraining. Motivated by the above challenges, we ask two scientific questions: Q1: Can neural operators understand both original PDEs and fundamental physics knowledge? Q2: Can neural operators benefit from explicit learning of fundamental physics knowledge?
In this paper, we highlight the importance of enforcing the learning of fundamental physical knowledge in neural operators. The key idea is to identify physically plausible basic terms that can be decomposed from original PDEs, and incorporate their simulations during training. Although often overlooked in SciML, our experiments demonstrate that these fundamental physical terms encode rich physical knowledge. Not only can they be utilized without incurring additional computational costs, but they also widely offer substantial and multifaceted benefits. This opens up a new door to improve the comprehensive generalization of neural operators with data efficiency.
We summarize our contributions below:
- Through comprehensive evaluations of public SciML models, we observe a strong correlation between performance on original PDEs and basic PDE terms, highlighting the importance of fundamental physical knowledge in neural operators (Section 2.2). 2. We propose to explicitly incorporate fundamental physical knowledge into neural operators.
Specifically, we design a simple and principled multiphysics strategy to train neural operators on simulations of both the original PDE and its basic form. (Section 3). 3. Our method exhibits three major benefits: 1) data efficiency (Section 4.2), 2) long-term physical consistency (Section 4.3), 3) strong generalization in OOD (Section 4.4) and real-world (Section 4.5) scenarios. We evaluate our method on a wide range of 1D/2D/3D PDEs, achieving consistent improvement in nRMSE (normalized root mean squared error, Section 4.2).
For time-dependent PDEs, the solution is a vector-valued mapping v : T × S × Θ → R d , defined on the temporal domain T , the spatial domain S, and the parameter space Θ, with d the number of dependent variables. Numerical solvers compute v θ (t, •) from ℓ ≥ 1 past steps, enabling finitedifference approximations:
, where ∆t is the temporal resolution. SciML aims to learn a surrogate operator N θ,ϕ , parameterized by physical parameters θ and learnable weights ϕ, that approximates this mapping. Given N simulations, D := v (i) ([0 : t max ], •) | i = 1, . . . , N , the model is trained by minimizing a loss L, often the normalized root mean squared error: nRMSE = ∥vpred-v∥2 ∥v∥2
, where v pred is the model prediction. (MPP (McCabe et al., 2023), DPOT (Hao et al., 2024), Hyena (Patil et al., 2023)) exhibit correlated yet worse performance on fundamental physics (x-axis).
We begin with a motivating example to highlight the importance of incorporating fundamental physical knowledge into neural operators. Specifically, we gather publicly released pretrained neural operators, with a focus on SciML foundatio
This content is AI-processed based on open access ArXiv data.