Structure-Preserving Learning Improves Geometry Generalization in Neural PDEs

We aim to develop physics foundation models for science and engineering that provide real-time solutions to Partial Differential Equations (PDEs) which preserve structure and accuracy under adaptation to unseen geometries. To this end, we introduce General-Geometry Neural Whitney Forms (Geo-NeW): a data-driven finite element method. We jointly learn a differential operator and compatible reduced finite element spaces defined on the underlying geometry. The resulting model is solved to generate predictions, while exactly preserving physical conservation laws through Finite Element Exterior Calculus. Geometry enters the model as a discretized mesh both through a transformer-based encoding and as the basis for the learned finite element spaces. This explicitly connects the underlying geometry and imposed boundary conditions to the solution, providing a powerful inductive bias for learning neural PDEs, which we demonstrate improves generalization to unseen domains. We provide a novel parameterization of the constitutive model ensuring the existence and uniqueness of the solution. Our approach demonstrates state-of-the-art performance on several steady-state PDE benchmarks, and provides a significant improvement over conventional baselines on out-of-distribution geometries.

💡 Research Summary

The paper introduces Geo‑NeW (General‑Geometry Neural Whitney Forms), a data‑driven finite‑element framework that learns both a differential operator and a reduced finite‑element space conditioned on geometry. Unlike conventional neural operators that regress a solution map from geometry and boundary data, Geo‑NeW treats the PDE operator itself as the learnable object. The method proceeds in three stages. First, a geometry encoder processes a mesh using coordinate‑free descriptors (heat‑kernel signature, harmonic coordinates, signed‑distance fields) and a transformer to produce a latent context vector z that captures topology and metric information. Second, this context conditions two neural fields that construct a reduced Whitney‑form basis: a set of scalar basis functions ψ₀ᵢ(x) obtained by weighting the original low‑order Whitney nodal functions with non‑negative, partition‑of‑unity weights Wᵢ(x, z). From these, edge‑based 1‑forms ψ₁_{ij}=ψ₀ᵢ∇ψ₀ⱼ−ψ₀ⱼ∇ψ₀ᵢ are derived, preserving the exact sequence property (∇W₀⊂W₁) essential for discrete conservation. The resulting reduced spaces generate mass matrices M₀, M₁ and an incidence matrix δ that encode metric and connectivity of the mesh. Third, the PDE is expressed as a mixed Galerkin system on this reduced space: a linear diffusion term ε δᵀM₁δ u (with ε>0 providing artificial viscosity) plus a nonlinear flux F_θ(u, z) learned by a geometry‑conditioned transformer. The discrete governing equation reads
ε δᵀM₁δ u + δᵀF_θ(u, z) − M₀ f_θ(z)=0,
with boundary conditions enforced exactly via the finite‑element formulation. Training minimizes an L₂ reconstruction loss subject to the PDE constraint, using implicit differentiation through a Newton solver and an adjoint linear solve for the Lagrange multiplier. The authors prove that, under the convexity of F_θ in the solution variables and ε>0, the learned system admits a unique solution. Experiments on several steady‑state PDE benchmarks (Laplace, electromagnetics, diffusion‑reaction) demonstrate state‑of‑the‑art accuracy on in‑distribution tasks and, crucially, superior generalization to out‑of‑distribution geometries such as domains with large angular steps or multiple irregular obstacles. Compared to transformer‑based baselines (e.g., Transolver), Geo‑NeW maintains low error even when the test geometry deviates substantially from the training distribution. Despite a reduced model capacity (the learned reduced basis has far fewer degrees of freedom than the full mesh), the method matches or exceeds the performance of larger unconstrained models. In summary, Geo‑NeW unifies (i) mesh‑based geometry encoding, (ii) transformer‑conditioned reduced Whitney forms, and (iii) Finite Element Exterior Calculus‑based structure preservation to deliver a physics‑consistent, geometry‑agnostic neural PDE solver capable of real‑time inference on unseen domains. This work marks a significant step toward scientific foundation models that can handle the geometric diversity encountered in real engineering applications.