Learning a Neural Solver for Parametric PDE to Enhance Physics-Informed Methods

Physics-informed deep learning often faces optimization challenges due to the complexity of solving partial differential equations (PDEs), which involve exploring large solution spaces, require numerous iterations, and can lead to unstable training. These challenges arise particularly from the ill-conditioning of the optimization problem caused by the differential terms in the loss function. To address these issues, we propose learning a solver, i.e., solving PDEs using a physics-informed iterative algorithm trained on data. Our method learns to condition a gradient descent algorithm that automatically adapts to each PDE instance, significantly accelerating and stabilizing the optimization process and enabling faster convergence of physics-aware models. Furthermore, while traditional physics-informed methods solve for a single PDE instance, our approach extends to parametric PDEs. Specifically, we integrate the physical loss gradient with PDE parameters, allowing our method to solve over a distribution of PDE parameters, including coefficients, initial conditions, and boundary conditions. We demonstrate the effectiveness of our approach through empirical experiments on multiple datasets, comparing both training and test-time optimization performance. The code is available at https://github.com/2ailesB/neural-parametric-solver.

💡 Research Summary

This paper tackles a fundamental bottleneck in physics‑informed deep learning, namely the difficulty of optimizing the highly ill‑conditioned loss that arises when training neural networks to satisfy partial differential equations (PDEs). Traditional physics‑informed neural networks (PINNs) minimize a loss composed of a residual term and a boundary‑condition term, but the Hessian of this loss often has a very large condition number, especially when the network contains high‑frequency components. The authors illustrate this problem analytically on a 1‑D Poisson equation, showing that the condition number grows as the fourth power of the maximum Fourier frequency, which in turn forces standard gradient‑based optimizers (SGD, Adam, L‑BFGS) to require thousands of iterations to converge.

To overcome this, the authors propose a “neural solver” – a meta‑optimizer that learns to pre‑condition the gradient of the physics‑informed loss. Concretely, given the current gradient ∇θ L_PDE and the PDE parameters (coefficients γ, source term f, boundary/initial data g), a neural network F_ρ outputs a transformed update direction. The update rule becomes
θ_{l+1}=θ_l−η F_ρ(∇θ L_PDE(θ_l), γ, f, g).
Thus, F_ρ acts as a learned, instance‑specific pre‑conditioner that reshapes the loss landscape into a more favorable form, allowing a small fixed number L of iterations to reach a low loss.

Training proceeds in two nested loops. In the inner loop (Algorithm 1), the current F_ρ is fixed and applied for L steps to a particular PDE instance, producing parameters θ_L. In the outer loop (Algorithm 2), the reconstructed solution u_{θ_L} is compared against ground‑truth data (generated by a high‑fidelity solver or measurements) using a data loss, and gradients are back‑propagated to update the meta‑optimizer parameters ρ. The dataset consists of many PDE instances sampled from a distribution over (γ, f, g), so the learned F_ρ can generalize to unseen instances without any further training.

The authors evaluate the approach on three benchmark families: (i) 1‑D Poisson problems with varying frequencies, (ii) 2‑D Darcy flow with spatially varying permeability, and (iii) time‑dependent nonlinear wave equations. Across all cases, the neural solver reduces the number of required iterations by one to two orders of magnitude compared with standard PINN training (Adam or L‑BFGS). Visualizations of loss landscapes and gradient trajectories demonstrate that F_ρ flattens steep valleys and aligns the descent direction with the most informative subspace, effectively acting as a learned pre‑conditioner that operates directly on the continuous residual, bypassing any discretization step.

Key contributions include: (1) framing the solution of parametric PDEs as a meta‑learning problem where the optimizer itself is trained from data; (2) a concrete instantiation that integrates the PDE parameters into the optimizer, enabling fast test‑time adaptation; (3) extensive empirical evidence that the method solves PDEs that cause standard PINNs to diverge, and that it scales from 1‑D static to 2‑D+time problems; (4) an open‑source implementation.

The paper positions its method as a bridge between traditional numerical pre‑conditioning (which requires hand‑crafted, problem‑specific designs) and modern neural operators (which learn a direct mapping from PDE coefficients to solutions). Unlike neural operators, the neural solver does not produce the solution in a single forward pass; instead, it provides a highly efficient optimization routine that converges in a few steps, preserving the flexibility of PINNs (e.g., handling arbitrary boundary conditions) while dramatically improving convergence speed.

Limitations noted include the dependence on a representative training set of PDE instances and the potential need for more expressive meta‑optimizers for extremely complex multi‑physics problems. Future work could explore transformer‑based meta‑optimizers, incorporation of physical invariants into F_ρ, and extension to stochastic PDEs or inverse problems.

In summary, the paper introduces a learned physics‑aware optimizer that dramatically accelerates and stabilizes the training of PINNs for parametric PDE families, offering a practical pathway toward real‑time scientific computing and large‑scale design optimization.