Functional normalizing flow for statistical inverse problems of partial differential equations

Inverse problems of partial differential equations are ubiquitous across various scientific disciplines and can be formulated as statistical inference problems using Bayes’ theorem. To address large-scale problems, it is crucial to develop discretization-invariant algorithms, which can be achieved by formulating methods directly in infinite-dimensional space. We propose a novel normalizing flow based infinite-dimensional variational inference method (NF-iVI) to extract posterior information efficiently. Specifically, by introducing well-defined transformations, the prior in Bayes’ formula is transformed into post-transformed measures that approximate the posterior. To circumvent the issue of mutually singular probability measures, we formulate general conditions for the employed transformations. As guiding principles, these conditions yield four concrete transformations. Additionally, to minimize computational demands, we have developed a conditional normalizing flow variant, termed CNF-iVI, which is adapt at processing measurement data of varying dimensions while requiring minimal computational resources. We apply the proposed algorithms to three typical inverse problems governed by the simple smooth equation, the steady-state Darcy flow equation, and the electric impedance tomography. Numerical results confirm our theoretical findings, illustrate the efficiency of our algorithms, and verify the discretization-invariant property.

💡 Research Summary

This paper tackles the challenging problem of Bayesian inference for inverse problems governed by partial differential equations (PDEs) in infinite‑dimensional function spaces. Traditional approaches either discretize the PDE first and then apply Bayesian methods, or formulate Bayes’ theorem in function space and discretize only after the algorithm is designed. The latter “Bayesian‑then‑discretize” strategy enjoys mesh‑independence but lacks scalable inference tools.
The authors introduce a functional normalizing flow (FNF) framework that brings the powerful idea of normalizing flows from finite‑dimensional Euclidean spaces to separable Hilbert spaces. The key idea is to start from a tractable prior measure μ₀ (typically a Gaussian) and apply a sequence of invertible, differentiable transformations Tθ. The push‑forward νθ = Tθ♯μ₀ serves as the variational family. To guarantee that the Radon‑Nikodym derivative dνθ/dμ₀ exists (i.e., the transformed and prior measures are mutually absolutely continuous), they formulate three sufficient conditions: (C1) Tθ is a continuous bijection on the Hilbert space, (C2) its Fréchet derivative is a trace‑class operator so that a determinant can be defined, and (C3) the Jacobian determinant is strictly positive. Under these conditions the KL divergence between νθ and the true posterior μ can be written as an expectation over νθ, which can be estimated by Monte‑Carlo sampling from μ₀ and differentiated with respect to θ using automatic differentiation.
Four concrete flow architectures satisfying the conditions are constructed:

1. Functional Householder flow – an infinite‑dimensional reflection that moves the mean while preserving covariance.
2. Functional projected flow – a projection onto a finite‑dimensional subspace followed by a linear map, allowing the flow to focus on dominant prior modes.
3. Functional planar flow – a rank‑one non‑linear perturbation of the form u ↦ u + h(wᵀu)v, with h a scalar non‑linearity; its Jacobian is I + v⊗w h′(·), a trace‑class operator.
4. Functional Sylvester flow – a composition of several rank‑k transformations u ↦ u + Q σ(Rᵀu), where Q and R are Hilbert‑Schmidt operators and σ a smooth activation. Each flow is proved to be invertible and to have a well‑defined Jacobian determinant, ensuring the validity of the variational objective.
   Because evaluating the KL divergence at each iteration can be costly, the authors propose a conditional functional normalizing flow (CNF‑iVI). Measurement data d are fed as a conditioning variable into a neural network that outputs the parameters θ(d) of the flow. After a single offline training phase on a diverse set of synthetic measurements, the network can instantly produce an approximate posterior for any new data vector, regardless of its dimension. If higher accuracy is required, a brief fine‑tuning on the specific datum can be performed.
   Theoretical analysis shows that the proposed flows are discretization‑invariant: the transformation is defined on the continuous function space, and any finite‑element or finite‑difference discretization merely approximates the prior sampling, leaving the variational objective unchanged. This property is demonstrated numerically.
   Three benchmark inverse problems are used to validate the methodology: (i) a simple 1‑D linear PDE, (ii) steady‑state Darcy flow for permeability estimation, and (iii) electrical impedance tomography (EIT). In all cases, NF‑iVI achieves posterior approximations comparable or superior to the preconditioned Crank‑Nicolson (pCN) MCMC sampler, while requiring far fewer forward solves. CNF‑iVI further shows robustness to varying numbers of sensors or measurement dimensions, delivering accurate posteriors without retraining. Metrics such as KL divergence, mean‑square error of the inferred fields, and posterior uncertainty are reported, confirming both efficiency and mesh‑independence.
   In conclusion, the paper delivers a rigorous, infinite‑dimensional normalizing‑flow variational inference framework, supplies four mathematically sound flow constructions, and extends them to a conditional setting that dramatically reduces computational overhead for new data. The work opens several avenues for future research, including non‑Gaussian priors, application to highly non‑linear PDEs (e.g., Navier‑Stokes), and automated design of flow architectures.