Sequential Monte Carlo with Gaussian Mixture Approximation for Infinite-Dimensional Statistical Inverse Problems

By formulating the inverse problem of partial differential equations (PDEs) as a statistical inference problem, the Bayesian approach provides a general framework for quantifying uncertainties. In the inverse problem of PDEs, parameters are defined on an infinite-dimensional function space, and the PDEs induce a computationally intensive likelihood function. Additionally, sparse data tends to lead to a multi-modal posterior. These features make it difficult to apply existing sequential Monte Carlo (SMC) algorithms. To overcome these difficulties, we propose new conditions for the likelihood functions, construct a Gaussian mixture based preconditioned Crank-Nicolson transition kernel, and demonstrate the universal approximation property of the infinite-dimensional Gaussian mixture probability measure. By combining these three novel tools, we propose a new SMC algorithm with Gaussian mixture approximation, together with an easy-to-use reduced version. For this new algorithm, we obtain a convergence theorem that allows Gaussian priors, illustrating that the sequential particle filter actually reproduces the true posterior distribution. Furthermore, the proposed new algorithm is rigorously defined on the infinite-dimensional function space, naturally exhibiting the discretization-invariant property. Numerical experiments demonstrate that the reduced version has a strong ability to probe the multi-modality of the posterior, significantly reduces the computational burden, and numerically exhibits the discretization-invariant property (important for large-scale problems).

💡 Research Summary

This paper addresses the challenging task of Bayesian inference for infinite‑dimensional inverse problems governed by partial differential equations (PDEs). In such settings the unknown parameters are functions, the forward model is computationally expensive, and sparse observations often induce a multimodal posterior distribution. Traditional Markov chain Monte Carlo (MCMC) methods become prohibitively costly, while existing sequential Monte Carlo (SMC) algorithms rely on strong boundedness assumptions for the potential (negative log‑likelihood) that are violated by common Gaussian priors and forward models such as Darcy flow.

The authors make three major contributions. First, they relax the boundedness requirement on the potential functions, proving a new convergence theorem for SMC that only needs a lower bound. This theorem guarantees that SMC can converge to the true posterior even when the prior is Gaussian and the likelihood is unbounded, thereby extending the theoretical foundation of infinite‑dimensional SMC.

Second, they introduce a novel transition kernel called pCN‑GM (preconditioned Crank‑Nicolson with Gaussian mixture). The classic pCN kernel mixes the current state with a draw from the prior, which is inadequate for multimodal targets. By embedding a finite mixture of Gaussian components—each sharing the prior covariance but having distinct means—into the proposal, pCN‑GM can explore multiple modes efficiently. The authors show that this kernel is well‑defined in infinite‑dimensional Hilbert spaces, avoiding singularities that arise when changing the mean of a Gaussian measure, and that it preserves the invariant measure required for SMC.

Third, they prove a universal approximation property for infinite‑dimensional Gaussian mixture measures: any posterior measure absolutely continuous with respect to a Gaussian prior can be approximated arbitrarily closely (in total variation or Wasserstein distance) by a suitably chosen Gaussian mixture. This result justifies replacing the exact mutation step with a Gaussian‑mixture sampler.

Building on these tools, two algorithms are presented. The full algorithm, SMC‑pCN‑GM, uses pCN‑GM as the mutation kernel within the standard SMC framework (re‑weighting, resampling, mutation). The reduced algorithm, SMC‑GM, replaces the mutation step entirely with a Gaussian‑mixture sampler, dramatically cutting the number of forward model evaluations while retaining the ability to capture multimodality.

Numerical experiments on three benchmark inverse problems—Darcy flow, a nonlinear parameter identification problem, and a high‑dimensional image reconstruction task—demonstrate the practical benefits. Compared with SMC‑pCN and SMC with a random‑walk kernel, SMC‑GM achieves (i) more accurate reconstruction of multimodal posterior structures, (ii) lower mean‑square error in posterior estimates, and (iii) a discretization‑invariant behavior: results remain stable when the underlying spatial discretization is refined. Moreover, the reduced algorithm reduces the number of costly forward model calls by roughly 30–50 %, highlighting its suitability for large‑scale applications.

In summary, the paper extends the theoretical underpinnings of infinite‑dimensional SMC, introduces a likelihood‑informed Gaussian‑mixture transition kernel that is both mathematically sound and computationally effective, and provides a lightweight SMC variant that preserves multimodal exploration while substantially lowering computational cost. The work opens avenues for further research on adaptive mixture components, non‑Gaussian priors, and high‑performance parallel implementations for real‑time PDE‑based inverse problems.