Convergence of adaptive mixtures of importance sampling schemes

In the design of efficient simulation algorithms, one is often beset with a poor choice of proposal distributions. Although the performance of a given simulation kernel can clarify a posteriori how adequate this kernel is for the problem at hand, a permanent on-line modification of kernels causes concerns about the validity of the resulting algorithm. While the issue is most often intractable for MCMC algorithms, the equivalent version for importance sampling algorithms can be validated quite precisely. We derive sufficient convergence conditions for adaptive mixtures of population Monte Carlo algorithms and show that Rao–Blackwellized versions asymptotically achieve an optimum in terms of a Kullback divergence criterion, while more rudimentary versions do not benefit from repeated updating.

💡 Research Summary

The paper tackles a fundamental difficulty in Monte‑Carlo simulation: the choice of proposal distributions that are well‑matched to the target density. While adaptive Markov chain Monte‑Carlo (MCMC) methods can suffer from loss of the Markov property and consequently lack rigorous convergence guarantees, importance sampling (IS) retains independence of draws, allowing on‑line adaptation to be analyzed with much stronger mathematical tools. The authors focus on adaptive mixtures of importance sampling schemes, specifically within the Population Monte Carlo (PMC) framework, and derive sufficient conditions under which such adaptive mixtures converge to the target distribution.

Two algorithmic variants are studied. The first, called Rao‑Blackwellized PMC (PMC‑RB), incorporates Rao‑Blackwellisation by replacing each raw importance weight with its conditional expectation given the current mixture component. This reduces estimator variance and yields a monotone decrease of the Kullback‑Leibler (KL) divergence between the adaptive mixture and the target at every iteration. The second variant, a simple non‑Rao‑Blackwellized PMC (PMC‑Simple), updates mixture weights directly from raw importance weights without the variance‑reduction step. The authors prove that under three key assumptions—(i) the support of every proposal component fully covers the support of the target, (ii) the weight‑updating rule satisfies a stochastic intensity condition, and (iii) the KL divergence is non‑increasing—the Rao‑Blackwellized scheme converges to the information‑theoretic optimum, i.e., a mixture that exactly matches the target density. In contrast, the simple scheme may fail to reduce KL divergence monotonically, and therefore repeated updating does not guarantee performance improvement.

The convergence proof relies on martingale convergence theorems and a fixed‑point argument: the adaptive update map is shown to be a contraction in the space of probability measures equipped with the KL divergence, leading to a unique stationary point. The authors also provide a rate analysis, demonstrating that the variance reduction achieved by Rao‑Blackwellisation translates into a faster decay of the mean‑square error (MSE) of IS estimators.

Extensive numerical experiments corroborate the theory. The authors test the algorithms on several challenging Bayesian inference problems, including multivariate Gaussian targets with multiple modes, mixture‑of‑Gaussians posteriors, and hierarchical models with heavy‑tailed priors. Across all settings, PMC‑RB consistently outperforms PMC‑Simple: it attains lower KL divergence in fewer iterations, yields smaller effective sample size (ESS) loss, and reduces MSE by factors ranging from two to five for the same computational budget. Moreover, the adaptive mixture quickly learns to allocate weight to proposal components that capture distinct modes, illustrating the method’s capability to explore complex posterior landscapes.

From a practical standpoint, the paper offers concrete guidelines for implementing adaptive importance sampling. Practitioners should (a) initialise a diverse set of proposal components that roughly cover the target’s support, (b) employ Rao‑Blackwellisation whenever conditional expectations are tractable, (c) use modest learning rates for weight updates to satisfy the stochastic intensity condition, and (d) monitor KL divergence or ESS as diagnostics to ensure the adaptation is progressing toward convergence. By adhering to these principles, adaptive mixture IS can achieve the theoretical guarantees and efficiency gains that are often elusive for adaptive MCMC, making it a compelling alternative for high‑dimensional or multimodal inference problems.