A cautionary tale on the efficiency of some adaptive Monte Carlo schemes

There is a growing interest in the literature for adaptive Markov chain Monte Carlo methods based on sequences of random transition kernels ${P_n}$ where the kernel $P_n$ is allowed to have an invariant distribution $\pi_n$ not necessarily equal to the distribution of interest $\pi$ (target distribution). These algorithms are designed such that as $n\to\infty$, $P_n$ converges to $P$, a kernel that has the correct invariant distribution $\pi$. Typically, $P$ is a kernel with good convergence properties, but one that cannot be directly implemented. It is then expected that the algorithm will inherit the good convergence properties of $P$. The equi-energy sampler of [Ann. Statist. 34 (2006) 1581–1619] is an example of this type of adaptive MCMC. We show in this paper that the asymptotic variance of this type of adaptive MCMC is always at least as large as the asymptotic variance of the Markov chain with transition kernel $P$. We also show by simulation that the difference can be substantial.

💡 Research Summary

The paper investigates a class of adaptive Markov chain Monte Carlo (MCMC) algorithms in which the transition kernel at iteration n, denoted Pₙ, is allowed to have its own invariant distribution πₙ that need not coincide with the target distribution π. The design principle behind these schemes is that as n → ∞ the sequence of kernels converges to a limiting kernel P that does have π as its invariant distribution. In many applications P is theoretically attractive—it possesses a large spectral gap, rapid mixing, or other desirable convergence properties—but it cannot be implemented directly. The hope is that the adaptive algorithm will inherit the good asymptotic behaviour of P while using a tractable, data‑driven approximation during the early stages.

The authors focus on the asymptotic variance of ergodic averages, a key measure of long‑run efficiency. For a test function f, the central limit theorem for a stationary Markov chain with kernel P yields an asymptotic variance σ²_P(f). They derive the analogous variance σ²_A(f) for the adaptive chain {Xₙ} generated by the sequence {Pₙ}. Their main theoretical result is an inequality  σ²_A(f) ≥ σ²_P(f), with equality only when the adaptive kernels become identical to P after a finite number of steps. The proof rests on a careful decomposition of the covariance structure of the adaptive process, showing that the additional randomness introduced by the adaptation step contributes a non‑negative term to the overall variance. In other words, the adaptive scheme cannot improve upon the variance of the ideal, non‑adaptive chain and will typically be worse.

To illustrate the practical impact of this result, the paper examines the equi‑energy (EE) sampler introduced by Kou, Zhou and Wong (Ann. Statist., 2006). The EE sampler constructs a hierarchy of “energy levels,” each associated with its own Markov chain, and periodically proposes swaps between chains at different levels. The algorithm is designed to facilitate global exploration by allowing the chain to jump between distant modes of the target distribution. The authors express the EE transition kernel explicitly and compare it with the limiting kernel P that would be obtained if swaps were always accepted.

Their analysis reveals a crucial weakness: when the probability of accepting swaps between levels is low—a situation that commonly occurs in high‑dimensional or multimodal targets—the EE sampler effectively behaves as a collection of nearly independent sub‑chains. The lack of inter‑level mixing inflates autocorrelations dramatically, leading to a substantial increase in asymptotic variance. The theoretical inequality therefore predicts, and the simulations confirm, that the EE sampler’s variance can be several times larger than that of a well‑tuned Metropolis–Hastings chain using the same computational budget.

The simulation study includes a one‑dimensional bimodal Gaussian mixture and a five‑dimensional mixture of multivariate normals. For each problem the authors run (i) a standard Metropolis–Hastings algorithm with a proposal tuned to achieve an optimal acceptance rate, and (ii) the EE sampler with various numbers of energy levels and swap frequencies. All runs are allocated the same total number of iterations and the same CPU time. Results show that the effective sample size per unit time for the EE sampler is markedly lower; the mean‑squared error of estimated expectations can be 2–10 times larger than that of the baseline Metropolis–Hastings chain. Even when the swap acceptance probability is artificially increased, the gain in efficiency is modest, indicating that the fundamental variance inflation stems from the adaptive structure itself rather than from a sub‑optimal choice of swap parameters.

In conclusion, the paper delivers three key messages. First, the intuitive belief that an adaptive algorithm automatically inherits the good convergence properties of its limiting kernel is unfounded; the adaptation introduces extra stochasticity that can only increase asymptotic variance. Second, asymptotic variance should be a primary diagnostic when evaluating adaptive MCMC methods, not just mixing time or spectral gap considerations. Third, for algorithms such as the equi‑energy sampler that rely on inter‑level exchanges, practitioners must ensure that exchange probabilities are sufficiently high; otherwise the method may perform substantially worse than a simple non‑adaptive chain. The authors recommend that developers of adaptive MCMC schemes conduct rigorous variance‑based comparisons with appropriate non‑adaptive baselines before deploying these methods in practice.