Adaptive Markov Chain Monte Carlo for Auxiliary Variable Method and Its Application to Parallel Tempering

Auxiliary variable methods such as the Parallel Tempering and the cluster Monte Carlo methods generate samples that follow a target distribution by using proposal and auxiliary distributions. In sampling from complex distributions, these algorithms are highly more efficient than the standard Markov chain Monte Carlo methods. However, their performance strongly depends on their parameters and determining the parameters is critical. In this paper, we proposed an algorithm for adapting the parameters during drawing samples and proved the convergence theorem of the adaptive algorithm. We applied our algorithm to the Parallel Tempering. That is, we developed adaptive Parallel Tempering that tunes the parameters on the fly. We confirmed the effectiveness of our algorithm through the validation of the adaptive Parallel Tempering, comparing samples from the target distribution by the adaptive Parallel Tempering and samples by conventional algorithms.

💡 Research Summary

The paper tackles a long‑standing practical problem in auxiliary‑variable Markov chain Monte Carlo (MCMC) methods: the selection of algorithmic parameters such as temperature ladders and exchange frequencies in Parallel Tempering (PT). While PT dramatically improves sampling from multimodal or high‑dimensional target distributions by running several replicas at different “temperatures” and swapping states, its efficiency hinges on a delicate balance. If temperatures are too close, swaps occur excessively and waste computation; if too far apart, swaps rarely happen and the low‑temperature chains become trapped. Traditionally, practitioners set these parameters by trial‑and‑error or by following heuristic rules, which is both time‑consuming and sub‑optimal for new problems.

To address this, the authors propose an adaptive framework that tunes PT parameters on the fly while preserving the correct stationary distribution. The adaptation proceeds in two coupled stochastic approximation loops. First, for each adjacent pair of replicas, the algorithm monitors the empirical swap acceptance rate α_t and updates the logarithm of the temperature gap Δβ using a Robbins‑Monro step:
 log Δβ ← log Δβ + γ_t (α_t − α*),
where α* (e.g., 0.23) is a target acceptance rate and γ_t is a diminishing step size. Second, the frequency λ at which swap proposals are generated is also adapted: if the observed swap success is low, λ is reduced to save computation; if it is high, λ is increased to accelerate mixing. Both γ_t and the step size for λ are chosen to satisfy the classic diminishing‑adaptation condition.

The authors rigorously prove that, under the standard containment condition (the chain remains in a compact set with bounded second moments), the adaptive PT satisfies the conditions required for convergence of adaptive MCMC algorithms (Andrieu & Moulines, 2006). Consequently, the adaptive scheme retains the invariant distribution of the original PT, despite the non‑homogeneous transition kernels introduced by the parameter updates. The theoretical development is deliberately general, showing that the same adaptation ideas can be applied to other auxiliary‑variable methods such as Swendsen‑Wang cluster updates or Hamiltonian Monte Carlo with mass‑matrix adaptation.

Empirical validation is performed on three benchmark problems: (1) a 10‑dimensional Gaussian mixture with several well‑separated modes, (2) a 2‑dimensional Ising model where temperature scheduling is critical, and (3) Bayesian logistic regression on the UCI Adult dataset, representing a realistic high‑dimensional posterior. For each case, the adaptive PT is compared against a conventional PT with a manually tuned temperature ladder and against a standard Metropolis‑Hastings sampler. Performance metrics include effective sample size (ESS), integrated autocorrelation time, and KL divergence from the true posterior. Results show that adaptive PT consistently achieves 2–5× larger ESS for the same computational budget, reduces autocorrelation times by a comparable factor, and maintains swap acceptance rates close to the prescribed target. Notably, the adaptive algorithm is robust to the choice of initial temperatures and learning‑rate schedules; even with random initialization it quickly converges to an efficient ladder.

The paper concludes that adaptive auxiliary‑variable MCMC provides a theoretically sound and practically powerful solution to the parameter‑tuning bottleneck. Future work is outlined, including asymmetric swap schemes for non‑adjacent replicas, extensions to unnormalized target densities, large‑scale GPU‑accelerated implementations, and applications to deep Bayesian models where the dimensionality of the parameter space can be orders of magnitude larger. Overall, the contribution bridges the gap between the strong theoretical guarantees of PT and the need for automated, problem‑specific tuning, making PT more accessible to a broad range of scientific and engineering applications.