Convergence of Multi-Level Markov Chain Monte Carlo Adaptive Stochastic Gradient Algorithms

Stochastic optimization in learning and inference often relies on Markov chain Monte Carlo (MCMC) to approximate gradients when exact computation is intractable. However, finite-time MCMC estimators are biased, and reducing this bias typically comes at a higher computational cost. We propose a multilevel Monte Carlo gradient estimator whose bias decays as $O(T_{n}^{-1} )$ while its expected computational cost grows only as $O(log T_n )$, where $T_n$ is the maximal truncation level at iteration n. Building on this approach, we introduce a multilevel MCMC framework for adaptive stochastic gradient methods, leading to new multilevel variants of Adagrad and AMSGrad algorithms. Under conditions controlling the estimator bias and its second and third moments, we establish a convergence rate of order $O(n^{-1/2} )$ up to logarithmic factors. Finally, we illustrate these results on Importance-Weighted Autoencoders trained with the proposed multilevel adaptive methods.

💡 Research Summary

The paper tackles a fundamental difficulty in stochastic optimization when gradients must be estimated via Markov chain Monte Carlo (MCMC): finite‑time MCMC estimators are biased, and reducing this bias traditionally requires long chains, leading to prohibitive computational cost. The authors introduce a Multi‑Level Monte Carlo (MLMC) gradient estimator that achieves a bias of order $O(T_n^{-1})$ while the expected per‑iteration cost grows only as $O(\log T_n)$. The estimator is built by sampling a geometric level $K_n\sim G(1/2)$, setting $t_n=2^{K_n}$, and forming a telescoping sum of differences between fine‑ and coarse‑level chain averages, truncated at a maximal level $T_n$.

Algorithm 1 embeds this estimator into a generic adaptive stochastic gradient update \