Irreversible Monte Carlo Algorithms for Efficient Sampling

arXiv:0809.0916v2 [cond-mat.stat-mech] 23 Sep 2008 Irreversible Monte Carlo Algorithms for Efficient Sampling Konstantin S. Turitsyn(a,b,c), Michael Chertkov(b,d), Marija Vucelja(d,b) a James Frank Institute, University of Chicago, Chicago, IL 60615, USA b Center for Nonlinear Studies & Theoretical Division, LANL, Los Alamos, NM 87545, USA c Landau Institute for Theoretical Physics, Moscow 142432, Russia d Department of Physics of Complex Systems, Weizmann Institute of Sciences, Rehovot 76100, Israel Equilibrium systems evolve according to Detailed Balance (DB). This principe guided development of the Monte-Carlo sampling techniques, of which Metropolis-Hastings (MH) algorithm is the famous representative. It is also known that DB is sufficient but not necessary. We construct irreversible deformation of a given reversible algorithm capable of dramatic improvement of sampling from known distribution. Our transformation modifies transition rates keeping the structure of transitions intact. To illustrate the general scheme we design an Irreversible version of Metropolis-Hastings (IMH) and test it on example of a spin cluster. Standard MH for the model suffers from the critical slowdown, while IMH is free from critical slowdown. PACS numbers: 02.70.Tt, 05.10.Ln, 64.75.Ef Recent decades have been marked by fruitful interaction be- tween physics and computer science, with one of the most striking examples of such interaction going back to 40s when physicists proposed a Markov Chain Monte Carlo (MCMC) algorithm [1, 2]. MCMC evaluates large sums, or integrals, approximately, in a sense imitating how nature would do effi- cient sampling itself. Development of this idea has became wide spread and proliferated a great variety of disciplines. (See [3, 4, 5] for a sample set of reviews in physics and com- puter science.) If one formally follows the letter of the orig- inal MCMC suggestion one ought to ensure that the Detailed Balance (DB) condition is satisfied. This condition reflects microscopic reversibility of the underlying equilibrium dy- namics. A reader, impressed with indisputable success of the reversible MCMC techniques, may still wonder if the equilib- rium dynamics is the most efficient strategy for sampling and evaluating the integrals? In this letter we argue that typically the answer is NO. Let us try to illustrate the ideas on a sim- ple everyday life example. Consider mixing sugar in a cup of coffee, which is similar to sampling, as long as the sugar particles have to explore the entire interior of the cup. DB dynamics corresponds to diffusion taking an enormous mix- ing time. This is certainly not the best way to mix. More- over, our everyday experience suggests a better solution – enhance mixing with a spoon. Spoon steering generates an out-of-equilibrium external flow which significantly acceler- ates mixing, while achieving the same final result – uniform distribution of sugar concentration over the cup. In this letter we show constructively, with a practical algorithm suggested, that similar strategy can be used to decrease mixing time of any known reversible MCMC algorithms. There are two main obstacles which prevent fast mixing by traditional MCMC methods. First, the effective energy land- scape can have high barriers, separating the energy minima. In this case mixing time is dominated by rare processes of over- coming the barriers. Second, slow mixing can originate from the high entropy of the states basin (too many comparably im- portant states) providing major contribution to the system par- tition function. In the later case mixing time is determined by the number of steps it takes for reversible (diffusive) random walk to explore all the relevant states. MCMC algorithms are best described on discrete example. Consider a graph with vertices i = 1, . . . , N each labeling a state of the system and edges (i →j) corresponding to “allowed” transitions between the states. For instance, an N- dimensional hypercube corresponds to a system of N spins (with N = 2N states) with single-spin flips allowed. An MCMC algorithm can be described in terms of the transition matrix Tij representing the probability of a single MCMC step from state j to state i. Probability of finding the system in state i at time t, P t i , evolves according to the following Mas- ter Equation (ME): P t+1 i = P j TijP t j . Stationary solution of ME, P t i = πi, satisfies the Balance Condition (BC): X j (Tijπj −Tjiπi) = 0. (1) Qij = Tijπj from the lhs of Eq. (1) can be interpreted as the stationary probability flux from state i to state j. Obviously, stationarity of the probability flow reads as the condition for incoming and outgoing fluxes at any state to sum up to zero. Note also, that Eq. (1) is nothing but incompressibility condi- tion of the stationary probability flow. The DB used in traditional MCMC algorithms is a more stringent condition, as it requires the piecewise balance of terms in the sum (1): for any pair of states with allowed

…(Full text truncated)…

This content is AI-processed based on ArXiv data.