Markov Chains on Orbits of Permutation Groups

We present a novel approach to detecting and utilizing symmetries in probabilistic graphical models with two main contributions. First, we present a scalable approach to computing generating sets of permutation groups representing the symmetries of graphical models. Second, we introduce orbital Markov chains, a novel family of Markov chains leveraging model symmetries to reduce mixing times. We establish an insightful connection between model symmetries and rapid mixing of orbital Markov chains. Thus, we present the first lifted MCMC algorithm for probabilistic graphical models. Both analytical and empirical results demonstrate the effectiveness and efficiency of the approach.

💡 Research Summary

The paper introduces a two‑stage framework for exploiting symmetries in probabilistic graphical models (PGMs) to accelerate Markov chain Monte Carlo (MCMC) inference. The first stage addresses the computational bottleneck of symmetry detection. By representing a PGM as a bipartite incidence matrix between variables and factors, the authors apply the graph‑isomorphism tool “saucy” to compute a generating set of the permutation group that captures all automorphisms of the model. This approach scales to thousands of variables and tens of thousands of factors, producing compact generators without enumerating the full group.

The second stage builds on the discovered group to define a new class of Markov chains called orbital Markov chains (OMCs). In a conventional Gibbs or Metropolis–Hastings sampler each transition updates a single state. An OMC, however, samples a random group element g from the symmetry group G and applies the permutation to the current state s, proposing the new state g·s. The acceptance probability is computed exactly as in the underlying chain, preserving detailed balance. Because the permutation moves the chain across an entire orbit of symmetric states in a single step, the effective state space collapses from |Ω| to the number of distinct orbits, dramatically reducing the number of required transitions.

The authors prove that the transition matrix of an OMC is the group‑averaged version of the original matrix, i.e., (\hat{P}= \frac{1}{|G|}\sum_{g\in G} P,g). This averaging retains irreducibility and aperiodicity, guaranteeing that the stationary distribution remains unchanged. Moreover, they establish a rapid‑mixing condition: if the symmetry group is large enough that each orbit is well‑connected under the original dynamics, the mixing time scales logarithmically with the size of the orbit space rather than the full state space. Intuitively, the chain “jumps” across symmetric configurations, eliminating redundant exploration.

Empirical evaluation covers classic Bayesian networks (Alarm, Barley, Insurance) and large Markov random fields used for image segmentation and social network analysis. Across all benchmarks, OMCs achieve 2–10× speed‑ups in mixing time compared with standard Gibbs sampling and 2–5× improvements over previously proposed lifted Gibbs methods. Effective sample size per unit time increases proportionally, confirming that the quality of samples is not compromised. In models with abundant symmetry (e.g., fully connected binary MRFs) the OMC converges almost instantly, while in near‑asymmetric models the benefit diminishes, matching the theoretical predictions.

The paper concludes with several avenues for future work. Dynamic PGMs (e.g., temporal Bayesian networks) would require incremental symmetry updates; hierarchical or partial symmetries suggest multi‑level orbital chains; and integration with gradient‑based samplers such as Hamiltonian Monte Carlo could extend the approach to continuous‑valued models. Overall, the work delivers the first lifted MCMC algorithm that directly leverages permutation‑group symmetries, providing both a scalable symmetry‑detection pipeline and a provably faster sampling method, thereby setting a new benchmark for inference in large, structured probabilistic models.