Score-based Metropolis-Hastings for Fractional Langevin Algorithms

Sampling from heavy-tailed and multimodal distributions is challenging when neither the target density nor the proposal density can be evaluated, as in $α$-stable Lévy-driven fractional Langevin algorithms. While the target distribution can be estimated from data via score-based or energy-based models, the $α$-stable proposal density and its score are generally unavailable, rendering classical density-based Metropolis–Hastings (MH) corrections impractical. Consequently, existing fractional Langevin methods operate in an unadjusted regime and can exhibit substantial finite-time errors and poor empirical control of tail behavior. We introduce the Metropolis-Adjusted Fractional Langevin Algorithm (MAFLA), an MH-inspired, fully score-based correction mechanism. MAFLA employs designed proxies for fractional proposal score gradients under isotropic symmetric $α$-stable noise and learns an acceptance function via Score Balance Matching. We empirically illustrate the strong performance of MAFLA on a series of tasks including combinatorial optimization problems where the method significantly improves finite time sampling accuracy over unadjusted fractional Langevin dynamics.

💡 Research Summary

The paper tackles the long‑standing difficulty of applying Metropolis–Hastings (MH) corrections to fractional Langevin dynamics driven by symmetric α‑stable Lévy noise. In such settings neither the target density nor the proposal density is tractable; the α‑stable proposal has no closed‑form density, and the target is often represented only by a score (∇ log p) learned via score‑based or energy‑based models. Consequently, existing fractional Langevin samplers (collectively called FULA) operate in an unadjusted regime, suffering from substantial finite‑step bias, poor tail‑behavior control, and extreme sensitivity to the step size.

The authors propose the Metropolis‑Adjusted Fractional Langevin Algorithm (MAFLA), an MH‑inspired correction that relies exclusively on score information. The key technical contributions are twofold. First, they exploit the location‑family structure of isotropic symmetric α‑stable distributions to derive a tractable approximation of the proposal score. For a proposal of the form
 x′ = x + τ \tilde b(x) + τ^{1/α} ξ, ξ ∼ SαS(1),
they define r = x′ − x − τ \tilde b(x) and show that, for α ∈ (1, 2] and small step size τ, the gradient of the log‑proposal can be approximated by ∇_{x′} log q(x′|x) ≈ −κ r with κ = 1/(α c_α τ). This first‑order approximation is exact in the limit τ → 0 and provides a closed‑form surrogate for the otherwise intractable proposal score.

Second, they adapt Score Balance Matching (SBM), a recently introduced score‑based learning objective, to enforce detailed balance at the gradient level. They parameterize an acceptance function a_φ(x′,x) and minimize an SBM loss that penalizes deviations from the gradient‑based detailed‑balance condition, using only the learned target score s_θ(x) and the approximated proposal score. Because the loss depends solely on scores, no evaluation of p(x) or q(x′|x) is required.

Theoretical analysis shows that MAFLA’s bias scales as O(τ²) compared with the O(τ) bias of unadjusted FULA, and that the acceptance function learned via SBM guarantees that the resulting Markov chain satisfies detailed balance with respect to the target distribution in the limit of accurate score approximations. Computationally, the method remains O(d) per iteration, as the proposal score approximation is linear in dimension.

Empirical evaluation spans synthetic heavy‑tailed mixtures (2‑D and 10‑D), image data perturbed with α‑stable noise, and combinatorial optimization problems (MaxCut and Minimum Vertex Cover) solved via continuous relaxations. Across all benchmarks, MAFLA consistently reduces KL divergence, improves effective sample size, and yields tighter tail‑distribution coverage relative to FULA. In the combinatorial tasks, MAFLA’s enhanced exploration from Lévy jumps combined with the MH‑style correction leads to higher‑quality solutions and greater robustness to step‑size choices.

In summary, the paper introduces a fully score‑based MH correction for fractional Langevin samplers, demonstrating that density‑free MH adjustments are feasible and beneficial. This opens the door to exact‑invariant MCMC methods for a broad class of models where only scores are available, including non‑normalized energy‑based models and other Lévy‑driven stochastic processes. Future directions include extending the framework to asymmetric or heterogeneous Lévy noises, adaptive step‑size schemes, and applications to manifold‑constrained sampling.