Finite-Particle Rates for Regularized Stein Variational Gradient Descent

We derive finite-particle rates for the regularized Stein variational gradient descent (R-SVGD) algorithm introduced by He et al. (2024) that corrects the constant-order bias of the SVGD by applying a resolvent-type preconditioner to the kernelized Wasserstein gradient. For the resulting interacting $N$-particle system, we establish explicit non-asymptotic bounds for time-averaged (annealed) empirical measures, illustrating convergence in the \emph{true} (non-kernelized) Fisher information and, under a $\mathrm{W}_1\mathrm{I}$ condition on the target, corresponding $\mathrm{W}_1$ convergence for a large class of smooth kernels. Our analysis covers both continuous- and discrete-time dynamics and yields principled tuning rules for the regularization parameter, step size, and averaging horizon that quantify the trade-off between approximating the Wasserstein gradient flow and controlling finite-particle estimation error.

💡 Research Summary

This paper provides the first non‑asymptotic finite‑particle convergence guarantees for the Regularized Stein Variational Gradient Descent (R‑SVGD) algorithm introduced by He et al. (2024). The authors start from the Wasserstein gradient flow (WGF) of the Kullback–Leibler (KL) divergence, whose exact dynamics are intractable for empirical measures. Classical SVGD approximates the WGF by replacing the true gradient with a kernel‑induced operator Tₖ,ρ, which introduces a constant‑order bias because Tₖ,ρ is generally non‑invertible and ‖Tₖ,ρ − I‖ₒₚ can be large.

R‑SVGD mitigates this bias by applying a resolvent‑type preconditioner