Self-normalized Cramér-type Moderate Deviation of Stochastic Gradient Langevin Dynamics

In this paper, we study the self-normalized Cramér-type moderate deviation of the empirical measure of the stochastic gradient Langevin dynamics (SGLD). Consequently, we also derive the Berry-Esseen bound for SGLD. Our approach is by constructing a stochastic differential equation (SDE) to approximate the SGLD and then applying Stein’s method as developed in [9,19], to decompose the empirical measure into a martingale difference series sum and a negligible remainder term.

💡 Research Summary

The paper investigates the self‑normalized Cramér‑type moderate deviation (SNCMD) and the Berry‑Esseen bound for the empirical measure generated by Stochastic Gradient Langevin Dynamics (SGLD). The authors consider a non‑convex stochastic loss function ψ(ω,ζ) defined on ℝ^d×ℝ^r, with ζ drawn from a distribution ν. The population objective is P(ω)=E_ζ