Polyak's Heavy Ball Method Achieves Accelerated Local Rate of Convergence under Polyak-Lojasiewicz Inequality

In this work, we analyze the convergence of Polyak’s heavy ball method in both continuous and discrete time for non-convex $C^4$-objective functions satisfying the Polyak-Lojasiewicz inequality. Under this weak assumption, we recover the asymptotic convergence rates originally derived by Polyak in [Polyak, U.S.S.R. Comput. Math. and Math. Phys., 1964] for strongly convex objectives. Our results demonstrate that the heavy ball method exhibits asymptotic local acceleration on this class of functions. In particular, in the discrete time setting, we prove local convergence of the iterates to a minimum once the method enters a sufficiently small neighborhood of the set of minima, for a broad range of hyperparameters, including aggressive choices for the momentum parameter and the step-size for which global convergence is known to fail. Instead of the usually employed Lyapunov-type arguments, our approach leverages a new differential geometric perspective of the Polyak-Lojasiewicz inequality proposed in [Rebjock and Boumal, Math. Program., 2025].

💡 Research Summary

This paper provides a comprehensive convergence analysis of Polyak’s heavy‑ball method in both continuous and discrete time for non‑convex (C^{4}) objective functions that satisfy the Polyak‑Łojasiewicz (PL) inequality. The authors show that the PL condition—a much weaker assumption than strong convexity—is sufficient to recover the original asymptotic convergence rates derived by Polyak in 1964.

Continuous‑time results.
Consider the heavy‑ball ordinary differential equation (ODE)
\