Optimal Subgradient Methods for Lipschitz Convex Optimization with Error Bounds

We study the iteration complexity of Lipschitz convex optimization problems satisfying a general error bound. We show that for this class of problems, subgradient descent with either Polyak stepsizes or decaying stepsizes achieves minimax optimal convergence guarantees for decreasing distance-to-optimality. The main contribution is a novel lower-bounding argument that produces hard functions simultaneously satisfying zero-chain conditions and global error bounds.

💡 Research Summary

This paper presents a rigorous investigation into the iteration complexity of Lipschitz convex optimization problems under the framework of a generalized error bound condition. While much of the existing literature focuses on the convergence of function values, this work shifts the focus to a more fundamental metric: the convergence of the distance to the optimal set, denoted as $|x_k - x^*|$. The authors demonstrate that subgradient descent methods, specifically those employing Polyak step-sizes or decaying step-size regimes, achieve the minimax optimal convergence rates for this distance-to-optimality metric within the specified class of problems.

The technical core of the paper lies in the establishment of a tight lower bound, which is essential for proving the minimax optimality of the proposed algorithms. To prove that no algorithm can surpass a certain convergence rate, one must construct a “worst-case” or “hard” function that adheres to the problem’s constraints but is inherently difficult to optimize. The primary challenge addressed by the authors is the simultaneous satisfaction of two distinct structural properties: the “zero-chain condition” and the “global error bound.” The zero-chain condition is a structural property used to create a “flat” landscape near the optimum, making it difficult for first-order methods to identify the precise location of the optimal solution. Conversely, the global error bound imposes a regularity constraint that prevents the function from being arbitrarily pathological.

The authors’ novel contribution is the construction of a mathematical instance that integrates these two properties. By successfully designing a function that is both difficult to optimize (via the zero-chain structure) and constrained by a global error bound, they provide a definitive lower bound on the complexity of the problem. This result effectively closes the gap between the upper bounds provided by subgradient descent and the lower bounds of the problem class, thereby establishing the minimax optimality of the subgradient method.

This research has significant implications for the field of non-smooth optimization. It provides a theoretical ceiling for the performance of first-order methods in settings where strong convexity is absent but error bounds are present. Such settings are ubiquitous in modern machine learning, particularly in training models with non-smooth loss functions or when dealing with large-scale optimization problems where only Lipschitz continuity can be guaranteed. The paper’s findings offer a robust mathematical foundation for designing and evaluating the efficiency of next-generation optimization algorithms.