Frank-Wolfe Algorithms for (L0, L1)-smooth functions

We propose a new version of the Frank-Wolfe method, called the (L0, L1)-Frank-Wolfe algorithm, developed for optimization problems with (L0, L1)-smooth objectives. We establish that this algorithm achieves superior theoretical convergence rates compared to the classical Frank-Wolfe method. In addition, we introduce a novel adaptive procedure, termed the Adaptive (L0, L1)-Frank-Wolfe algorithm, which dynamically adjusts the smoothness parameters to further improve performance and stability. Comprehensive numerical experiments confirm the theoretical results and demonstrate the clear practical advantages of both proposed algorithms over existing Frank-Wolfe variants.

💡 Research Summary

The paper introduces a novel variant of the Frank‑Wolfe (FW) algorithm specifically designed for objective functions that satisfy a (L₀, L₁)‑smoothness condition. Unlike the classical L‑smoothness assumption (‖∇²f(x)‖ ≤ L), (L₀, L₁)‑smoothness allows the Hessian norm to grow linearly with the gradient norm: ‖∇²f(x)‖ ≤ L₀ + L₁‖∇f(x)‖. This broader class includes many machine‑learning losses (e.g., logistic loss, exponential loss, ‖x‖ⁿ) for which the standard Lipschitz constant can be overly conservative.

The authors modify the classic “short‑step” rule of FW. At iteration k they compute the linear minimization oracle (LMO) to obtain sₖ, set the direction dₖ = sₖ − xₖ, and choose a step size

αₖ = min{ 1, −∇f(xₖ)ᵀdₖ /