Convergence Rates of Inexact Proximal-Gradient Methods for Convex Optimization

We consider the problem of optimizing the sum of a smooth convex function and a non-smooth convex function using proximal-gradient methods, where an error is present in the calculation of the gradient of the smooth term or in the proximity operator with respect to the non-smooth term. We show that both the basic proximal-gradient method and the accelerated proximal-gradient method achieve the same convergence rate as in the error-free case, provided that the errors decrease at appropriate rates.Using these rates, we perform as well as or better than a carefully chosen fixed error level on a set of structured sparsity problems.

💡 Research Summary

The paper addresses the composite convex optimization problem f(x)=g(x)+h(x) where g is a smooth convex function with an L‑Lipschitz continuous gradient and h is a possibly nonsmooth, lower‑semicontinuous convex regularizer. Classical proximal‑gradient (PG) and accelerated proximal‑gradient (APG) methods assume exact computation of both the gradient of g and the proximal operator of h. In many large‑scale applications (total‑variation, overlapping group ℓ₁, nuclear‑norm, etc.) the proximal step cannot be solved analytically and must be approximated, and stochastic or inexact gradient evaluations are also common. The authors therefore introduce two error sequences: eₖ for the gradient error and εₖ for the inexactness of the proximal subproblem, defined by the inequality L/2‖xₖ−yₖ‖²+h(xₖ) ≤ εₖ+minₓ {L/2‖x−yₖ‖²+h(x)}.

The core contribution consists of four propositions that give explicit convergence bounds for both PG and APG under these errors, for the general convex case and for the strongly convex case (μ‑strong convexity of g).

Proposition 1 (PG, convex) shows that the average iterate satisfies
  f( \bar{x}_k )−f(x*) ≤ (L/2k)‖x₀−x*‖² +