A Unifying Analysis of Projected Gradient Descent for $ell_p$-constrained Least Squares

In this paper we study the performance of the Projected Gradient Descent(PGD) algorithm for $\ell_{p}$-constrained least squares problems that arise in the framework of Compressed Sensing. Relying on the Restricted Isometry Property, we provide convergence guarantees for this algorithm for the entire range of $0\leq p\leq1$, that include and generalize the existing results for the Iterative Hard Thresholding algorithm and provide a new accuracy guarantee for the Iterative Soft Thresholding algorithm as special cases. Our results suggest that in this group of algorithms, as $p$ increases from zero to one, conditions required to guarantee accuracy become stricter and robustness to noise deteriorates.

💡 Research Summary

This paper investigates the performance of Projected Gradient Descent (PGD) for ℓₚ‑constrained least‑squares problems that arise in compressed sensing, covering the entire range 0 ≤ p ≤ 1. The authors formulate the estimation task as minimizing the squared error ½‖Ax − y‖₂² subject to an ℓₚ‑ball constraint ‖x‖ₚᵖ ≤ c. By interpreting the classic Iterative Hard Thresholding (IHT) algorithm (p = 0) and the Iterative Soft Thresholding (IST) algorithm (p = 1) as special cases of ℓₚ‑PGD, they provide a unified analytical framework.

The core of the analysis relies on the Restricted Isometry Property (RIP) of the measurement matrix A. Rather than the usual symmetric RIP constants, the paper adopts an asymmetric formulation with lower and upper bounds βₛ and αₛ such that βₛ‖x‖₂² ≤ ‖Ax‖₂² ≤ αₛ‖x‖₂² for all s‑sparse vectors x. From these constants the authors define a normalized quantity ρₛ = (αₛ − βₛ)/(αₛ + βₛ), which plays the role of the conventional δₛ. They also introduce a function ξ(p) = √p·√(2 − p) − ½ − 1/p that monotonically increases with p and captures how the non‑convexity of the feasible set influences convergence.

The main result (Theorem 2.1) states that if A satisfies a 3s‑RIP with constants α₍₃ₛ₎, β₍₃ₛ₎, the feasible ℓₚ‑ball radius is chosen as b_c = (1 − ε)^{1/p}‖x*‖ₚ (ε ∈