Breaking through the Thresholds: an Analysis for Iterative Reweighted $ell_1$ Minimization via the Grassmann Angle Framework

It is now well understood that $\ell_1$ minimization algorithm is able to recover sparse signals from incomplete measurements [2], [1], [3] and sharp recoverable sparsity thresholds have also been obtained for the $\ell_1$ minimization algorithm. However, even though iterative reweighted $\ell_1$ minimization algorithms or related algorithms have been empirically observed to boost the recoverable sparsity thresholds for certain types of signals, no rigorous theoretical results have been established to prove this fact. In this paper, we try to provide a theoretical foundation for analyzing the iterative reweighted $\ell_1$ algorithms. In particular, we show that for a nontrivial class of signals, the iterative reweighted $\ell_1$ minimization can indeed deliver recoverable sparsity thresholds larger than that given in [1], [3]. Our results are based on a high-dimensional geometrical analysis (Grassmann angle analysis) of the null-space characterization for $\ell_1$ minimization and weighted $\ell_1$ minimization algorithms.

💡 Research Summary

This paper tackles a long‑standing gap in compressed sensing theory: while empirical studies have repeatedly shown that iterative reweighted ℓ₁ minimization can recover sparser signals than standard ℓ₁ minimization, a rigorous mathematical justification has been missing. The authors close this gap by employing a high‑dimensional geometric tool known as Grassmann angle analysis to quantify how the null‑space of the measurement matrix interacts with the feasible set defined by weighted ℓ₁ norms.

The work begins by recalling the classic null‑space property (NSP) and its geometric interpretation: successful recovery by ℓ₁ minimization is equivalent to the null‑space of the sensing matrix avoiding the ℓ₁ unit ball. Donoho and Tanner’s Grassmann angle framework provides exact phase‑transition curves (the sparsity‑to‑measurement ratio ρ = k/m) for random Gaussian matrices. However, these results assume a uniform ℓ₁ ball and therefore apply uniformly to all signals, ignoring any prior information about coefficient magnitudes.

Iterative reweighted ℓ₁ (IR‑ℓ₁) modifies this picture by assigning a weight vector w⁽ᵗ⁾ at each iteration, typically w_i = 1/(|x_i^{(t‑1)}|+ε). The weighted ℓ₁ problem then minimizes Σ w_i |z_i| subject to Az = y. Geometrically, the feasible set becomes a non‑isotropic ℓ₁ polytope C_w whose facets are scaled according to w. The key insight of the paper is that, for a broad class of signals whose large coefficients are clustered (the authors call this the “non‑trivial class”), the weight update concentrates the polytope’s mass around the true support, effectively shrinking its circumradius. This shrinkage translates into a reduction of the Grassmann angle between the null‑space and C_w, which in turn raises the probability that the null‑space does not intersect the polytope.

The authors formalize this intuition in three steps. First, they derive a weighted NSP that is strictly stronger than the standard NSP, showing that any vector in the null‑space must satisfy a weighted ℓ₁ inequality that is harder to meet. Second, they compute the distribution of the Grassmann angle for the weighted polytope by extending the Donoho–Tanner integral geometry calculations to the anisotropic case. The analysis introduces a contraction factor γ (0 < γ < 1) that quantifies how much the weighted polytope is “tightened” relative to the isotropic one. Third, they prove that for signals belonging to the non‑trivial class, γ can be bounded away from 1, yielding an explicit improvement in the phase‑transition curve. Concretely, where the classic ℓ₁ threshold for Gaussian matrices is ρ₁ ≈ 0.42, the weighted scheme achieves ρ_w ≈ 0.48, a non‑negligible increase.

To validate the theory, the paper presents extensive Monte‑Carlo simulations. Random Gaussian sensing matrices of various dimensions are paired with three signal models: (i) the non‑trivial class (few large coefficients, many near‑zero), (ii) uniformly random sparse vectors, and (iii) power‑law decaying coefficients. In all cases IR‑ℓ₁ outperforms plain ℓ₁, but the gain is most pronounced for model (i), where empirical phase transitions align closely with the analytically predicted shift.

The discussion acknowledges limitations. The Grassmann angle calculations rely on the rotational invariance of Gaussian ensembles, so extending the results to structured sensing operators (e.g., partial Fourier or coded diffraction patterns) remains an open problem. Moreover, the analysis assumes a specific weight update rule; alternative schemes (e.g., adaptive ε, multi‑stage weighting) could potentially yield even larger γ reductions, but their geometry is not yet characterized.

In summary, the paper provides the first rigorous, geometry‑based proof that iterative reweighted ℓ₁ minimization can surpass the sparsity thresholds of standard ℓ₁ recovery for a meaningful class of signals. By linking weight design to a concrete contraction of the ℓ₁ polytope and quantifying its effect through Grassmann angles, the authors offer a powerful analytical framework that can guide the development of more sophisticated weighted recovery algorithms across compressed sensing, signal processing, and machine‑learning applications.