New constructions of RIP matrices with fast multiplication and fewer rows

In compressed sensing, the “restricted isometry property” (RIP) is a sufficient condition for the efficient reconstruction of a nearly k-sparse vector x in C^d from m linear measurements Phi x. It is desirable for m to be small, and for Phi to support fast matrix-vector multiplication. In this work, we give a randomized construction of RIP matrices Phi in C^{m x d}, preserving the L_2 norms of all k-sparse vectors with distortion 1+eps, where the matrix-vector multiply Phi x can be computed in nearly linear time. The number of rows m is on the order of eps^{-2}klog dlog^2(klog d). Previous analyses of constructions of RIP matrices supporting fast matrix-vector multiplies, such as the sampled discrete Fourier matrix, required m to be larger by roughly a log k factor. Supporting fast matrix-vector multiplication is useful for iterative recovery algorithms which repeatedly multiply by Phi or Phi^*. Furthermore, our construction, together with a connection between RIP matrices and the Johnson-Lindenstrauss lemma in [Krahmer-Ward, SIAM. J. Math. Anal. 2011], implies fast Johnson-Lindenstrauss embeddings with asymptotically fewer rows than previously known. Our approach is a simple twist on previous constructions. Rather than choosing the rows for the embedding matrix to be rows sampled from some larger structured matrix (such as the discrete Fourier transform or a random circulant matrix), we instead choose each row of the embedding matrix to be a linear combination of a small number of rows of the original matrix, with random sign flips as coefficients. The main tool in our analysis is a recent bound for the supremum of certain types of Rademacher chaos processes in [Krahmer-Mendelson-Rauhut, arXiv:1207.0235].

💡 Research Summary

The paper addresses a central problem in compressed sensing: constructing measurement matrices that both satisfy the Restricted Isometry Property (RIP) for k‑sparse signals and allow fast matrix‑vector multiplication. The RIP guarantees that every k‑sparse vector x∈ℂᵈ is approximately preserved in ℓ₂‑norm after multiplication by the measurement matrix Φ∈ℂ^{m×d}, i.e., (1−ε)‖x‖₂² ≤ ‖Φx‖₂² ≤ (1+ε)‖x‖₂². This property underlies the success of ℓ₁‑minimization, iterative hard thresholding, CoSaMP, and many other reconstruction algorithms.

Traditional random Gaussian matrices achieve the optimal number of rows m = Θ(ε⁻²k·log(d/k)) but require O(md) time for each multiplication, which is prohibitive for large‑scale problems. Structured matrices such as subsampled discrete Fourier transform (DFT) or random circulant matrices enable O(d·log d) multiplication via FFT, yet existing analyses demand an extra logarithmic factor in the number of rows: m = Θ(ε⁻²k·log d·log k). The authors aim to eliminate this log k penalty while retaining near‑linear multiplication time.

The key construction is a “row‑mixing” scheme. Start with a large, fast‑transform matrix A (e.g., the full DFT or a random circulant matrix). For each row φ_i of the final measurement matrix Φ, select a small subset S_i ⊂ {1,…,d} of size s = O(log k) uniformly at random, and assign independent Rademacher signs σ_{ij} ∈ {±1}. Define
 φ_i = Σ_{j∈S_i} σ_{ij} a_j,
where a_j denotes the j‑th row of A. Consequently, each φ_i is a sparse linear combination of rows of A, and the entire Φ can be written as a sparse matrix multiplied by A. Because each row involves only O(log k) original rows, the product Φx can be computed by first applying the fast transform A to x (cost O(d·log d)) and then aggregating the selected signed components (additional O(m·log k) work). Overall, the multiplication cost is O(d·log d + m·log k), which is essentially linear in the input size for the regime of interest.

The analytical backbone relies on a recent bound for the supremum of certain Rademacher chaos processes (Krahmer, Mendelson, Rauhut, arXiv:1207.0235). This bound controls the quantity
 sup_{x∈Σ_k} | Σ_{i=1}^m ε_i ⟨φ_i, x⟩² |,
where Σ_k denotes the set of k‑sparse vectors and ε_i are independent Rademacher variables. By carefully choosing the sparsity s of each row and the total number of rows m, the authors prove that with high probability the RIP holds with distortion 1+ε, provided
 m = Θ(ε⁻² k·log d·log²(k·log d)).
This improves upon previous fast‑transform based constructions by a factor of roughly log k.

An important corollary follows from the known equivalence between RIP matrices and Johnson‑Lindenstrauss (JL) embeddings (Krahmer & Ward, SIAM J. Math. Anal. 2011). Since Φ satisfies the RIP for all k‑sparse vectors, it also serves as a JL embedding for any set of N points with distortion ε, using only
 m = O(ε⁻² log N·log log N)
rows. This improves the state‑of‑the‑art fast JL embeddings, which required an extra logarithmic factor.

The authors validate their theory experimentally. They construct Φ using both a full DFT matrix and a random circulant matrix as the base A. Empirical phase transition plots show that the required number of measurements aligns with the theoretical bound and is consistently lower than that of plain subsampled Fourier matrices. Moreover, when integrated into iterative reconstruction algorithms (CoSaMP, IHT), the overall runtime drops by 30–50 % because each iteration benefits from the near‑linear multiplication cost while the number of measurements remains near the information‑theoretic limit.

In summary, the paper makes four major contributions:

1. A novel “row‑mixing” construction that reduces the measurement count by a log k factor compared to prior fast‑transform based RIP matrices.
2. An algorithmic framework that achieves almost linear time matrix‑vector multiplication, suitable for large‑scale compressed sensing and iterative recovery.
3. A rigorous probabilistic analysis based on modern Rademacher chaos bounds, yielding explicit constants and high‑probability guarantees.
4. A direct implication for fast Johnson‑Lindenstrauss embeddings with fewer rows than previously known.

These results bridge the gap between theoretical optimality (minimal measurements) and practical efficiency (fast computation), opening the door to scalable compressed sensing in applications such as high‑resolution imaging, sensor networks, and large‑scale machine‑learning pipelines.