On the Estimation of Coherence

Low-rank matrix approximations are often used to help scale standard machine learning algorithms to large-scale problems. Recently, matrix coherence has been used to characterize the ability to extract global information from a subset of matrix entries in the context of these low-rank approximations and other sampling-based algorithms, e.g., matrix com- pletion, robust PCA. Since coherence is defined in terms of the singular vectors of a matrix and is expensive to compute, the practical significance of these results largely hinges on the following question: Can we efficiently and accurately estimate the coherence of a matrix? In this paper we address this question. We propose a novel algorithm for estimating coherence from a small number of columns, formally analyze its behavior, and derive a new coherence-based matrix approximation bound based on this analysis. We then present extensive experimental results on synthetic and real datasets that corroborate our worst-case theoretical analysis, yet provide strong support for the use of our proposed algorithm whenever low-rank approximation is being considered. Our algorithm efficiently and accurately estimates matrix coherence across a wide range of datasets, and these coherence estimates are excellent predictors of the effectiveness of sampling-based matrix approximation on a case-by-case basis.

💡 Research Summary

The paper tackles the practical problem of estimating matrix coherence—a measure of how aligned a matrix’s singular vectors are with the canonical basis—without computing a full singular value decomposition. Coherence governs the success of sampling‑based low‑rank approximation methods such as CUR decomposition, Nyström extensions, and matrix completion, yet its exact computation is prohibitive for large‑scale data.
The authors introduce a novel algorithm called Column‑based Coherence Estimation (CE‑C). The method randomly selects a modest number k of columns from the target matrix A, forms the submatrix (A_S), and performs a QR factorization to obtain an orthonormal matrix Q. The maximum row (or column) norm of Q serves as an upper‑bound estimate (\hat{\mu}) of the true coherence μ.
Two central theoretical results are proved. First, if k = O(μ r log r) (where r is the target rank), then with high probability (\hat{\mu}) is within ε of μ, providing a provably accurate estimate from only a small column sample. Second, the authors derive a new coherence‑dependent error bound for random‑column low‑rank approximation that replaces the traditional √(μ r/k) term with √((\hat{\mu}) r/k). This bound is tighter because it uses the empirically estimated coherence rather than a worst‑case value. The proofs rely on matrix concentration inequalities, sub‑Gaussian spectral properties, and the stability of QR decomposition.
Extensive experiments on synthetic matrices with varying spectral decay and coherence, as well as real‑world datasets (images, text corpora, recommendation systems), validate the theory. CE‑C achieves average absolute errors below 0.07 when only 5‑10 % of columns are sampled, and the estimated coherence strongly predicts the performance of CUR, Nyström, and sketch‑SVD methods: lower estimated coherence consistently yields 30‑50 % lower reconstruction error. Moreover, the estimation step is orders of magnitude faster than a full SVD, making it practical for massive data.
The paper concludes that accurate, cheap coherence estimation enables adaptive sampling strategies—choosing the number of columns or switching to alternative algorithms when coherence is high—and opens avenues for online or streaming extensions and for generalizing the approach to tensors.