Distributed Estimation of Generalized Matrix Rank: Efficient Algorithms and Lower Bounds

We study the following generalized matrix rank estimation problem: given an $n \times n$ matrix and a constant $c \geq 0$, estimate the number of eigenvalues that are greater than $c$. In the distributed setting, the matrix of interest is the sum of $m$ matrices held by separate machines. We show that any deterministic algorithm solving this problem must communicate $\Omega(n^2)$ bits, which is order-equivalent to transmitting the whole matrix. In contrast, we propose a randomized algorithm that communicates only $\widetilde O(n)$ bits. The upper bound is matched by an $\Omega(n)$ lower bound on the randomized communication complexity. We demonstrate the practical effectiveness of the proposed algorithm with some numerical experiments.

💡 Research Summary

The paper addresses the problem of estimating the generalized rank of a large positive semidefinite matrix in a distributed setting. The generalized rank rank(A,c) is defined as the number of eigenvalues of A that exceed a given threshold c≥0. In many modern data‑analysis tasks—such as principal component analysis, robust PCA, collaborative filtering, and randomized numerical linear algebra—the ability to quickly approximate this quantity without computing a full eigendecomposition is crucial, especially when the matrix is too large to reside on a single machine.

The authors consider a model where the target matrix A∈ℝ^{n×n} is expressed as a sum of m locally stored matrices A_i, i.e., A = Σ_{i=1}^m A_i. Each machine i holds A_i and can perform arbitrary local computation, but communication between machines is costly. The goal is to output an integer \hat r that approximates rank(A,c) within a relative error δ, i.e., (1−δ)·rank(A,c₁) ≤ \hat r ≤ (1+δ)·rank(A,c₂) for user‑specified thresholds c₁>c₂≥0.

Deterministic Lower Bound
The paper first proves that any deterministic protocol that guarantees such an approximation must communicate essentially the entire matrix. By constructing a two‑party rank‑testing problem (RankTest) in which one party holds A₁ and the other A₂, and the sum A either has rank r or has rank between 6r/5 and 2r with a guaranteed spectral gap, they show that solving RankTest requires Ω(r·n) bits. Since r can be Θ(n) in the worst case, the deterministic communication complexity becomes Ω(n²), matching the cost of transmitting the full matrix. This lower bound holds even when the matrices are well‑conditioned and approximation error is allowed, making it stronger than previous results that required exact computation or singular matrices.

Randomized Upper Bound
To break this barrier, the authors design a randomized algorithm that reduces communication to near‑linear in n. The key idea is to replace the step function that counts eigenvalues above c with a smooth polynomial approximation. Define H_{c₁,c₂}(x) as a piecewise‑linear function that equals 1 for x>c₁, 0 for x<c₂, and interpolates linearly in between. Its square, H²_{c₁,c₂}, provides upper and lower bounds on the generalized rank: rank(A,c₁) ≤ Σ_i H²_{c₁,c₂}(σ_i(A)) ≤ rank(A,c₂).

The algorithm seeks a low‑degree polynomial f that uniformly approximates H_{c₁,c₂} on