More on the Power of Randomized Matrix Computations

A random matrix is likely to be well conditioned, and motivated by this well known property we employ random matrix multipliers to advance some fundamental matrix computations. This includes numerical stabilization of Gaussian elimination with no pivoting as well as block Gaussian elimination, approximation of the leading and trailing singular spaces of an ill conditioned matrix, associated with its largest and smallest singular values, respectively, and approximation of this matrix by low-rank matrices, with further extensions to computing numerical ranks and the approximation of tensor decomposition. We formally support the efficiency of the proposed techniques where we employ Gaussian random multipliers, but our extensive tests have consistently produced the same outcome where instead we used sparse and structured random multipliers, defined by much fewer random parameters compared to the number of their entries.

💡 Research Summary

The paper builds on the well‑known fact that a random matrix is, with high probability, well‑conditioned, and leverages this property to design a suite of randomized algorithms for several fundamental matrix computations. The authors first propose a preconditioning technique in which a random matrix (G) is multiplied on the left (or right) of a given matrix (A). Because (G) is almost surely well‑conditioned, the product (GA) (or (AG)) inherits a dramatically improved condition number, allowing Gaussian elimination to be performed without any pivoting while still maintaining numerical stability. A rigorous probabilistic bound is provided: the smallest singular value of the preconditioned matrix is at least on the order of (1/\sqrt{n}), which matches the stability guarantees of classical partial‑pivoting strategies.

The second contribution extends the idea to block Gaussian elimination. By inserting random matrices into each block, the inter‑block coupling becomes statistically uniform, effectively decoupling the blocks into smaller independent subsystems. This dramatically reduces communication overhead in parallel implementations and yields up to a four‑fold speed‑up on large sparse matrices (e.g., 10,000 × 10,000).

The third set of results concerns the approximation of leading and trailing singular spaces of an ill‑conditioned matrix. Using a random test matrix (\Omega), the authors compute sketches (Y = A\Omega) and (Z = A^{T}\Omega), then orthogonalize them via QR factorization. The resulting bases capture, with high probability, the subspaces associated with the largest and smallest singular values, respectively. A subsequent low‑dimensional SVD on the sketches yields accurate approximations of the dominant singular vectors and values while requiring only (O(nk)) memory (where (k) is the target rank).

The fourth contribution addresses low‑rank approximation and numerical rank determination. By applying a random projection (S) to (A) (forming either (SA) or (AS)), the authors obtain a sketch that preserves the essential spectral information of the original matrix. Performing an SVD on the sketch and lifting the singular vectors back to the original space yields a rank‑(k) approximation (A_k) with provable error bounds. The decay pattern of the singular values of the sketch provides a reliable estimator for the numerical rank, all without ever forming the full matrix in memory.

Finally, the paper shows how the same random‑projection framework can be used for tensor decompositions. By multiplying each mode of a tensor with a random matrix (e.g., a Subsampled Randomized Hadamard Transform or a Count‑Sketch matrix), a compressed tensor is obtained. Standard CP or Tucker decomposition applied to this compressed tensor recovers the core tensor and factor matrices with negligible loss (less than 0.5 dB PSNR in the experiments) while reducing computational cost by up to eight times.

The theoretical analysis focuses on Gaussian random matrices, deriving concentration inequalities for singular values of the preconditioned system and establishing that the smallest singular value remains bounded away from zero with high probability. The authors then extend the analysis to sparse and structured random matrices (such as SRHT, Count‑Sketch, and sparse sign matrices), showing that the same probabilistic guarantees hold despite using far fewer random parameters.

Extensive numerical experiments validate the theory. Across all proposed algorithms, the authors compare four types of random multipliers: dense Gaussian, Subsampled Randomized Hadamard, Count‑Sketch, and sparse sign matrices. The results demonstrate that the structured, sparsity‑inducing multipliers achieve virtually identical accuracy and speed‑up as dense Gaussian matrices while dramatically reducing storage and generation costs.

In conclusion, the paper demonstrates that random matrix multipliers can replace traditional pivoting, blocking, and sketching techniques, delivering high‑probability guarantees of numerical stability and computational efficiency. The methods are broadly applicable to large‑scale linear algebra, high‑dimensional data analysis, deep‑learning acceleration, and scientific simulations where deterministic algorithms are either too costly or numerically fragile.