On sparse representations of linear operators and the approximation of matrix products

Thus far, sparse representations have been exploited largely in the context of robustly estimating functions in a noisy environment from a few measurements. In this context, the existence of a basis in which the signal class under consideration is sparse is used to decrease the number of necessary measurements while controlling the approximation error. In this paper, we instead focus on applications in numerical analysis, by way of sparse representations of linear operators with the objective of minimizing the number of operations needed to perform basic operations (here, multiplication) on these operators. We represent a linear operator by a sum of rank-one operators, and show how a sparse representation that guarantees a low approximation error for the product can be obtained from analyzing an induced quadratic form. This construction in turn yields new algorithms for computing approximate matrix products.

💡 Research Summary

**
The paper introduces a novel framework for sparsifying linear operators with the explicit goal of reducing the computational cost of matrix multiplication. While sparse representations have traditionally been employed in signal processing and compressed sensing—where a signal is assumed to be sparse in a suitable basis to lower the number of required measurements—the authors repurpose the concept for numerical analysis. They model any linear operator (A\in\mathbb{R}^{m\times n}) as a sum of rank‑one components obtained from its singular value decomposition (SVD): \