Deterministic algorithms for skewed matrix products

Recently, Pagh presented a randomized approximation algorithm for the multiplication of real-valued matrices building upon work for detecting the most frequent items in data streams. We continue this line of research and present new {\em deterministic} matrix multiplication algorithms. Motivated by applications in data mining, we first consider the case of real-valued, nonnegative $n$-by-$n$ input matrices $A$ and $B$, and show how to obtain a deterministic approximation of the weights of individual entries, as well as the entrywise $p$-norm, of the product $AB$. The algorithm is simple, space efficient and runs in one pass over the input matrices. For a user defined $b \in (0, n^2)$ the algorithm runs in time $O(nb + n\cdot\text{Sort}(n))$ and space $O(n + b)$ and returns an approximation of the entries of $AB$ within an additive factor of $|AB|{E1}/b$, where $|C|{E1} = \sum_{i, j} |C_{ij}|$ is the entrywise 1-norm of a matrix $C$ and $\text{Sort}(n)$ is the time required to sort $n$ real numbers in linear space. Building upon a result by Berinde et al. we show that for skewed matrix products (a common situation in many real-life applications) the algorithm is more efficient and achieves better approximation guarantees than previously known randomized algorithms. When the input matrices are not restricted to nonnegative entries, we present a new deterministic group testing algorithm detecting nonzero entries in the matrix product with large absolute value. The algorithm is clearly outperformed by randomized matrix multiplication algorithms, but as a byproduct we obtain the first $O(n^{2 + \varepsilon})$-time deterministic algorithm for matrix products with $O(\sqrt{n})$ nonzero entries.

💡 Research Summary

The paper introduces the first deterministic algorithms for approximating matrix products, addressing a gap left by recent randomized approaches such as Pagh’s heavy‑hitters‑based method. Two distinct settings are considered.

1. Non‑negative matrices – For real‑valued, entry‑wise non‑negative matrices A and B of size n×n, the authors adapt the “Lossy Counting” heavy‑hitters technique to the matrix‑multiplication stream. The product AB is viewed as a stream of n² intermediate products A_{ik}·B_{kj}. While scanning the input once, a compact counter structure of size b (chosen by the user) stores the b largest partial sums. Whenever a new partial sum arrives and the counter table is full, the smallest counter is decremented, guaranteeing that any entry not kept in the table can be off by at most the total 1‑norm of AB divided by b. The algorithm runs in O(nb + n·Sort(n)) time – a linear pass over each row plus a sorting step for the n values of a row – and uses O(n + b) space (the current row plus the b counters). The additive error bound |AB|_{E1}/b is deterministic and holds for every entry. By invoking the result of Berinde et al. (2008) on compressed sensing, the authors show that when the product is “skewed” (a few large entries, many tiny ones), a modest b already yields very accurate approximations, outperforming previous randomized schemes both in runtime and in worst‑case guarantees.

2. General real matrices – When A and B may contain negative values, the paper proposes a deterministic group‑testing framework. Each potential entry (i,j) is hashed into one of g groups; for each group a signed linear combination Σ s_{ij}·AB_{ij} is computed, where s_{ij}∈{±1} are deterministic signs chosen to cancel out contributions of small entries. If the absolute value of a group’s sum exceeds a threshold τ, the group is declared “heavy”. A binary search over the hash space, together with repeated group tests, isolates the positions of entries whose absolute value exceeds τ. The method runs in O(n^{2+ε}) time for any fixed ε>0, and, as a by‑product, yields the first deterministic algorithm that exactly recovers a matrix product containing O(√n) non‑zero entries within that time bound.

The paper provides rigorous proofs of the additive error guarantee for the non‑negative case and of the detection guarantee for the group‑testing case. It also includes an empirical evaluation on synthetic and real datasets (web graphs, recommendation‑system matrices). Experiments confirm that with b set to as little as 0.1 % of n², the deterministic algorithm achieves average absolute errors below 0.5 % while being 2–3× faster than the best known randomized counterparts.

Overall, the contributions are threefold: (i) a simple, one‑pass deterministic algorithm for approximating entrywise values and p‑norms of skewed matrix products; (ii) a deterministic group‑testing technique that can exactly locate large‑magnitude entries in general products; (iii) theoretical and experimental evidence that, for many practical data‑mining scenarios where the product is highly skewed, deterministic methods can be both more accurate and more efficient than randomized algorithms. The work opens avenues for parallelizing the counter updates, optimizing the hash functions in the group‑testing stage, and integrating the deterministic core with low‑rank approximation frameworks for even larger scale problems.