Parallel Heuristic Exploration for Additive Complexity Reduction in Fast Matrix Multiplication

This paper presents a parallel random-search method for reducing additive complexity in fast matrix multiplication algorithms with ternary coefficients ${-1,0,1}$. The approach replaces expensive exact evaluation with fast heuristic scoring, including the new Greedy-Intersections strategy. The method runs many independent common subexpression elimination processes in parallel, exploring the search space through random pair substitutions and diverse selection strategies while sharing promising partial solutions. Tested on 149 ternary-coefficient schemes, the method achieves lower addition counts than the state-of-the-art Greedy-Potential on 102 schemes (including 57 new best-known results for optimal-rank schemes), matches it on 45, and is outperformed on only 2. For most schemes, it provides equal or better results while being significantly faster, making it practical for algorithm exploration. All software and results are open source.

💡 Research Summary

The paper addresses the problem of minimizing the additive (addition/subtraction) cost of fast matrix multiplication algorithms that use only ternary coefficients (‑1, 0, 1). While low‑rank schemes reduce the number of scalar multiplications, the actual runtime is often dominated by the number of additions required to form the linear combinations of input matrix entries and to reconstruct the output. This additive cost can be reduced by Common Subexpression Elimination (CSE), a combinatorial optimization problem that is NP‑hard for the dense expression graphs typical of fast matrix multiplication schemes.

The authors propose a parallel heuristic search framework that runs many independent CSE processes concurrently, each guided by a different selection strategy for choosing which pair of variables (x_i ± x_j) to substitute next. The core CSE loop iteratively (1) enumerates all canonical pairs appearing in the current set of linear expressions, (2) selects a pair according to a heuristic, (3) introduces a fresh variable representing that pair, and (4) rewrites all occurrences. The loop stops when no pair appears more than once.

Four basic strategies are implemented: Greedy (always pick the most frequent pair), Greedy‑Alternative (randomly pick among the most frequent), Weighted‑Random (probability proportional to immediate gain c‑1), and Greedy‑Random (a mixture of the two). The novel contribution is the Greedy‑Intersections (gi) heuristic, which estimates the future benefit of a candidate pair without performing a full trial substitution. For a candidate pair s_p with frequency c_sp, the score is

H(s_p) = (c_sp − 1) + α · ∑_{s_q} I(s_p, s_q),

where α ∈