Faster Algorithms for Rectangular Matrix Multiplication

Let {\alpha} be the maximal value such that the product of an n x n^{\alpha} matrix by an n^{\alpha} x n matrix can be computed with n^{2+o(1)} arithmetic operations. In this paper we show that \alpha>0.30298, which improves the previous record \alpha>0.29462 by Coppersmith (Journal of Complexity, 1997). More generally, we construct a new algorithm for multiplying an n x n^k matrix by an n^k x n matrix, for any value k\neq 1. The complexity of this algorithm is better than all known algorithms for rectangular matrix multiplication. In the case of square matrix multiplication (i.e., for k=1), we recover exactly the complexity of the algorithm by Coppersmith and Winograd (Journal of Symbolic Computation, 1990). These new upper bounds can be used to improve the time complexity of several known algorithms that rely on rectangular matrix multiplication. For example, we directly obtain a O(n^{2.5302})-time algorithm for the all-pairs shortest paths problem over directed graphs with small integer weights, improving over the O(n^{2.575})-time algorithm by Zwick (JACM 2002), and also improve the time complexity of sparse square matrix multiplication.

💡 Research Summary

The paper “Faster Algorithms for Rectangular Matrix Multiplication” by François Le Gall advances the state of the art in the algebraic complexity of matrix multiplication, focusing on the rectangular case. The author studies the exponent ω(1,1,k), defined as the smallest τ such that multiplying an n × ⌊n^k⌋ matrix by an ⌊n^k⌋ × n matrix can be done in O(n^τ) arithmetic operations, and the related parameter α = sup {k | ω(1,1,k)=2}. Previously, Coppersmith (1997) had shown α > 0.29462. Le Gall improves this lower bound to α > 0.30298, meaning that for any ε > 0 one can multiply an n × n^{0.30298} matrix by an n^{0.30298} × n matrix using O(n^{2+ε}) operations.

The technical contribution rests on a refined use of the Coppersmith‑Winograd tensor framework. The classic approach uses a basic trilinear form F_q (parameter q) and, through Schöenhage’s asymptotic sum inequality, derives upper bounds on ω. Le Gall replaces the basic construction with its tensor square F_q ⊗ F_q, and then systematically analyzes this construction for rectangular products. By formulating the resulting complexity as a non‑linear optimization problem over parameters such as the tensor dimensions, splitting ratios, and recursion depth, the author obtains concrete numerical bounds for many values of k.

The paper presents a table of upper bounds for ω(1,1,k) across a wide range of k, both below and above 1. For 0 ≤ k ≤ 1 the new bounds are dramatically better than those obtainable from the simple linear interpolation formula ω(1,1,k) ≤ α·2 + (ω−2)·(k−α)/(1−α). For example:

• ω(1,1,0.5) < 2.04668,
• ω(1,1,0.8) < 2.22479,
• ω(1,1,1.0) < 2.37547 (matching the classic Coppersmith‑Winograd bound).

For k > 1 the algorithm also outperforms previous rectangular multiplication results (e.g., Huang‑Pan, Ke‑Zeng‑Han‑Pan). Notably, for k = 2 the new bound is ω(1,1,2) < 3.25668, improving on the prior best of 3.2699.

The author then translates these theoretical improvements into concrete algorithmic speed‑ups for several well‑studied problems that rely on rectangular matrix multiplication:

1. All‑Pairs Shortest Paths (APSP) with small integer weights – Zwick’s reduction yields a running time O(n^{2+μ}) where μ satisfies ω(1,1,μ) = 1 + 2μ. Using the new bound ω(1,1,0.5302) < 2.0604 gives μ < 0.5302, leading to an O(n^{2.5302}) randomized algorithm, improving over Zwick’s O(n^{2.575}) algorithm.

2. Sparse Square Matrix Multiplication – Yuster and Zwick showed that the product of two n × n matrices with at most m non‑zero entries can be computed in O(min{nm, n^{ω(1,1,λ_m)} , n^{ω}}) time, where λ_m solves λ_m + ω(1,1,λ_m) = 2 log_n m. Plugging the new ω(1,1,k) values yields, for instance, a complexity of O(n^{2.087}) when m ≈ n^{4/3}, improving on the previous O(n^{2.1293}) bound.

3. Dynamic Transitive Closure – The same rectangular multiplication bounds improve the update time for maintaining the transitive closure of a directed graph under edge insertions/deletions, though the paper sketches this application rather than providing a full analysis.

The paper also discusses how the new bounds can be combined with the latest best known bound for square matrix multiplication, ω < 2.3727 (Vassilevska Williams 2012), to obtain marginal improvements for k close to 1 via convexity arguments, though these gains are small.

In the methodological section, Le Gall explains the optimization framework in detail. The analysis starts from the fourth tensor power F_q^{⊗4} (the “Coppersmith‑Winograd” tensor) and proceeds to its square, deriving a system of linear constraints that capture the asymptotic sum inequality. By solving this system numerically (using linear programming and a fine‑grained search over q and other parameters), the author extracts the concrete exponents reported. The approach generalizes the earlier “laser method” and “group‑theoretic” techniques, showing that higher tensor powers can be harnessed for rectangular products without incurring prohibitive overhead.

Finally, the paper outlines future directions: exploring higher tensor powers (e.g., F_q^{⊗r} for r > 2) to push α even higher, refining the numerical optimization to reduce constants, and integrating the rectangular algorithms with practical fast matrix multiplication schemes (such as Strassen or practical Winograd variants) to mitigate the large hidden constants that currently limit practical applicability.

In summary, Le Gall’s work delivers a significant theoretical breakthrough by raising the lower bound on α to 0.30298 and providing a unified algorithmic framework that yields the best known exponents for rectangular matrix multiplication across all k. This directly translates into faster algorithms for APSP with small integer weights, sparse matrix multiplication, and dynamic graph problems, and it opens a promising avenue for further reductions in the matrix multiplication exponent.