The arithmetic complexity of tensor contractions

We investigate the algebraic complexity of tensor calulus. We consider a generalization of iterated matrix product to tensors and show that the resulting formulas exactly capture VP, the class of polynomial families efficiently computable by arithmetic circuits. This gives a natural and robust characterization of this complexity class that despite its naturalness is not very well understood so far.

💡 Research Summary

The paper investigates the algebraic‑complexity landscape of tensor calculus and establishes a precise correspondence between a natural class of tensor‑contraction formulas and the well‑studied arithmetic‑circuit class VP (the family of polynomial‑size, polynomial‑degree circuits). The authors begin by observing that the traditional characterizations of VP—via arithmetic circuits, algebraic branching programs, or iterated matrix products—while powerful, each capture only a limited view of the underlying computational model. In particular, iterated matrix multiplication is essentially a two‑dimensional operation and does not readily generalize to higher‑order tensors that appear in many modern applications (quantum physics, machine learning, etc.).

To bridge this gap, the authors define a tensor‑contraction operation. Given two tensors A and B and a set S of matching index positions, the contraction ∑{s∈S} A{…s…}·B_{…s…} simultaneously multiplies and sums over the chosen dimensions, thereby reducing the total order of the resulting tensor. A tensor‑contraction formula is a sequence of such operations applied to a collection of input tensors (variables or constants). The size of a formula is measured by the total number of entries in all intermediate tensors, which is required to be polynomial in the input size.

The central result (Theorem 1) states that a family of polynomials {f_n} belongs to VP if and only if there exists a family of polynomial‑size tensor‑contraction formulas that computes f_n exactly. The proof proceeds in two directions:

1. From VP to tensor‑contraction. Starting from an arithmetic circuit C of size s(n), each gate (addition or multiplication) is simulated by a small tensor. The authors embed the circuit’s wiring into the index structure of tensors: each wire becomes an index, and each gate corresponds to a contraction that merges the appropriate indices. Because each gate has constant fan‑in, the resulting tensors have bounded order, and the total number of entries grows only polynomially. Consequently, the whole circuit can be flattened into a sequence of O(s(n)) contractions, yielding a polynomial‑size formula.

2. From tensor‑contraction to VP. Given a contraction formula, the authors construct a standard arithmetic circuit that mimics each contraction step. An index of a tensor is represented by a set of wires; a contraction over a set S is realized by a layer of multiplication gates followed by addition gates that sum over the values of S. Since each contraction reduces the order by at most |S|, the depth contributed by a single contraction is bounded, and the total number of gates is polynomial in the size of the formula. Hence the formula can be simulated by a polynomial‑size circuit, placing the computed polynomial inside VP.

Beyond the equivalence, the paper emphasizes robustness: the class of functions computable by tensor‑contraction formulas is closed under natural transformations such as reordering of contraction steps, merging consecutive contractions on the same indices, or inserting dummy dimensions. This invariance shows that the characterization does not depend on a particular syntactic representation, addressing a common criticism of circuit‑based definitions that they may be overly sensitive to low‑level implementation details.

The authors also compare their model with the classical iterated matrix product characterization. While matrix products correspond to a special case where all tensors are 2‑dimensional, tensor contraction subsumes this case and can express computations that would require exponentially many matrix multiplications if forced into a purely two‑dimensional framework. Consequently, subclasses such as VBP (bounded‑width branching programs) embed naturally as restrictions on the width of intermediate tensors, and the authors sketch how the same technique could be adapted to explore relationships with VNP.

Finally, the paper outlines several open directions. One line of inquiry concerns depth vs. width trade‑offs within tensor‑contraction formulas: how does limiting the number of simultaneous contracted indices affect the expressive power? Another asks whether the tensor‑contraction viewpoint can yield new lower‑bound techniques for separating VP from VNP, perhaps by exploiting known hardness results for tensor rank. A third direction suggests practical algorithmic applications: can the explicit tensor‑contraction representation be leveraged to design faster algorithms for specific families of polynomials (e.g., determinants, permanents, or graph‑homomorphism polynomials) in the same way that fast matrix multiplication algorithms have impacted circuit design?

In summary, the paper provides a clean, mathematically elegant characterization of VP via tensor‑contraction formulas, demonstrating that this higher‑order generalization of iterated matrix multiplication captures exactly the class of efficiently computable polynomials. The result not only enriches our theoretical understanding of arithmetic complexity but also opens a promising bridge between complexity theory and the tensor‑network methods prevalent in physics and data science.