An Exponential Lower Bound on the Sub-Packetization of MSR Codes

An $(n,k,\ell)$-vector MDS code is a $\mathbb{F}$-linear subspace of $(\mathbb{F}^\ell)^n$ (for some field $\mathbb{F}$) of dimension $k\ell$, such that any $k$ (vector) symbols of the codeword suffice to determine the remaining $r=n-k$ (vector) symbols. The length $\ell$ of each codeword symbol is called the sub-packetization of the code. Such a code is called minimum storage regenerating (MSR), if any single symbol of a codeword can be recovered by downloading $\ell/r$ field elements (which is known to be the least possible) from each of the other symbols. MSR codes are attractive for use in distributed storage systems, and by now a variety of ingenious constructions of MSR codes are available. However, they all suffer from exponentially large sub-packetization $\ell \gtrsim r^{k/r}$. Our main result is an almost tight lower bound showing that for an MSR code, one must have $\ell \ge \exp(\Omega(k/r))$. This settles a central open question concerning MSR codes that has received much attention. Previously, a lower bound of $\approx \exp(\sqrt{k/r})$, and a tight lower bound for a restricted class of “optimal access” MSR codes, were known.

💡 Research Summary

The paper addresses a central open problem in the theory of minimum‑storage regenerating (MSR) codes: how small can the sub‑packetization level ℓ (the length of the vector stored at each node) be for an (n, k, ℓ) MSR code? An (n, k, ℓ) vector MDS code stores kℓ field symbols across n nodes, and the MSR property requires that any single failed node can be repaired by downloading exactly ℓ/(n − k) symbols from each of the remaining n − 1 nodes, which is the information‑theoretic optimum (the cut‑set bound). Existing explicit MSR constructions achieve ℓ on the order of r^{k/r} (where r = n − k), which is exponential in k when r is constant, and all known lower bounds were far weaker (ℓ ≥ exp(Θ(√k/r))).

The authors introduce a new algebraic framework called an “MSR subspace family”. From any (n, k, ℓ) MSR code they extract (k − 1)·ℓ/r‑dimensional subspaces H_i ⊂ 𝔽^ℓ together with r − 1 linear maps φ_{i, j} that satisfy strong invariance properties: each φ_{i, j} fixes all H_{i′} for i′ ≠ i, maps H_i to a disjoint subspace, and leaves the rest of the space invariant. The existence of such a family translates the combinatorial repair requirement into a purely linear‑algebraic constraint on collections of subspaces and linear transformations.

The core technical contribution is a counting argument that bounds the dimension of the space of linear maps that simultaneously preserve a given collection of subspaces. As the number of subspaces grows, this dimension shrinks exponentially. By quantifying this decay, the authors derive the inequality

 ℓ > exp ((k − 1)(r − 1) / (2r²))

which simplifies to ℓ ≥ exp(Ω(k/r)). This lower bound is essentially tight: the best known constructions achieve ℓ ≈ r^{k/r}, which matches the bound up to the base of the exponent. The result therefore shows that the large sub‑packetization observed in all existing MSR codes is not an artifact of current constructions but an inherent limitation.

In addition to the main exponential bound, the paper improves auxiliary results. It shows an O(r log ℓ) upper bound on the size of any MSR subspace family, sharpening earlier O(r log² ℓ) bounds. It also provides an alternative construction of a subspace family that works over any field with more than two elements, avoiding the huge field size required in prior work. The authors discuss the special case r = 1 (trivial MSR code with ℓ = 1) and note that their theorem excludes this degenerate case, consistent with known results.

The paper situates its contribution within a broad literature on regenerating codes, optimal‑access MSR codes, and related secret‑sharing schemes. It clarifies that while optimal‑access codes admit a stronger lower bound ℓ ≥ r^{k/r}, the present work extends a comparable exponential barrier to the general linear MSR setting where helper nodes may transmit arbitrary linear combinations.

Finally, the authors outline future directions: tightening the constant factors in the exponent (potentially achieving ℓ ≥ exp(c·k/r) with c = 1), exploring whether non‑linear or multi‑failure repair models can bypass the bound, and investigating whether the subspace‑family framework can be leveraged to design new MSR codes that approach the lower bound more closely. In summary, the paper settles a long‑standing conjecture by proving that any MSR code must have sub‑packetization exponential in k/r, thereby establishing a fundamental trade‑off between repair bandwidth optimality and storage granularity in distributed storage systems.