Fast Product-Matrix Regenerating Codes

Distributed storage systems support failures of individual devices by the use of replication or erasure correcting codes. While erasure correcting codes offer a better storage efficiency than replication for similar fault tolerance, they incur higher CPU consumption, higher network consumption and higher disk I/Os. To address these issues, codes specific to storage systems have been designed. Their main feature is the ability to repair a single lost disk efficiently. In this paper, we focus on one such class of codes that minimize network consumption during repair, namely regenerating codes. We implement the original Product-Matrix Regenerating codes as well as a new optimization we propose and show that the resulting optimized codes allow achieving 790 MB/s for encoding in typical settings. Reported speeds are significantly higher than previous studies, highlighting that regenerating codes can be used with little CPU penalty.

💡 Research Summary

The paper addresses the practical performance limitations of regenerating codes, specifically the Product‑Matrix (PM) construction, when applied to modern distributed storage systems. While erasure correcting codes such as Reed‑Solomon (RS) provide good storage efficiency, they impose high CPU, network, and disk I/O costs, especially during repair operations that require downloading k blocks to reconstruct a single lost block. Regenerating codes, and in particular Minimum‑Storage Regenerating (MSR) and Minimum‑Bandwidth Regenerating (MBR) variants, were introduced to reduce the repair bandwidth, but existing implementations have suffered from low encoding/decoding throughput, making them unsuitable for real‑world deployments.

The authors propose two complementary optimizations that dramatically improve the throughput of PM regenerating codes:

1. Linearization of PM codes – By constructing an explicit generator matrix G from the original encoding matrix Ψ and the index matrix L (which maps the original data blocks into the message matrix M), the authors transform the inherently non‑linear PM algorithm into a standard linear code formulation Y = G·X. This enables the reuse of highly optimized Galois‑field libraries (GF‑Complete) and erasure‑code frameworks (Jerasure 2.0). Linearization also simplifies the creation of systematic codes: the first k·Δ encoded symbols are made identical to the original data, and only the remaining n·α − k·Δ symbols need to be computed, reducing both computational work and memory copies.

2. Sparse matrix construction – The original PM design uses a Vandermonde matrix for Ψ, which is dense. When converted to systematic form, the resulting generator matrix G₀ becomes completely dense (0 % sparsity), causing a large number of unnecessary finite‑field multiplications. The authors replace Φ (the left block of Ψ) with a concatenation of an identity matrix and a Cauchy matrix, while Λ remains a diagonal matrix with distinct entries. This new Ψ =