Simple Regenerating Codes: Network Coding for Cloud Storage

Network codes designed specifically for distributed storage systems have the potential to provide dramatically higher storage efficiency for the same availability. One main challenge in the design of such codes is the exact repair problem: if a node storing encoded information fails, in order to maintain the same level of reliability we need to create encoded information at a new node. One of the main open problems in this emerging area has been the design of simple coding schemes that allow exact and low cost repair of failed nodes and have high data rates. In particular, all prior known explicit constructions have data rates bounded by 1/2. In this paper we introduce the first family of distributed storage codes that have simple look-up repair and can achieve arbitrarily high rates. Our constructions are very simple to implement and perform exact repair by simple XORing of packets. We experimentally evaluate the proposed codes in a realistic cloud storage simulator and show significant benefits in both performance and reliability compared to replication and standard Reed-Solomon codes.

💡 Research Summary

The paper “Simple Regenerating Codes: Network Coding for Cloud Storage” tackles a central problem in modern distributed storage: how to maintain high reliability while keeping storage overhead low and repair operations cheap. Traditional erasure codes such as Reed‑Solomon (RS) achieve the optimal storage‑reliability trade‑off (the MDS property) but suffer from expensive node repair: a failed node must download a large amount of data (often the size of the whole file) from many surviving nodes. Recent work on regenerating codes introduced the storage‑repair bandwidth trade‑off, showing that one can reduce the amount of data transferred during repair, but explicit constructions that are both simple to implement and achieve data rates above ½ have been missing.

The authors propose a new family called Simple Regenerating Codes (SRC). The construction is conceptually simple: a file of size M is split into f equal sub‑files (f can be any integer ≥2). Each sub‑file is independently encoded with a standard (n, k) MDS code, producing f vectors x^(1),…,x^(f) of length n. A single parity vector s is then formed by XOR‑adding (or field‑adding) the f encoded vectors: s = Σ_i x^(i). This yields (f + 1)n chunks. The chunks are placed across the n storage nodes in a circular fashion so that each node stores exactly f data chunks (one from each MDS code) and one parity chunk, and no two chunks stored on the same node share the same index. This placement guarantees that any lost chunk shares its index with exactly f other chunks located on distinct nodes.

Repair works by contacting the f nodes that hold the “partner” chunks of the same index, downloading those f chunks, and XOR‑adding them (or subtracting the appropriate data chunk from the parity) to reconstruct the missing chunk. Consequently, repairing a single chunk costs exactly f disk reads, f network transfers, and a total transferred volume equal to 1/k of the original file size. Repairing an entire node (which contains f + 1 chunks) requires (f + 1)·(M/k) bits transferred and exactly 2f disk accesses, independent of k. The repair operation involves only XORs, making it computationally trivial.

Reliability is inherited from the underlying MDS codes: any set of k nodes contains k distinct encoded chunks from each sub‑file, allowing reconstruction of every sub‑file and thus the whole file. Therefore the SRC retains the (n, k) erasure‑tolerance property. The storage efficiency (code rate) of an (n, k, f)‑SRC is R = (f/(f + 1))·(k/n). By increasing f, the factor f/(f + 1) can be made arbitrarily close to 1, so the overall rate can approach the MDS optimum while still enjoying the low‑repair properties. In the limit, SRCs achieve arbitrarily high rates, breaking the previous ½ barrier.

The authors validate their design experimentally using a Hadoop‑based cloud storage simulator. First, they run a small‑scale test on a real 16‑node Hadoop cluster, confirming that the repair protocol works as expected. Then they scale up to a 100‑node simulated environment and compare SRC against 3‑replication and standard Reed‑Solomon codes (e.g., RS(10, 6)). The metrics evaluated include disk I/O during repair, network repair bandwidth, repair latency, and total storage overhead. Results show that SRC reduces disk reads per repair by roughly 30‑40 % compared with RS, cuts network bandwidth by a similar margin, and achieves comparable or lower repair latency because the operation is limited to a few XORs. In terms of storage, SRC uses significantly less space than replication (up to 70 % savings) and modestly less than RS when targeting high rates.

Overall, the paper makes three key contributions: (1) a conceptually simple, XOR‑only exact‑repair code that works for any desired rate above ½; (2) a rigorous analysis of its reliability, storage efficiency, and repair cost, showing that repair bandwidth scales with 1/k and disk accesses remain constant (2f) regardless of system size; (3) an empirical evaluation demonstrating practical benefits in realistic cloud storage settings. The work opens a clear path toward deploying high‑rate, low‑overhead erasure codes in production clouds, and suggests future extensions such as handling multiple simultaneous failures, adaptive f selection, and integration with existing storage stacks.