Scalable Generation of Scale-free Graphs

We explain how massive instances of scale-free graphs following the Barabasi-Albert model can be generated very quickly in an embarrassingly parallel way. This makes this popular model available for studying big data graph problems. As a demonstration, we generated a Petaedge graph in less than an hour.

💡 Research Summary

The paper presents a highly scalable, embarrassingly parallel algorithm for generating massive scale‑free graphs that follow the Barabási‑Albert preferential‑attachment model. Traditional implementations of the model, such as the classic Batagelj‑Brandes algorithm, generate edges sequentially and rely on a growing edge array E to sample existing endpoints. This sequential dependence creates a severe bottleneck when the target graph contains billions of nodes and trillions of edges, because each new edge requires a memory access to the already‑generated part of E.

The authors observe that the preferential‑attachment probability can be reproduced simply by uniformly sampling an existing edge and using its endpoint as the target of the new edge. Building on this insight, they devise a method that completely eliminates the need to read from E during edge generation. For each edge i, the source endpoint is computed directly as ⌊i/d⌋, while the target endpoint is obtained by repeatedly applying a hash function h to a randomly chosen index r until r becomes even; the final target is then ⌊r/2d⌋. The hash function maps an integer to a pseudorandom number in the range