Tight Bounds for Parallel Randomized Load Balancing

We explore the fundamental limits of distributed balls-into-bins algorithms. We present an adaptive symmetric algorithm that achieves a bin load of two in log* n+O(1) communication rounds using O(n) messages in total. Larger bin loads can be traded in for smaller time complexities. We prove a matching lower bound of (1-o(1))log* n on the time complexity of symmetric algorithms that guarantee small bin loads at an asymptotically optimal message complexity of O(n). For each assumption of the lower bound, we provide an algorithm violating it, in turn achieving a constant maximum bin load in constant time. As an application, we consider the following problem. Given a fully connected graph of n nodes, where each node needs to send and receive up to n messages, and in each round each node may send one message over each link, deliver all messages as quickly as possible to their destinations. We give a simple and robust algorithm of time complexity O(log* n) for this task and provide a generalization to the case where all nodes initially hold arbitrary sets of messages. A less practical algorithm terminates within asymptotically optimal O(1) rounds. All these bounds hold with high probability.

💡 Research Summary

The paper investigates the fundamental limits of distributed balls‑into‑bins processes, a canonical model for parallel load balancing, under the constraints of symmetry, limited communication, and high‑probability guarantees. The authors first revisit the seminal lower bound of Adler et al., which shows that non‑adaptive, symmetric algorithms cannot achieve a maximum bin load better than Θ(log log n / log log log n) within the same number of rounds. They argue that the two restrictive assumptions—non‑adaptivity (each ball fixes a constant number of candidate bins before any communication) and symmetry (candidate bins are chosen uniformly at random and ties are broken identically)—are the main obstacles to faster, more balanced solutions.

The core contribution is a simple adaptive symmetric algorithm that reaches a maximum load of two in log* n + O(1) synchronous communication rounds while using only O(1) messages per ball in expectation, i.e., O(n) messages overall. The algorithm proceeds in phases; in each phase every ball contacts a constant‑size set of bins, learns their current loads, and then selects the least‑loaded bin among them for placement. Because the set of contacted bins is chosen adaptively based on the information gathered in previous phases, the number of “unplaced” balls shrinks exponentially fast. After O(log* n) phases only a constant number of balls remain, which can be placed immediately, guaranteeing a final load of two. The authors also show how to trade a slightly larger constant load for fewer rounds, and how to generalize the method to the case where the number of balls differs from the number of bins.

To complement the upper bound, the paper proves a matching lower bound of (1 − o(1))·log* n rounds for any symmetric algorithm that (i) respects an overall O(n) message budget and (ii) operates with anonymous bins (i.e., balls have no globally unique bin identifiers). The proof departs from classic information‑theoretic arguments; instead it leverages a combinatorial “symmetry‑breaking” argument reminiscent of Linial’s lower bound for distributed coloring. The authors demonstrate that, under the two stated assumptions, any algorithm must spend at least log* n rounds to acquire enough asymmetry to keep the maximum load bounded by a constant. They further strengthen the result by constructing, for each violated assumption, an algorithm that achieves constant load in constant time, thereby showing the necessity of both constraints.

As an application, the authors consider a fully connected network of n processors where each processor must send and receive up to n messages, i.e., up to n² messages in total, with the restriction that each link can carry at most one message per round. By treating each processor as a “bin” and each message as a “ball”, they apply their symmetric adaptive technique to obtain a robust O(log* n)‑round algorithm that delivers all messages with high probability. A more aggressive variant, allowing a modest increase in total communication (e.g., O(n log n) messages), can finish in O(1) rounds, though it is less practical for realistic values of n.

The paper’s contributions can be summarized as follows: (1) an adaptive symmetric balls‑into‑bins algorithm achieving constant maximum load in log* n + O(1) rounds with optimal O(n) total messages; (2) a matching lower bound showing that any symmetric algorithm respecting the same message budget and bin anonymity must take at least (1 − o(1))·log* n rounds; (3) explicit constructions that violate each lower‑bound assumption and achieve constant‑time, constant‑load placement, establishing the tightness of the assumptions; (4) a concrete parallel load‑balancing protocol for fully connected networks that runs in O(log* n) rounds (or O(1) rounds with higher message overhead). All results hold with high probability. The work thus closes a long‑standing gap between upper and lower bounds for parallel randomized load balancing under realistic communication constraints, and provides practically implementable algorithms for large‑scale distributed systems.