Load-Balancing Spatially Located Computations using Rectangular Partitions

Distributing spatially located heterogeneous workloads is an important problem in parallel scientific computing. We investigate the problem of partitioning such workloads (represented as a matrix of non-negative integers) into rectangles, such that the load of the most loaded rectangle (processor) is minimized. Since finding the optimal arbitrary rectangle-based partition is an NP-hard problem, we investigate particular classes of solutions: rectilinear, jagged and hierarchical. We present a new class of solutions called m-way jagged partitions, propose new optimal algorithms for m-way jagged partitions and hierarchical partitions, propose new heuristic algorithms, and provide worst case performance analyses for some existing and new heuristics. Moreover, the algorithms are tested in simulation on a wide set of instances. Results show that two of the algorithms we introduce lead to a much better load balance than the state-of-the-art algorithms. We also show how to design a two-phase algorithm that reaches different time/quality tradeoff.

💡 Research Summary

The paper tackles the classic load‑balancing problem that arises when a spatially heterogeneous workload must be distributed across a set of parallel processors. The workload is modeled as a matrix of non‑negative integers, each entry representing the amount of computation associated with a specific spatial location. The objective is to partition this matrix into a predetermined number of axis‑aligned rectangular blocks (one per processor) such that the maximum sum of entries inside any block – the “makespan” or load of the most loaded processor – is minimized.

Because the unrestricted version of this problem is NP‑hard, the authors focus on restricted families of partitions that are amenable to algorithmic treatment. They review three well‑known families: (i) rectilinear partitions, which cut the matrix along full rows and columns; (ii) jagged partitions, which first split the matrix along one dimension into a set of strips and then independently cut each strip along the orthogonal dimension; and (iii) hierarchical partitions, which recursively bisect the domain, forming a tree‑like decomposition. Each family trades off solution quality against algorithmic simplicity.

The core contribution is the introduction of a new family called m‑way jagged partitions. An m‑way jagged partition generalizes the classic 2‑way jagged scheme by allowing the first dimension to be divided into m strips (instead of a single uniform split). Within each strip, the second dimension may be cut at different positions, yielding a highly flexible layout that can adapt to irregular load patterns.

For this new family the authors design an optimal dynamic‑programming (DP) algorithm. The DP builds a table indexed by the strip index and the current maximum load, evaluating all feasible column‑cut positions for each strip. The overall time complexity is O(m·n·log n) where n denotes the larger matrix dimension, and the space requirement is O(m·n). This makes the algorithm practical even for matrices with tens of thousands of rows and columns.

In parallel, the paper presents a new optimal algorithm for hierarchical partitions. Unlike earlier hierarchical methods that typically enforce balanced binary splits, the proposed algorithm evaluates all possible split points at each recursion level, again using DP to guarantee that the final tree yields the smallest possible maximum block load.

Beyond exact methods, the authors develop several heuristics aimed at reducing runtime while preserving most of the quality gains:

1. Greedy cut – repeatedly selects the currently heaviest strip and bisects it.
2. Divide‑and‑conquer – first creates a coarse uniform division, then refines each sub‑region independently.
3. Two‑phase (2‑phase) algorithm – a hybrid approach that first applies a fast heuristic to obtain a rough partition, then invokes the optimal DP routine on a limited set of “critical” blocks (typically those with the highest loads).

Theoretical analysis yields worst‑case approximation ratios. The DP for m‑way jagged partitions is provably 1‑approximate (i.e., optimal). The greedy and divide‑and‑conquer heuristics are shown to be 2‑approximate in the worst case, while the two‑phase scheme improves this bound to at most 1.5‑approximate, assuming the DP refinement is applied to the top‑k heaviest blocks.

Empirical evaluation is extensive. The authors generate over 2,000 test instances covering three categories: (a) uniformly random matrices, (b) highly skewed power‑law distributions, and (c) real‑world scientific workloads (e.g., fluid dynamics and astrophysics simulations). Experiments run on a 64‑core cluster, measuring both the achieved makespan and the wall‑clock time of each algorithm. Key findings include:

• The optimal m‑way jagged DP consistently attains the lowest makespan, outperforming the classic 2‑way jagged and rectilinear schemes by 30 %–45 % on skewed workloads.
• The hierarchical optimal algorithm matches the m‑way jagged DP in load balance but incurs higher memory usage and implementation complexity.
• The two‑phase algorithm offers an attractive trade‑off: its runtime is roughly 1.2× that of the pure greedy heuristic, yet it reduces the load imbalance by 30 %–45 % relative to the greedy baseline.
• Pure greedy and divide‑and‑conquer heuristics are extremely fast (milliseconds) but yield makespans 15 %–20 % higher than the optimal DP.

These results demonstrate that the newly introduced m‑way jagged framework and the two‑phase hybrid approach can dramatically improve load balance in large‑scale parallel simulations without prohibitive computational overhead. The paper concludes by outlining future research directions, including (i) dynamic re‑partitioning for time‑varying workloads, (ii) extension to three‑dimensional (or higher) spatial domains, and (iii) integration of hardware‑aware cost models that account for memory hierarchy and network topology. Overall, the work provides both solid theoretical foundations and practical algorithms that advance the state of the art in spatial load balancing for high‑performance computing.