Parallelization of Loops with Variable Distance Data Dependences

The extent of parallelization of a loop is largely determined by the dependences between its statements. While dependence free loops are fully parallelizable, those with loop carried dependences are not. Dependence distance is a measure of absolute difference between a pair of dependent iterations. Loops with constant distance data dependence, because of uniform distance between the dependent iterations, lead to easy partitioning of the iteration space and hence they have been successfully dealt with. Parallelization of loops with variable distance data dependences is a considerably difficult problem. It is our belief that partitioning the iteration space in such loops cannot be done without examining solutions of the corresponding Linear Diophantine Equations. Focus of this work is to study variable distance data dependences and examine the relation between dependent iterations. Our analysis based on parametric solution leads to a mathematical formulation capturing dependence between iterations. Our approach shows the existence of reasonable exploitable parallelism in variable distance data dependences loops with multiple LDEs.

💡 Research Summary

The paper addresses the challenging problem of parallelizing loops whose data‑dependence distances are not constant but vary across iterations (Variable Distance Data Dependences, VD). While loops with constant distance dependences (CD) can be easily partitioned because the dependence distance is uniform, VD loops exhibit irregular dependence patterns that resist traditional partitioning techniques. The authors propose a systematic approach based on solving two‑variable linear Diophantine equations (LDEs) of the form ax + by = c, assuming a normalized representation with a > 0 and a ≤ |b|.

Using the well‑known parametric solution {α − mb, α + ma} (m ∈ ℤ), every pair of dependent iterations is expressed precisely. Each solution pair is treated as a vertex in a dependence graph, with a directed edge from the source iteration to the sink iteration. The graph is then decomposed into undirected connected components (CCs). Within a CC the iterations must be executed sequentially, but distinct CCs are independent and can be run in parallel without synchronization.

To avoid enumerating all vertices explicitly, the authors introduce the concept of a “seed”: a representative iteration that can generate the entire CC through the parametric formula. For a single LDE every iteration is a seed of its own CC; for multiple LDEs seeds may overlap. Overlapping seeds are classified as “common seeds” (appear in solutions of more than one LDE) while “unique seeds” belong to a single LDE. By intersecting solution sets of different LDEs the algorithm extracts common seeds, then builds a global partition by merging the CCs associated with these seeds.

The paper derives analytical bounds on the number of CCs |P| and the maximum CC length Lmax for a given iteration range R =