Automated Synthesis of Divide and Conquer Parallelism

This paper focuses on automated synthesis of divide-and-conquer parallelism, which is a common parallel programming skeleton supported by many cross-platform multithreaded libraries. The challenges of producing (manually or automatically) a correct divide-and-conquer parallel program from a given sequential code are two-fold: (1) assuming that individual worker threads execute a code identical to the sequential code, the programmer has to provide the extra code for dividing the tasks and combining the computation results, and (2) sometimes, the sequential code may not be usable as is, and may need to be modified by the programmer. We address both challenges in this paper. We present an automated synthesis technique for the case where no modifications to the sequential code are required, and we propose an algorithm for modifying the sequential code to make it suitable for parallelization when some modification is necessary. The paper presents theoretical results for when this {\em modification} is efficiently possible, and experimental evaluation of the technique and the quality of the produced parallel programs.

💡 Research Summary

The paper tackles the problem of automatically converting a sequential loop into a divide‑and‑conquer parallel program. Two challenges are addressed: (1) synthesizing the join operator that combines partial results when the original loop can be used unchanged, and (2) extending the loop when a join does not exist, by automatically adding auxiliary computations.

For the first challenge, the authors formalize the join as a ⊙‑homomorphism: a function h over sequences such that h(x·y) = h(x) ⊙ h(y). They employ syntax‑guided synthesis (SyGuS) to generate the join. The loop body is turned into a template where unknown operators and variables replace concrete parts; this template defines a search space that is expressive enough to contain the correct join yet small enough for modern SMT solvers. The synthesis engine finds a concrete join, and a separate proof‑generation phase automatically verifies that the join holds for all inputs, not just the examples used during synthesis.

When a join cannot be expressed using only the original loop’s state, the paper introduces the notion of extension. An extension adds new state variables (auxiliary accumulators) to the loop so that enough information is produced for a join to exist. The authors present an algorithm that analyses the unfolded recurrence of the loop, searches for “accumulators” that can be propagated across sub‑problems, and inserts them into the loop body. The algorithm is backed by theoretical results that characterize when such extensions are possible and when they can be discovered in polynomial time.

The contributions are: (i) an algorithm for automatic join synthesis with machine‑checked correctness proofs; (ii) an algorithm for automatic loop extension when a join is absent; (iii) a theoretical framework that delineates the class of single‑pass computable functions that are efficiently parallelizable via divide‑and‑conquer; (iv) an implementation called PARSYNTH and an experimental evaluation on about 30 benchmark loops (sum, second‑smallest, sorted‑check, maximum‑tail‑sum, etc.). In most cases the tool finds a join or an extension within seconds, and the generated parallel code (targeting libraries such as Intel TBB) achieves speed‑ups ranging from 2× to 8× over the sequential version, with modest overhead from the added auxiliary computations.

Overall, the work demonstrates that the combination of SyGuS‑based join synthesis and automatic loop extension can automate a large portion of the manual effort traditionally required to write correct and efficient divide‑and‑conquer parallel programs. The approach opens avenues for extending the technique to other parallel skeletons (map‑reduce, scan) and to more complex loop nests and data structures.