Hierarchical QR factorization algorithms for multi-core cluster systems

This paper describes a new QR factorization algorithm which is especially designed for massively parallel platforms combining parallel distributed multi-core nodes. These platforms make the present and the foreseeable future of high-performance computing. Our new QR factorization algorithm falls in the category of the tile algorithms which naturally enables good data locality for the sequential kernels executed by the cores (high sequential performance), low number of messages in a parallel distributed setting (small latency term), and fine granularity (high parallelism).

💡 Research Summary

The paper addresses the challenge of designing QR factorization algorithms for the next generation of massively parallel machines that combine distributed-memory clusters with multi‑core nodes. Traditional dense linear algebra libraries such as LAPACK and ScaLAPACK rely on large block‑column panels and are ill‑suited for the deep hierarchy of modern hardware. Tile‑based QR algorithms improve locality and expose fine‑grained parallelism, but they still suffer from a trade‑off between using a single “killer” tile (high sequential performance with TS kernels) and multiple killers (greater parallelism with TT kernels). The authors propose a novel hierarchical reduction scheme composed of four distinct levels. The lowest level (TS level) employs cache‑friendly TS kernels to convert square tiles into triangular form, achieving high per‑core efficiency. The next level (low level) builds independent intra‑node reduction trees that can proceed without inter‑node communication, thereby exploiting the many cores within a node. The third level (coupling level) sits between intra‑node and inter‑node reductions; it resolves dependencies between the low‑level trees and the global reduction, mixing TS and TT kernels to keep both locality and parallelism high. The topmost level (high level) implements an inter‑node reduction tree that is aware of the 2‑D block‑cyclic data distribution, minimizing the number of MPI messages and thus achieving a communication‑avoiding property. Because each level can use any tree shape, the algorithm can be tuned to the shape of the input matrix (square, tall‑skinny, wide, etc.). Implementation is carried out with the DAGUE runtime system, which automatically translates the task graph into a mix of shared‑memory operations and MPI communications, relieving programmers from low‑level synchronization concerns. Experimental results on a cluster of multi‑core nodes show that each of the four levels contributes measurable performance gains; the coupling level, in particular, eliminates bottlenecks between local and global reductions. Compared with state‑of‑the‑art QR solvers (ScaLAPACK, PLASMA, Elemental), the proposed method delivers speed‑ups ranging from 1.5× to 3× across a variety of matrix dimensions. The study also provides insight into how different reduction trees affect performance for different matrix shapes, and discusses scalability toward exascale platforms. In summary, the work expands the design space of tiled QR factorizations, introduces a flexible, modular, and highly scalable hierarchical algorithm, and demonstrates that a carefully crafted reduction hierarchy combined with a task‑based runtime can substantially outperform existing solutions on contemporary multi‑core cluster architectures.