Exploring the Limits of GPUs With Parallel Graph Algorithms

In this paper, we explore the limits of graphics processors (GPUs) for general purpose parallel computing by studying problems that require highly irregular data access patterns: parallel graph algorithms for list ranking and connected components. Such graph problems represent a worst case scenario for coalescing parallel memory accesses on GPUs which is critical for good GPU performance. Our experimental study indicates that PRAM algorithms are a good starting point for developing efficient parallel GPU methods but require non-trivial modifications to ensure good GPU performance. We present a set of guidelines that help algorithm designers adapt PRAM graph algorithms for parallel GPU computation. We point out that the study of parallel graph algorithms for GPUs is of wider interest for discrete and combinatorial problems in general because many of these problems require similar irregular data access patterns.

💡 Research Summary

The paper investigates how far modern graphics processors (GPUs) can be pushed for general‑purpose parallel computing when faced with algorithms that exhibit highly irregular memory access patterns. The authors focus on two classic graph problems—list ranking and connected components—because they are archetypal worst‑case scenarios for memory coalescing, a key factor in achieving high throughput on GPUs.

Starting from well‑known PRAM (Parallel Random‑Access Machine) algorithms, the authors first implement these algorithms directly in CUDA without any GPU‑specific modifications. Profiling reveals severe performance bottlenecks: global memory accesses are scattered and non‑coalesced, and divergent control flow within warps leads to low occupancy and wasted cycles. In other words, a naïve port of PRAM methods does not exploit the SIMD‑style execution model of GPUs.

To overcome these limitations, the paper proposes three systematic adaptation strategies.

1. Data layout transformation – Graphs are stored in CSR (Compressed Sparse Row) format and reordered so that threads within a warp read contiguous memory regions. This alignment enables coalesced memory transactions, dramatically increasing effective bandwidth.

2. Warp‑level work grouping – The core steps of list ranking (pointer jumping) and component merging are reorganized so that all threads in a warp perform the same logical operation on data that belongs to the same “phase”. By synchronizing work at the warp level, branch divergence is minimized, and the warp scheduler can keep all lanes busy.

3. Minimizing atomic operations and exploiting shared memory – In the connected‑components algorithm, label propagation and compression are performed primarily in shared memory and registers, with only the final label writes issued to global memory. This reduces contention on atomic primitives and cuts global memory traffic.

The authors also introduce a heuristic for tuning kernel launch parameters (block size, number of warps per block) based on the average degree of the input graph. This automatic tuning alleviates the need for labor‑intensive manual experimentation and helps keep the GPU’s SM resources well balanced between compute and memory.

Experimental evaluation uses both synthetic random graphs with uniform degree and real‑world web‑graph datasets ranging up to millions of vertices and tens of millions of edges. Results show that the GPU‑adapted versions achieve speed‑ups of up to 12× for list ranking and up to 9× for connected components compared with the unmodified PRAM‑derived kernels. Importantly, the performance gains scale with graph size, demonstrating that the optimizations preserve high memory bandwidth utilization even for very large inputs.

Beyond the two case studies, the paper argues that the presented guidelines are broadly applicable to any combinatorial or discrete‑optimization problem that suffers from irregular data accesses—such as matching, scheduling, or various graph‑based dynamic programming tasks. The central message is that PRAM algorithms provide a solid conceptual foundation, but successful GPU implementation requires careful redesign of data layout, work granularity, and synchronization to match the GPU’s architectural strengths.

In summary, the study shows that GPUs are not limited to regular, dense computations; with thoughtful algorithmic transformations they can efficiently handle worst‑case irregular workloads. This opens the door for future research on GPU‑accelerated discrete algorithms and provides a practical set of design principles for developers aiming to port PRAM‑style graph algorithms to modern many‑core accelerators.