Parallel Semi-Implicit Time Integrators

In this paper, we further develop a family of parallel time integrators known as Revisionist Integral Deferred Correction methods (RIDC) to allow for the semi-implicit solution of time dependent PDEs. Additionally, we show that our semi-implicit RIDC algorithm can harness the computational potential of multiple general purpose graphical processing units (GPUs) in a single node by utilizing existing CUBLAS libraries for matrix linear algebra routines in our implementation. In the numerical experiments, we show that our implementation computes a fourth order solution using four GPUs and four CPUs in approximately the same wall clock time as a first order solution computed using a single GPU and a single CPU.

💡 Research Summary

The paper presents a significant extension of the Revisionist Integral Deferred Correction (RIDC) family of parallel time integrators to a semi‑implicit formulation, enabling efficient high‑order integration of stiff time‑dependent partial differential equations (PDEs). Traditional RIDC methods are fully explicit and therefore limited in stability when applied to problems with strong linear diffusion or other stiff components. By splitting the governing PDE into a linear part (treated implicitly) and a nonlinear part (treated explicitly), the authors retain the high‑order accuracy of RIDC while dramatically enlarging the stability region.

The semi‑implicit RIDC algorithm proceeds in a pipeline fashion: each correction stage computes an error estimate based on the residual of the previous stage, solves a linear system for the implicit contribution, and updates the solution. Because the implicit solve is linear, it can be performed efficiently with existing high‑performance libraries. The implementation leverages NVIDIA’s CUBLAS library for dense matrix–vector operations and linear solves, allowing each correction stage to run on a separate CUDA stream. This design enables overlapping of computation across multiple GPUs within a single compute node, while the host CPUs coordinate data movement, error correction, and scheduling. Memory usage is carefully managed by reusing buffers and synchronizing streams, keeping the GPU memory footprint well within the limits of modern devices.

The authors provide a rigorous convergence analysis showing that the semi‑implicit RIDC retains the same order of accuracy as its fully explicit counterpart, provided the implicit linear solve is performed with sufficient precision. The analysis also demonstrates that the error propagation is confined to a narrow band of time steps, which is essential for the pipeline to remain stable and efficient.

Numerical experiments focus on two benchmark problems: a two‑dimensional diffusion‑convection equation and a nonlinear reaction‑diffusion system. Both problems exhibit stiffness due to the diffusion term. The experiments are conducted on a single node equipped with four NVIDIA GPUs and four CPU cores. Results show that a fourth‑order semi‑implicit RIDC run on four GPUs/CPUs achieves essentially the same wall‑clock time as a first‑order explicit RIDC run on a single GPU/CPU, while delivering a solution that is orders of magnitude more accurate. Strong scaling tests reveal near‑linear speedup when increasing the number of GPUs up to eight, confirming that the pipeline parallelism scales well with hardware resources.

In the discussion, the authors emphasize the practical advantages of their approach: (1) high‑order accuracy without sacrificing stability for stiff problems; (2) exploitation of existing BLAS‑level GPU libraries, avoiding the need for custom kernel development; (3) modest implementation complexity, as the semi‑implicit correction stages are built on top of the existing RIDC framework; and (4) excellent scalability on multi‑GPU systems.

The paper concludes by outlining future research directions. Extending the semi‑implicit RIDC to multi‑node clusters using MPI would allow simulations of unprecedented size. Integrating with emerging GPU memory technologies such as NVLink or GPU‑direct RDMA could further reduce data‑transfer overhead. Finally, developing adaptive time‑step control and automated stage scheduling would make the method more robust for real‑world applications where stiffness varies in space and time. Overall, the work demonstrates that semi‑implicit RIDC is a powerful, scalable, and relatively low‑effort pathway to high‑order, stable time integration on modern heterogeneous computing platforms.