SAPPORO: A way to turn your graphics cards into a GRAPE-6

We present Sapporo, a library for performing high-precision gravitational N-body simulations on NVIDIA Graphical Processing Units (GPUs). Our library mimics the GRAPE-6 library, and N-body codes currently running on GRAPE-6 can switch to Sapporo by a simple relinking of the library. The precision of our library is comparable to that of GRAPE-6, even though internally the GPU hardware is limited to single precision arithmetics. This limitation is effectively overcome by emulating double precision for calculating the distance between particles. The performance loss of this operation is small (< 20%) compared to the advantage of being able to run at high precision. We tested the library using several GRAPE-6-enabled N-body codes, in particular with Starlab and phiGRAPE. We measured peak performance of 800 Gflop/s for running with 10^6 particles on a PC with four commercial G92 architecture GPUs (two GeForce 9800GX2). As a production test, we simulated a 32k Plummer model with equal mass stars well beyond core collapse. The simulation took 41 days, during which the mean performance was 113 Gflop/s. The GPU did not show any problems from running in a production environment for such an extended period of time.

💡 Research Summary

The paper introduces SAPPORO, a software library that enables high‑precision gravitational N‑body simulations on NVIDIA graphics processing units (GPUs) while preserving full compatibility with the well‑established GRAPE‑6 hardware interface. By mimicking the GRAPE‑6 API, existing GRAPE‑6‑enabled codes such as Starlab and phiGRAPE can be switched to GPU execution simply by relinking against the SAPPORO library, without any source‑code modifications. This design choice lowers the barrier for researchers who have invested in GRAPE‑6 software ecosystems to adopt modern, commodity hardware.

A central technical challenge addressed by SAPPORO is the limitation of the G92‑class GPUs, which provide only single‑precision (24‑bit mantissa) arithmetic in hardware. Accurate N‑body calculations, however, require double‑precision (53‑bit mantissa) accuracy for the computation of inter‑particle distances, because round‑off errors in the distance directly affect the force evaluation and energy conservation. SAPPORO solves this by implementing a software double‑precision scheme: each particle coordinate is split into a high‑order and a low‑order single‑precision component. When the distance between two particles is needed, the library computes the difference of the high components, the difference of the low components, and then combines them using a Kahan‑style compensation algorithm. The result reproduces a full double‑precision value while the underlying arithmetic remains single‑precision. This emulation incurs roughly a 20 % increase in floating‑point operations for the distance calculation, a modest overhead compared with the O(N²) force loop that dominates runtime.

Performance engineering is another major focus. SAPPORO distributes the O(N²) pairwise interactions across multiple GPUs using a block‑wise decomposition. Each GPU processes a subset of particle pairs, exploiting shared memory and registers to keep data resident on the device and to minimize global memory traffic. Data transfers between host and device are performed asynchronously over PCI‑Express 2.0, and synchronization points are kept to a minimum, allowing the CPU to overlap communication with computation. In benchmark tests on a workstation equipped with four G92 GPUs (two GeForce 9800GX2 cards, each containing two GPUs), SAPPORO achieved a peak throughput of 800 Gflop s⁻¹ when simulating one million particles. This represents more than an order of magnitude speed‑up over comparable CPU‑only implementations.

To demonstrate scientific viability, the authors conducted a production‑scale simulation of a 32 k‑particle equal‑mass Plummer sphere, evolving the system well beyond core collapse. The run lasted 41 days of wall‑clock time, during which the average sustained performance was 113 Gflop s⁻¹. Throughout the extended run the GPUs exhibited stable thermal and power behavior, indicating that commodity graphics hardware can be trusted for long‑duration, high‑precision astrophysical calculations.

The paper also discusses limitations. The G92 architecture provides only 512 MB of device memory, which caps the maximum particle count to a few million before memory exhaustion occurs. Moreover, newer GPU generations now include native double‑precision units, making software emulation unnecessary and potentially more efficient. Nevertheless, the authors argue that for the price range of legacy GPUs, SAPPORO offers a cost‑effective alternative to dedicated GRAPE‑6 hardware, delivering comparable numerical accuracy with substantially higher raw performance.

In conclusion, SAPPORO bridges the gap between legacy GRAPE‑6 software and modern GPU hardware, delivering double‑precision‑level accuracy through clever software techniques while retaining a familiar programming interface. Its successful deployment in a long‑term, production‑grade astrophysical simulation validates both its performance and reliability. As GPU architectures continue to evolve, the concepts pioneered by SAPPORO—particularly the seamless API compatibility and the hybrid precision strategy—are likely to influence future high‑performance computational tools for stellar dynamics and other domains requiring precise, large‑scale particle interactions.