GRTX: Efficient Ray Tracing for 3D Gaussian-Based Rendering

3D Gaussian Splatting has gained widespread adoption across diverse applications due to its exceptional rendering performance and visual quality. While most existing methods rely on rasterization to render Gaussians, recent research has started investigating ray tracing approaches to overcome the fundamental limitations inherent in rasterization. However, current Gaussian ray tracing methods suffer from inefficiencies such as bloated acceleration structures and redundant node traversals, which greatly degrade ray tracing performance. In this work, we present GRTX, a set of software and hardware optimizations that enable efficient ray tracing for 3D Gaussian-based rendering. First, we introduce a novel approach for constructing streamlined acceleration structures for Gaussian primitives. Our key insight is that anisotropic Gaussians can be treated as unit spheres through ray space transformations, which substantially reduces BVH size and traversal overhead. Second, we propose dedicated hardware support for traversal checkpointing within ray tracing units. This eliminates redundant node visits during multi-round tracing by resuming traversal from checkpointed nodes rather than restarting from the root node in each subsequent round. Our evaluation shows that GRTX significantly improves ray tracing performance compared to the baseline ray tracing method with a negligible hardware cost.

💡 Research Summary

The paper addresses the inefficiencies that arise when trying to render 3D Gaussian splatting (3DGS) scenes with ray tracing. While 3DGS achieves impressive visual quality and real‑time performance using rasterization, rasterization cannot handle highly distorted camera models, complex lighting effects, or per‑ray depth sorting without artifacts. Recent attempts to use ray tracing for Gaussian primitives have relied on converting each anisotropic Gaussian into a bounding mesh proxy (typically a set of triangles) and building a single monolithic BVH over all proxies. This approach inflates the BVH size, increases memory consumption, and leads to redundant traversal work, especially in multi‑round tracing schemes where the traversal restarts from the root node for each round.

GRTX proposes a two‑pronged solution that combines software‑level data‑structure redesign with modest hardware extensions. The first insight is that an anisotropic Gaussian can be treated as a unit sphere after applying a ray‑space transformation that incorporates the Gaussian’s covariance matrix and orientation. Modern ray‑tracing hardware already performs such transformations at leaf (instance) nodes, so the authors embed the transformation matrix in TLAS instances. Consequently, all Gaussians share a single bottom‑level acceleration structure (BLAS) that contains only a unit‑sphere mesh. The top‑level acceleration structure (TLAS) merely references this shared BLAS for each Gaussian and stores per‑instance scale, rotation, and translation. This dramatically reduces the number of BVH nodes, cuts the memory footprint by roughly 70 %, and improves cache locality during traversal.

The second contribution is “traversal checkpointing.” In multi‑round Gaussian ray tracing, a ray’s traversal interval (t_min, t_max) is progressively narrowed as the k‑closest Gaussians are collected. When a node is found to lie outside the current interval, the traversal of its subtree is deferred to a later round, but the path from the root to that node is re‑computed each time, causing duplicated work. GRTX adds a lightweight hardware mechanism that records the node and its parent stack entries as a checkpoint when it is first encountered. In subsequent rounds the ray can resume directly from the checkpoint, bypassing the already‑visited upper levels. The checkpoint data can be stored in a small per‑ray register file or a dedicated checkpoint buffer, and a new traceRay flag enables this mode. Experiments show that checkpointing reduces total node visits by about 30 % and saves roughly 15 % of dynamic power on RTX‑style hardware.

The authors evaluate GRTX using a cycle‑accurate Vulkan‑Sim simulator and an NVIDIA RTX 5090 GPU. The combined software‑hardware version (GRTX) achieves an average speed‑up of 4.36× over a baseline that uses an icosahedron bounding mesh for each Gaussian. A software‑only variant (GRTX‑SW) runs on commodity GPUs and still delivers 1.44–2.15× speed‑ups across a suite of real‑world scenes (indoor, outdoor, vehicle, etc.). The paper also presents detailed breakdowns of traversal time, sorting overhead, and blending cost, confirming that the majority of the gain comes from the reduced BVH size and the elimination of redundant traversal passes.

In summary, the paper makes four key contributions: (1) a systematic analysis of why existing Gaussian ray‑tracing pipelines are inefficient; (2) a novel acceleration‑structure design that maps anisotropic Gaussians to a shared unit‑sphere BLAS via per‑instance ray‑space transforms; (3) a hardware‑supported checkpoint‑and‑replay mechanism that avoids re‑traversing already‑visited BVH branches in multi‑round tracing; and (4) a thorough experimental validation showing substantial performance and power benefits with minimal hardware cost. By bridging the gap between the high‑quality, flexible lighting capabilities of ray tracing and the efficiency of Gaussian‑based scene representations, GRTX opens the door to real‑time, physically‑accurate rendering for robotics, AR/VR, gaming, and other interactive media applications.