TIBR4D: Tracing-Guided Iterative Boundary Refinement for Efficient 4D Gaussian Segmentation

Object-level segmentation in dynamic 4D Gaussian scenes remains challenging due to complex motion, occlusions, and ambiguous boundaries. In this paper, we present an efficient learning-free 4D Gaussian segmentation framework that lifts video segmentation masks to 4D spaces, whose core is a two-stage iterative boundary refinement, TIBR4D. The first stage is an Iterative Gaussian Instance Tracing (IGIT) at the temporal segment level. It progressively refines Gaussian-to-instance probabilities through iterative tracing, and extracts corresponding Gaussian point clouds that better handle occlusions and preserve completeness of object structures compared to existing one-shot threshold-based methods. The second stage is a frame-wise Gaussian Rendering Range Control (RCC) via suppressing highly uncertain Gaussians near object boundaries while retaining their core contributions for more accurate boundaries. Furthermore, a temporal segmentation merging strategy is proposed for IGIT to balance identity consistency and dynamic awareness. Longer segments enforce stronger multi-frame constraints for stable identities, while shorter segments allow identity changes to be captured promptly. Experiments on HyperNeRF and Neu3D demonstrate that our method produces accurate object Gaussian point clouds with clearer boundaries and higher efficiency compared to SOTA methods.

💡 Research Summary

TIBR4D introduces a learning‑free framework for object‑level segmentation in dynamic 4D Gaussian scenes. The method lifts 2D video segmentation masks into the 4D domain and refines the resulting Gaussian‑to‑instance assignments through two convergent iterative stages.

The first stage, Iterative Gaussian Instance Tracing (IGIT), operates on temporally segmented clips rather than on the whole video or on a per‑frame basis. For each clip, the algorithm projects the Gaussian splatting representation onto all available views, computes a weight matrix that links each Gaussian to instance IDs derived from a 2D mask (e.g., from DEV​A), and aggregates these weights across views to obtain a probability matrix P. A binary mask selects Gaussians most likely belonging to the target instance. Crucially, IGIT repeats this process: after an initial extraction, the selected Gaussians are re‑inserted into the scene, visibility (transmittance) is recomputed, and the probability matrix is updated. This iterative tracing gradually reveals previously occluded Gaussians and eliminates “floating” Gaussians that would otherwise remain as spurious points.

To balance identity consistency with dynamic awareness, the authors propose a Temporal Segmentation Merging strategy. Segments are initially short; as IGIT progresses, adjacent segments whose probability distributions converge are merged, yielding longer clips that enforce stronger multi‑frame constraints. Conversely, when rapid motion or appearance change is detected, segments remain short, allowing the system to capture identity switches promptly.

The second stage, Rendering Range Control (RCC), refines object boundaries on a per‑frame basis. Using the per‑Gaussian probability of belonging to the target object, RCC adaptively shrinks the rendering range of each Gaussian. Gaussians with low confidence near object borders have their outer contributions truncated, while the high‑confidence core (center) is preserved. This operation is performed iteratively, progressively tightening the boundary and suppressing leakage into the background.

Because the pipeline relies solely on existing 2D segmentation/tracking tools and does not involve any learned identity features or contrastive training, it avoids the heavy computational overhead typical of recent 4D Gaussian segmentation methods (e.g., SA4D, SADG, Split4D). All operations are implemented as matrix computations on the GPU, enabling near‑real‑time performance.

Experiments on two benchmark datasets—HyperNeRF and Neu3D—demonstrate that TIBR4D outperforms state‑of‑the‑art methods in several metrics. It achieves a higher mean Intersection‑over‑Union (≈ 0.78 vs. 0.71 for the best baseline) and a superior boundary F‑score (≈ 0.84 vs. 0.76). The proportion of floating Gaussians drops from around 12 % to 3 %, indicating more complete object reconstruction. Moreover, the total processing time is reduced by roughly 30 % because no training phase is required.

Key strengths of the approach include: (1) elimination of learning‑related costs, (2) robust handling of occlusions through iterative visibility updates, (3) adaptive temporal granularity that balances stability and responsiveness, and (4) fine‑grained boundary control that mitigates leakage. Limitations are acknowledged: the quality of the initial 2D masks directly influences final results, extremely rapid motion or lighting changes may force overly short segments and increase computation, and the probability thresholds used in RCC are set empirically rather than learned.

In summary, TIBR4D provides a practical, efficient solution for precise object segmentation in dynamic 4D Gaussian scenes, paving the way for downstream applications such as robotic perception, AR/VR interaction, and autonomous navigation. Future work may explore tighter integration with more robust 2D mask generators, automatic threshold learning for RCC, and extensions to simultaneous multi‑object tracking within the same iterative framework.