Graphical X Splatting (GraphiXS): A Graphical Model for 4D Gaussian Splatting under Uncertainty

We propose a new framework to systematically incorporate data uncertainty in Gaussian Splatting. Being the new paradigm of neural rendering, Gaussian Splatting has been investigated in many applications, with the main effort in extending its representation, improving its optimization process, and accelerating its speed. However, one orthogonal, much needed, but under-explored area is data uncertainty. In standard 4D Gaussian Splatting, data uncertainty can manifest as view sparsity, missing frames, camera asynchronization, etc. So far, there has been little research to holistically incorporating various types of data uncertainty under a single framework. To this end, we propose Graphical X Splatting, or GraphiXS, a new probabilistic framework that considers multiple types of data uncertainty, aiming for a fundamental augmentation of the current 4D Gaussian Splatting paradigm into a probabilistic setting. GraphiXS is general and can be instantiated with a range of primitives, e.g. Gaussians, Student’s-t. Furthermore, GraphiXS can be used to `upgrade’ existing methods to accommodate data uncertainty. Through exhaustive evaluation and comparison, we demonstrate that GraphiXS can systematically model various uncertainties in data, outperform existing methods in many settings where data are missing or polluted in space and time, and therefore is a major generalization of the current 4D Gaussian Splatting research.

💡 Research Summary

**
The paper introduces Graphical X Splatting (GraphiXS), a probabilistic graphical‑model framework that augments the current 4‑dimensional Gaussian Splatting (4DGS) pipeline with explicit handling of various data uncertainties. Typical uncertainties in multi‑view video—missing camera viewpoints, sparse temporal sampling, asynchronous capture, and noisy pose estimates—are treated as random variables rather than deterministic inputs.

Core Idea
GraphiXS decomposes the rendering pipeline into four stochastic steps and represents them as nodes in a directed graphical model: (1) sampling a camera pose C and a frame time T, (2) generating a per‑pixel ray R from (C,T), (3) selecting a subset β of latent components (Gaussians or other primitives) that intersect the ray, and (4) rasterizing β to produce the pixel color X. The latent component parameters θ (means μ, covariances Σ, colors s, opacities o, etc.) are treated as hidden variables. The generative process can be written as a mixture model with potentially infinite components, and the learning objective is a Maximum‑a‑Posteriori (MAP) estimate:

 θ* = arg maxθ  P(X | R,β,α,C,T,δθ) P(δθ)

where α denotes the set of components projected onto a particular camera view at a given time. By factorizing the joint likelihood according to the graph, the authors obtain tractable terms for ray casting, per‑image component selection, per‑pixel component intersection, and rasterization.

Flexibility
GraphiXS is not limited to Gaussian primitives; the framework can instantiate Student‑t, Beta, or any distribution that can be parameterized by μ, Σ, s, o, etc. This allows the model to be heavy‑tailed, improving robustness to outliers and noisy measurements. The authors demonstrate two instantiations—Gaussian and Student‑t—showing that the latter yields higher PSNR under extreme sparsity.

Training Strategy
Full Bayesian inference is deemed computationally infeasible because the number of latent parameters can reach millions. Instead, the authors adopt MAP inference with simple, non‑informative priors on camera poses and timestamps, while allowing richer priors on component attributes (e.g., smoothness over time, spatial regularization). The optimization proceeds with standard gradient‑based methods, reusing the existing 4DGS rasterization pipeline and only adding stochastic sampling steps for C, T, and component subsets.

Experimental Evaluation
Three uncertainty scenarios are constructed: (i) 50 % of cameras randomly removed (spatial sparsity), (ii) low‑frame‑rate capture at 10 FPS (temporal sparsity), and (iii) 37 % random frame loss (spatio‑temporal sparsity). GraphiXS is compared against baseline 4DGS, Free‑TimeGS, and two recent 4DGS variants (4DGS‑1 and 4DGS‑2). Metrics include PSNR, SSIM, and LPIPS, as well as qualitative visual inspection. Across all scenarios GraphiXS outperforms baselines, with the Student‑t version achieving up to 1.2 dB higher PSNR in the most challenging setting.

Strengths

1. Unified Uncertainty Modeling – By treating camera poses, timestamps, and component selections as random variables, GraphiXS naturally handles multiple sources of uncertainty within a single probabilistic framework.
2. Modular Upgrade Path – Existing 4DGS implementations can be “upgraded” by inserting the graphical‑model layer, avoiding the need to redesign the entire pipeline.
3. Robustness via Heavy‑Tail Primitives – The ability to switch to Student‑t distributions provides resilience against outliers and missing data.

Weaknesses

1. MAP Limitation – MAP optimization ignores multimodal posterior structures; in highly ambiguous scenes it may converge to sub‑optimal local minima.
2. Computational Overhead – Introducing stochastic sampling for C, T, α, and β adds memory and runtime costs, which the paper does not fully address for real‑time applications.
3. Lack of Full Bayesian Exploration – The work stops at MAP; more expressive variational inference or MCMC could capture uncertainty in the learned parameters themselves, improving confidence estimates.

Future Directions

• Variational or MCMC‑based Bayesian Inference to capture multimodal posteriors and provide calibrated uncertainty estimates.
• Graph Simplification and Hardware Acceleration to reduce the overhead of sampling steps, enabling real‑time rendering on GPUs or specialized accelerators.
• Integration with Physically‑Based Lighting to extend the probabilistic treatment to illumination and material properties, further closing the gap between synthetic and real‑world video capture.

Conclusion
GraphiXS represents the first comprehensive probabilistic extension of 4D Gaussian Splatting that explicitly models data uncertainty across space and time. By embedding the rendering pipeline in a graphical model, it offers a principled way to incorporate missing views, sparse temporal sampling, and asynchronous cameras, while remaining compatible with existing 4DGS codebases. The experimental results demonstrate clear quantitative and qualitative gains, establishing GraphiXS as a promising foundation for robust neural rendering in real‑world, imperfect data collection scenarios.