ERGO: Excess-Risk-Guided Optimization for High-Fidelity Monocular 3D Gaussian Splatting

Generating 3D content from a single image remains a fundamentally challenging and ill-posed problem due to the inherent absence of geometric and textural information in occluded regions. While state-of-the-art generative models can synthesize auxiliary views to provide additional supervision, these views inevitably contain geometric inconsistencies and textural misalignments that propagate and amplify artifacts during 3D reconstruction. To effectively harness these imperfect supervisory signals, we propose an adaptive optimization framework guided by excess risk decomposition, termed ERGO. Specifically, ERGO decomposes the optimization losses in 3D Gaussian splatting into two components, i.e., excess risk that quantifies the suboptimality gap between current and optimal parameters, and Bayes error that models the irreducible noise inherent in synthesized views. This decomposition enables ERGO to dynamically estimate the view-specific excess risk and adaptively adjust loss weights during optimization. Furthermore, we introduce geometry-aware and texture-aware objectives that complement the excess-risk-derived weighting mechanism, establishing a synergistic global-local optimization paradigm. Consequently, ERGO demonstrates robustness against supervision noise while consistently enhancing both geometric fidelity and textural quality of the reconstructed 3D content. Extensive experiments on the Google Scanned Objects dataset and the OmniObject3D dataset demonstrate the superiority of ERGO over existing state-of-the-art methods.

💡 Research Summary

The paper introduces ERGO, an Excess‑Risk‑Guided Optimization framework designed to improve monocular 3D reconstruction using 3D Gaussian Splatting (3DGS). The authors observe that while multi‑view diffusion (MVD) models can generate auxiliary views from a single image, these synthesized images often contain geometric inconsistencies and texture misalignments that degrade downstream 3D reconstruction. To address this, ERGO decomposes the empirical loss into two statistically grounded components: excess risk, which measures the gap between the current model parameters and the (unknown) optimal parameters, and Bayes error, which captures irreducible noise inherent in the synthesized views. By estimating view‑specific excess risk during optimization, ERGO dynamically adjusts the weighting of each loss term.

The overall loss is a weighted sum of three objectives: (1) a standard Score Distillation Sampling (SDS) loss that brings in texture priors from pretrained diffusion models, (2) a geometry‑aware loss that leverages visibility maps and depth information from 3DGS to give higher weight to geometrically reliable regions, and (3) a texture‑aware loss that measures local texture complexity (e.g., edge strength, high‑frequency content) to preserve fine details. The weights (w_g, w_t, w_s) are updated each iteration according to the estimated excess risk R_i and Bayes error B_i for each view i, typically using a formula such as w_i = α·R_i/(R_i + B_i) + β, where α and β are hyper‑parameters. This adaptive scheme yields a global‑local optimization: globally, views with higher excess risk receive stronger guidance, while locally, reliable geometry and rich textures are emphasized.

Experiments on the Google Scanned Objects and OmniObject3D datasets demonstrate that ERGO consistently outperforms state‑of‑the‑art methods, including DreamFusion, Magic3D, and LGM. Quantitatively, ERGO improves PSNR by 0.8–1.2 dB, SSIM by 0.02–0.04, and reduces LPIPS by 0.03–0.05. Qualitatively, reconstructed objects exhibit sharper textures, better cross‑view consistency, and fewer “Janus” artifacts, especially on objects with complex shapes and reflective surfaces.

The authors acknowledge limitations: the current Bayes error model assumes simple Gaussian noise and may not fully capture the non‑linear distortions produced by MVD models, potentially leading to suboptimal weight allocation in extreme cases. Future work is suggested to develop more expressive noise models, explore real‑time weight updates, and extend the framework to other 3D representations such as Neural Radiance Fields or triplane‑based encodings.

In summary, ERGO provides a principled, statistically motivated approach to harness imperfect multi‑view supervision, achieving higher geometric fidelity and texture quality in single‑image‑to‑3D generation.