Rebenchmarking Unsupervised Monocular 3D Occupancy Prediction

Inferring the 3D structure from a single image, particularly in occluded regions, remains a fundamental yet unsolved challenge in vision-centric autonomous driving. Existing unsupervised approaches typically train a neural radiance field and treat the network outputs as occupancy probabilities during evaluation, overlooking the inconsistency between training and evaluation protocols. Moreover, the prevalent use of 2D ground truth fails to reveal the inherent ambiguity in occluded regions caused by insufficient geometric constraints. To address these issues, this paper presents a reformulated benchmark for unsupervised monocular 3D occupancy prediction. We first interpret the variables involved in the volume rendering process and identify the most physically consistent representation of the occupancy probability. Building on these analyses, we improve existing evaluation protocols by aligning the newly identified representation with voxel-wise 3D occupancy ground truth, thereby enabling unsupervised methods to be evaluated in a manner consistent with that of supervised approaches. Additionally, to impose explicit constraints in occluded regions, we introduce an occlusion-aware polarization mechanism that incorporates multi-view visual cues to enhance discrimination between occupied and free spaces in these regions. Extensive experiments demonstrate that our approach not only significantly outperforms existing unsupervised approaches but also matches the performance of supervised ones. Our source code and evaluation protocol will be made available upon publication.

💡 Research Summary

This paper tackles two fundamental shortcomings of current unsupervised monocular 3D occupancy prediction methods that are built on neural radiance fields (NeRF). First, existing works treat the network’s raw density output σ as a direct occupancy probability and binarize it with a fixed 0.5 threshold, even though σ is a scale‑dependent pointwise quantity whose magnitude varies with the sampling interval δ. Consequently, σ does not correspond to the voxel‑wise occupancy ground truth used for evaluation, creating a mismatch between training and testing protocols. Second, the prevalent use of 2‑D supervision hides the intrinsic ambiguity of occluded regions, because photometric reconstruction loss provides little signal for voxels that are never visible in the target view.

The authors propose a comprehensive re‑benchmarking framework that resolves both issues. They reinterpret the physically meaningful occupancy probability as the opacity α = 1 − exp(−σ·δ), which inherently integrates the sampling interval and represents the occupancy of a finite volume rather than an infinitesimal point. α is naturally bounded in (0, 1) and aligns with voxel‑wise ground‑truth probabilities, allowing a consistent binarization o(x) =