TransBridge: Boost 3D Object Detection by Scene-Level Completion with Transformer Decoder

3D object detection is essential in autonomous driving, providing vital information about moving objects and obstacles. Detecting objects in distant regions with only a few LiDAR points is still a challenge, and numerous strategies have been developed to address point cloud sparsity through densification.This paper presents a joint completion and detection framework that improves the detection feature in sparse areas while maintaining costs unchanged. Specifically, we propose TransBridge, a novel transformer-based up-sampling block that fuses the features from the detection and completion networks.The detection network can benefit from acquiring implicit completion features derived from the completion network. Additionally, we design the Dynamic-Static Reconstruction (DSRecon) module to produce dense LiDAR data for the completion network, meeting the requirement for dense point cloud ground truth.Furthermore, we employ the transformer mechanism to establish connections between channels and spatial relations, resulting in a high-resolution feature map used for completion purposes.Extensive experiments on the nuScenes and Waymo datasets demonstrate the effectiveness of the proposed framework.The results show that our framework consistently improves end-to-end 3D object detection, with the mean average precision (mAP) ranging from 0.7 to 1.5 across multiple methods, indicating its generalization ability. For the two-stage detection framework, it also boosts the mAP up to 5.78 points.

💡 Research Summary

This paper introduces “TransBridge,” a novel joint learning framework designed to address the critical challenge of 3D object detection in sparse regions, particularly for distant objects in autonomous driving scenarios. The core problem stems from the inherent sparsity and non-uniformity of LiDAR point clouds, which leads to difficulty in distinguishing between “transparent voxels” (empty space) and “invisible voxels” (occluded or undersampled areas containing objects). Traditional densification methods improve detection but often at the cost of significantly increased computational overhead during inference.

The proposed TransBridge framework elegantly solves this by jointly training a 3D object detection network and a point cloud completion network within a single architecture, sharing a common feature encoder. This shared encoder, typically based on a state-of-the-art detector like CenterPoint, extracts multi-scale feature maps. The key innovation lies in the “Completion Decoder,” which is built upon novel “TransBridge blocks.” Each TransBridge block consists of two main components: an Up-Sampling Bridge (UB) that increases spatial resolution, and an Interpreting Bridge (IB). The IB acts as a mediator, leveraging a transformer mechanism to adapt and translate the semantically rich features from the detection task into geometry-aware features suitable for the completion task. This allows the completion decoder to guide the shared encoder to produce more discriminative features, especially for sparse or occluded areas, without altering the detection network’s inference pipeline. To maintain efficiency, a Sparsity Controlling Module (SCM) filters out features from transparent voxels during upsampling.

A significant contribution is the method for generating high-quality training data for the completion network, termed Dynamic-Static Reconstruction (DSRecon). Instead of simply merging consecutive LiDAR sweeps—which creates “trailing smear” artifacts from moving objects—DSRecon separates dynamic foreground objects and static background. It aligns the dynamic objects to their respective coordinates and then applies a surface reconstruction technique (NKSR) to both parts before merging them. This process yields clean, dense point clouds that serve as superior ground truth for scene-level completion learning.

Extensive experiments on the nuScenes and Waymo Open Dataset benchmarks validate the framework’s effectiveness and generalization capability. When integrated with various modern 3D object detectors (including single-stage and two-stage methods), TransBridge consistently boosts detection performance. Improvements in mean Average Precision (mAP) range from 0.7 to 1.5 points for end-to-end detectors, and up to 5.78 points for a two-stage framework, without introducing additional computational cost during inference. The results demonstrate that TransBridge acts as a versatile “performance booster,” enhancing the detector’s ability to perceive objects in sparse regions by learning a more robust feature representation through the auxiliary completion task. This work provides a practical and efficient solution to a fundamental perception problem in autonomous driving, aiming to improve safety and reliability.