3DSGrasp: 3D Shape-Completion for Robotic Grasp

Real-world robotic grasping can be done robustly if a complete 3D Point Cloud Data (PCD) of an object is available. However, in practice, PCDs are often incomplete when objects are viewed from few and sparse viewpoints before the grasping action, leading to the generation of wrong or inaccurate grasp poses. We propose a novel grasping strategy, named 3DSGrasp, that predicts the missing geometry from the partial PCD to produce reliable grasp poses. Our proposed PCD completion network is a Transformer-based encoder-decoder network with an Offset-Attention layer. Our network is inherently invariant to the object pose and point’s permutation, which generates PCDs that are geometrically consistent and completed properly. Experiments on a wide range of partial PCD show that 3DSGrasp outperforms the best state-of-the-art method on PCD completion tasks and largely improves the grasping success rate in real-world scenarios. The code and dataset will be made available upon acceptance.

💡 Research Summary

The paper “3DSGrasp: 3D Shape-Completion for Robotic Grasp” presents a novel and integrated framework designed to significantly improve robotic grasping success rates in real-world scenarios where only partial 3D visual data is available. The core problem addressed is that point cloud data (PCD) captured from a single or a few sparse viewpoints is inherently incomplete, missing occluded parts of an object. This incompatibility leads existing grasp pose detection methods (like GPD) to generate erroneous or unstable grasp candidates, resulting in frequent failures.

To overcome this, the authors propose a two-pronged solution centered on a 3D shape completion network. The primary contribution is a novel neural network architecture for completing partial point clouds. It builds upon a Transformer-based encoder-decoder structure but innovates by incorporating an Offset-Attention layer. This layer is key because it measures the difference between the input features and the self-attention output, inherently enforcing invariance to rigid transformations (rotation and translation) of the input point cloud. This property is crucial for robustness in real applications where the object’s pose relative to the camera is arbitrary. The network processes the input by first dividing the partial PCD into local regions using Farthest Point Sampling (FPS) and K-Nearest Neighbors (KNN). Features from these regions are extracted using a DGCNN backbone, combined with positional embeddings, and fed into the Offset-Attention Transformer. The decoder then generates a dense, completed point cloud representing the missing geometry, which is merged with the original observed partial scan.

A critical and often overlooked data alignment pre-processing step forms the second major contribution. Standard completion methods normalize training data using the centroid of the complete ground-truth shape. However, in a real test scenario, only the partial point cloud is available for calculating the centroid, leading to a misalignment between the predicted completion and the actual observed object part. 3DSGrasp solves this pragmatically: during training, it calculates normalization parameters (translation offset and scale) solely from the partial input PCD and applies the same parameters to the ground-truth PCD. This ensures the network learns to complete shapes relative to the coordinate frame defined by the visible partial scan, enabling seamless transfer from simulation to reality.

The proposed system is evaluated comprehensively. First, on a PCD completion benchmark derived from the YCB dataset, 3DSGrasp outperforms state-of-the-art methods, including PoinTr, in terms of reconstruction accuracy (e.g., Chamfer Distance). More importantly, the authors implement and test a full robotic grasping pipeline using a Kinova arm equipped with a RealSense camera. The pipeline involves capturing a depth image, segmenting the object PCD, completing it with their network, generating grasp candidates with GPD, and executing the best feasible grasp via MoveIt!. Real-world grasping experiments demonstrate a substantial improvement in the success rate compared to using only the partial point cloud, proving that shape completion directly translates to enhanced physical robot performance. The paper also includes extensive ablation studies validating architectural choices like the Offset-Attention layer and skip connections. The code and dataset are promised to be released upon acceptance.