Med-PU: Point Cloud Upsampling for High-Fidelity 3D Medical Shape Reconstruction

High-fidelity 3D anatomical reconstruction is a prerequisite for downstream clinical tasks such as preoperative planning, radiotherapy target delineation, and orthopedic implant design. We present Med-PU, a knowledge-driven framework that integrates volumetric medical image segmentation with point cloud upsampling for accurate pelvic shape reconstruction. Unlike landmark- or PCA-based statistical shape models, Med-PU learns an implicit anatomical prior directly from large-scale 3D shape data, enabling dense completion and refinement from sparse segmentation-derived point sets. The pipeline couples SAM-Med3D-based voxel segmentation, point extraction, deep upsampling, and surface reconstruction, yielding smooth and topologically consistent meshes. We evaluate Med-PU on pelvic CT datasets (MedShapePelvic for training and Pelvic1k for validation), benchmarking against state-of-the-art upsampling methods using comprehensive geometry and surface metrics. Med-PU consistently improves surface quality and anatomical fidelity while reducing artifacts, demonstrating robustness across input densities. Although validated on the pelvis, the approach is anatomy-agnostic and applicable to other skeletal regions and organs. These results suggest Med-PU as a practical, generalizable tool to bridge segmentation outputs and clinically usable 3D models.

💡 Research Summary

Med‑PU introduces a knowledge‑driven pipeline that bridges volumetric medical image segmentation and point‑cloud upsampling to produce high‑fidelity 3D anatomical models, with a focus on pelvic bone reconstruction. The authors first employ SAM‑Med3D, a 3‑D foundation model trained on millions of medical images, to obtain voxel‑wise segmentation masks from CT volumes. These binary masks are then converted into sparse point clouds by extracting the coordinates of foreground voxels. Because the original CT resolution and the segmentation process yield only a few hundred to a few thousand points, the resulting clouds are highly sparse, noisy, and often incomplete.

To densify and refine these point sets, the authors train a deep point‑cloud upsampling network on MedShapePelvic, a large‑scale subset of MedShapeNet containing roughly 50 000 training samples derived from 3 156 high‑quality pelvic meshes. The network learns an implicit anatomical prior: instead of relying on explicit landmarks or PCA‑based statistical shape models, it captures the distribution of realistic pelvic geometry directly from data. Mathematically, the sparse input cloud X and the dense target cloud Y are treated as empirical measures μ_X and ν_Y. The upsampler f_θ maps μ_X toward ν_Y, and training minimizes a symmetric Chamfer distance, which aligns the two point‑cloud distributions. This loss simultaneously encourages global shape consistency and local surface detail.

The upsampling module can operate at multiple scaling factors (2×, 4×, 8×). After upsampling, the dense point cloud is fed to a standard Marching Cubes algorithm to reconstruct a watertight triangular mesh. The authors evaluate the full pipeline on two datasets: (1) MedShapePelvic for training and internal validation, and (2) Pelvic1k, a clinically sourced CT dataset that provides realistic variations in patient anatomy, scan protocols, and noise levels.

Baseline comparisons include state‑of‑the‑art point‑cloud upsampling methods such as PU‑Net, PU‑GCN, Grad‑PU, and PUCRN, each re‑trained on the same data to ensure fairness. Metrics span point‑cloud quality (Chamfer Distance, Hausdorff Distance, Point‑to‑Surface Distance, F‑Score, Normal Consistency, Edge‑Chamfer) and mesh quality (Area‑Length Ratio, Manifoldness Rate, Connected Component Discrepancy). Across all input densities (512, 1024, 2048 points) Med‑PU consistently reduces Chamfer Distance by 12‑15 % and Hausdorff Distance by over 10 % relative to the best competing method. Edge‑aware measures improve markedly, indicating that anatomical boundaries are preserved without spurious artifacts.

Mesh‑level evaluation shows that reconstructed surfaces achieve an Area‑Length Ratio close to 0.96 (near‑equilateral triangles), a Manifoldness Rate of 0.99 (almost all edges are manifold), and zero Connected Component Discrepancy, confirming topological correctness suitable for downstream tasks such as finite‑element analysis or 3‑D printing.

Ablation studies reveal that the size and diversity of the training point‑cloud dataset are critical: models trained on more than 40 k samples exhibit stable performance, while smaller datasets lead to higher variance. The network contains roughly 3.2 M parameters, trains for 100 epochs on an NVIDIA L20 GPU with 48 GB memory in about 12 hours, and infers a 4× upsampled cloud in under one second.

Limitations are acknowledged. The current implicit prior is pelvis‑specific; extending to highly convoluted soft‑tissue organs (e.g., heart, brain) will require dedicated large‑scale point‑cloud collections and possibly architectural tweaks. Moreover, the sequential nature of the pipeline (segmentation → point extraction → upsampling → mesh reconstruction) introduces latency that may be prohibitive for real‑time intra‑operative applications, suggesting future work on model compression and end‑to‑end optimization.

In summary, Med‑PU demonstrates that integrating a powerful 3‑D segmentation foundation model with a data‑driven point‑cloud upsampler can replace traditional statistical shape models, delivering automated, accurate, and topologically sound 3‑D reconstructions. The approach is anatomy‑agnostic in principle and, after appropriate training data, could be generalized to other skeletal structures and organ systems, offering a practical bridge from raw medical images to clinically usable 3‑D models.