Through the Perspective of LiDAR: A Feature-Enriched and Uncertainty-Aware Annotation Pipeline for Terrestrial Point Cloud Segmentation

Accurate semantic segmentation of terrestrial laser scanning (TLS) point clouds is limited by costly manual annotation. We propose a semi-automated, uncertainty-aware pipeline that integrates spherical projection, feature enrichment, ensemble learning, and targeted annotation to reduce labeling effort, while sustaining high accuracy. Our approach projects 3D points to a 2D spherical grid, enriches pixels with multi-source features, and trains an ensemble of segmentation networks to produce pseudo-labels and uncertainty maps, the latter guiding annotation of ambiguous regions. The 2D outputs are back-projected to 3D, yielding densely annotated point clouds supported by a three-tier visualization suite (2D feature maps, 3D colorized point clouds, and compact virtual spheres) for rapid triage and reviewer guidance. Using this pipeline, we build Mangrove3D, a semantic segmentation TLS dataset for mangrove forests. We further evaluate data efficiency and feature importance to address two key questions: (1) how much annotated data are needed and (2) which features matter most. Results show that performance saturates after ~12 annotated scans, geometric features contribute the most, and compact nine-channel stacks capture nearly all discriminative power, with the mean Intersection over Union (mIoU) plateauing at around 0.76. Finally, we confirm the generalization of our feature-enrichment strategy through cross-dataset tests on ForestSemantic and Semantic3D. Our contributions include: (i) a robust, uncertainty-aware TLS annotation pipeline with visualization tools; (ii) the Mangrove3D dataset; and (iii) empirical guidance on data efficiency and feature importance, thus enabling scalable, high-quality segmentation of TLS point clouds for ecological monitoring and beyond. The dataset and processing scripts are publicly available at https://fz-rit.github.io/through-the-lidars-eye/.

💡 Research Summary

The paper introduces a semi‑automated, uncertainty‑aware annotation pipeline tailored for terrestrial laser scanning (TLS) point clouds, addressing the two major bottlenecks that have limited the adoption of deep‑learning segmentation in ecological contexts: (1) the scarcity of high‑quality labeled data due to the labor‑intensive nature of full‑resolution TLS annotation, and (2) the lack of robust feature sets that capture the rich geometric and radiometric information inherent to LiDAR returns.

The core of the method is a conversion of raw 3D points into a 2D equirectangular (spherical) image. Each point’s azimuth and elevation are mapped to pixel coordinates, while a stack of nine channels is assembled per pixel: raw range, intensity, return count, plus eight derived geometric descriptors (surface normal components, curvature, local PCA‑based roughness, height‑relative differences, point‑density metrics, etc.). This multi‑channel “range image” preserves the spatial relationships of the original cloud while providing a dense, structured representation that can be processed efficiently by conventional 2D convolutional networks.

Three heterogeneous segmentation backbones—UNet++, DeepLabV3+, and SegFormer—are trained on the multi‑channel images and combined in an ensemble. The pixel‑wise variance among the three predictions serves as an epistemic uncertainty estimate, eliminating the need for Monte‑Carlo dropout while still highlighting regions where the model lacks confidence.

Uncertainty maps drive an active‑learning loop: high‑uncertainty pixels are presented to a human annotator for correction, whereas low‑uncertainty predictions are automatically promoted to pseudo‑labels (self‑training). This hybrid strategy dramatically reduces the number of manual edits required while preventing the error propagation typical of fully automated self‑training pipelines. After each iteration, pseudo‑labels are merged with the human‑corrected set, the model is retrained, and the process repeats until the uncertainty map stabilizes.

The annotated 2D masks are back‑projected onto the original 3D points, yielding a densely labeled point cloud. To facilitate rapid verification, the authors provide a three‑tier visualization suite: (i) the multi‑channel spherical images, (ii) color‑coded 3D point clouds, and (iii) compact virtual spheres that encapsulate local neighborhoods for quick triage.

Using this pipeline, the authors built Mangrove3D, the first TLS semantic‑segmentation benchmark focused on structurally complex mangrove forests. The dataset comprises 39 scans (31.3 M points) collected in Palau, annotated into five ecological classes (ground & water, stem, canopy, root, object) plus a void label. Experiments on a 27/3/9 train/validation/test split reveal that performance (mean Intersection‑over‑Union, mIoU) saturates after roughly 12 annotated scans, reaching a plateau around 0.76. Adding more scans yields diminishing returns, confirming strong data efficiency.

A systematic ablation study evaluates the contribution of each feature channel. Geometric descriptors (normals, curvature, roughness) provide the largest boost, while intensity and range remain useful but less decisive. The nine‑channel stack captures nearly all discriminative power, and reducing the stack further leads to measurable drops in mIoU, establishing a practical guideline for feature selection in TLS‑only pipelines.

Cross‑dataset validation on ForestSemantic and Semantic3D demonstrates that the feature‑enrichment and uncertainty‑driven active learning approach generalizes beyond mangrove scenes, improving segmentation quality on both forest and urban point‑cloud benchmarks.

In summary, the paper makes three substantive contributions: (1) a robust, uncertainty‑aware TLS annotation pipeline with integrated visualization tools, (2) the publicly released Mangrove3D dataset, and (3) empirical insights into data efficiency and feature importance that enable scalable, high‑quality TLS segmentation for ecological monitoring and related applications. The open‑source code and dataset are available at the provided URL, inviting the community to adopt and extend the workflow.