Unsupervised Segmentation of Micro-CT Scans of Polyurethane Structures By Combining Hidden-Markov-Random Fields and a U-Net

Extracting digital material representations from images is a necessary prerequisite for a quantitative analysis of material properties. Different segmentation approaches have been extensively studied in the past to achieve this task, but were often lacking accuracy or speed. With the advent of machine learning, supervised convolutional neural networks (CNNs) have achieved state-of-the-art performance for different segmentation tasks. However, these models are often trained in a supervised manner, which requires large labeled datasets. Unsupervised approaches do not require ground-truth data for learning, but suffer from long segmentation times and often worse segmentation accuracy. Hidden Markov Random Fields (HMRF) are an unsupervised segmentation approach that incorporates concepts of neighborhood and class distributions. We present a method that integrates HMRF theory and CNN segmentation, leveraging the advantages of both areas: unsupervised learning and fast segmentation times. We investigate the contribution of different neighborhood terms and components for the unsupervised HMRF loss. We demonstrate that the HMRF-UNet enables high segmentation accuracy without ground truth on a Micro-Computed Tomography ($μ$CT) image dataset of Polyurethane (PU) foam structures. Finally, we propose and demonstrate a pre-training strategy that considerably reduces the required amount of ground-truth data when training a segmentation model.

💡 Research Summary

The paper introduces HMRF‑UNet, a novel unsupervised segmentation framework that combines Hidden Markov Random Field (HMRF) theory with a conventional U‑Net architecture to segment micro‑computed tomography (µCT) images of polyurethane (PU) foam structures without requiring ground‑truth labels. Traditional segmentation methods—threshold‑based, region‑based, model‑based, and pixel‑classification—either lack accuracy, are computationally slow, or depend heavily on annotated data. While supervised convolutional neural networks (CNNs), especially U‑Nets, achieve state‑of‑the‑art performance, they demand large labeled datasets that are costly or unavailable for many material imaging tasks. Conversely, classic HMRF approaches can operate without labels but rely on iterative optimization (EM‑ICM or evolutionary algorithms) that is orders of magnitude slower and cannot leverage prior experience across images.

The authors address these limitations by embedding the HMRF energy formulation directly into the loss function of a U‑Net. They first recast the discrete HMRF energy terms—data likelihood and neighborhood regularization—into differentiable “fuzzy” versions that operate on the soft probability maps produced by the network. Class means and variances are computed as weighted averages using the network’s confidence values, yielding fuzzy mean (eµ) and fuzzy standard deviation (eσ). The data loss L_d (Eq. 12) is a log‑likelihood based on these fuzzy statistics. Two neighborhood penalties are considered: a fuzzy Potts term (average Euclidean distance between neighboring confidence vectors) and a fuzzy Banerjee term (which also incorporates differences in fuzzy class means and variances). Voxel‑wise adaptive weights α_s are introduced, derived from local standard‑deviation thresholds, to modulate the strength of the neighborhood regularizer.

The overall loss is a weighted sum L = λ_d L_d + λ_n L_n, with λ_d = 1 − λ_n. Hyper‑parameter searches (Bayesian optimization) identified a three‑level U‑Net (64 initial filters, three convolutions per level, max‑pooling) and an optimal λ_n ≈ 0.31. The network is trained on two datasets: (1) a synthetic 2‑D PU‑foam collection (ArtPUF) comprising 20 000 images with ground‑truth masks, and (2) a real 3‑D µCT volume (RealPUF) from which 2‑D slices are extracted. For RealPUF, a cuboid‑based split prevents data leakage between training, validation, and test sets; data augmentation (contrast scaling, rotations, flips) expands the training pool to 42 000 slices.

Extensive experiments evaluate the impact of neighborhood type (Potts vs. Banerjee), weighting (plain vs. voxel‑wise), and the σ_thresh parameter governing α_s. Results show that the weighted Banerjee configuration (HMRF‑UNet wban) with λ_n = 0.31 and σ_thresh ≈ 0.10 achieves the highest Dice coefficient (≈ 0.96) on the synthetic test set, outperforming plain Potts models by ~5 %. The method also dramatically reduces inference time: while EM‑ICM or evolutionary solvers require seconds to minutes per volume, HMRF‑UNet processes a full 3‑D volume in ~0.02 s on an NVIDIA A100 GPU.

A pre‑training study demonstrates that initializing a supervised U‑Net with weights learned from HMRF‑UNet on the synthetic data substantially improves performance when only a small fraction (5–20 %) of labeled real images are available. Dice scores improve by 0.07–0.12 compared to training from scratch, highlighting the practical benefit of unsupervised pre‑training for domains where annotations are scarce.

In summary, the paper delivers a compelling solution that merges the statistical rigor of HMRF with the computational efficiency of deep learning. By formulating differentiable fuzzy HMRF losses, the authors enable end‑to‑end training of a segmentation network that requires no manual labels yet attains near‑supervised accuracy and real‑time inference. The approach is broadly applicable to other volumetric imaging fields (e.g., biomedical microscopy, medical CT/MRI) and opens avenues for multi‑class extensions, multi‑scale architectures, and integration with other probabilistic graphical models.