MONAI Label: A framework for AI-assisted Interactive Labeling of 3D Medical Images

The lack of annotated datasets is a major bottleneck for training new task-specific supervised machine learning models, considering that manual annotation is extremely expensive and time-consuming. To address this problem, we present MONAI Label, a free and open-source framework that facilitates the development of applications based on artificial intelligence (AI) models that aim at reducing the time required to annotate radiology datasets. Through MONAI Label, researchers can develop AI annotation applications focusing on their domain of expertise. It allows researchers to readily deploy their apps as services, which can be made available to clinicians via their preferred user interface. Currently, MONAI Label readily supports locally installed (3D Slicer) and web-based (OHIF) frontends and offers two active learning strategies to facilitate and speed up the training of segmentation algorithms. MONAI Label allows researchers to make incremental improvements to their AI-based annotation application by making them available to other researchers and clinicians alike. Additionally, MONAI Label provides sample AI-based interactive and non-interactive labeling applications, that can be used directly off the shelf, as plug-and-play to any given dataset. Significant reduced annotation times using the interactive model can be observed on two public datasets.

💡 Research Summary

The paper introduces MONAI Label, an open‑source framework designed to accelerate the creation of annotated 3D medical imaging datasets by integrating AI‑assisted interactive labeling with existing clinical workflows. The system follows a client‑server architecture: the server, built on PyTorch and the MONAI library, hosts deep learning models, active‑learning pipelines, and a heuristic planner, exposing all functionality through a RESTful API. On the client side, the framework currently supports two widely used viewers—3D Slicer (desktop) and OHIF (web)—and can communicate with any DICOM‑compliant system via standard DICOM‑REST services (QIDO‑RS, WADO‑RS, STOW‑RS), enabling seamless integration with PACS, XNAT, or Image Data Commons.

MONAI Label offers three distinct interactive segmentation approaches. DeepGrow implements click‑based guidance: users place positive (foreground) and negative (background) clicks, which are transformed into distance maps and concatenated as extra input channels to a U‑Net‑style network. During training, simulated clicks derived from false‑positive/false‑negative regions teach the model to respond to user input. DeepEdit extends DeepGrow by providing an initial click‑free inference (standard segmentation) followed by optional click‑based refinement; its training alternates between click‑free epochs and epochs with simulated clicks, yielding a model that works both autonomously and interactively. The Scribbles‑based method accepts free‑hand scribbles, either to generate a segmentation directly via energy‑based graph‑cut optimization or to refine a CNN prediction by feeding the scribble masks as additional channels. These options give annotators flexibility in the amount of interaction required for simple versus complex structures.

For fully automatic annotation, MONAI Label can deploy any segmentation network available in the MONAI core (e.g., 3D U‑Net, HighResNet, DynUNet). The API mirrors that of the interactive methods, but without click channels, allowing the output to be edited later using standard tools in the client viewer.

A key contribution is the built‑in active‑learning engine. The framework estimates both aleatoric and epistemic uncertainty using Monte‑Carlo dropout at inference time. Images with the highest uncertainty are prioritized for expert labeling, dramatically reducing the number of samples that need manual annotation while still improving model performance. The heuristic planner automatically selects optimal batch sizes, learning rates, and GPU memory configurations based on the hardware, ensuring efficient utilization of computational resources.

The authors evaluated the system on two public datasets (including a Medical Segmentation Decathlon task and the KiTS19 kidney tumor dataset). Compared with conventional manual annotation, the interactive models (DeepGrow and DeepEdit) reduced labeling time by roughly 30–50 % without sacrificing Dice similarity scores; in some cases, the interactive refinement even improved accuracy over the baseline automatic model. When active learning was employed, only about 20 % of the unlabeled pool needed to be annotated to reach comparable performance, confirming the efficiency of the uncertainty‑driven sampling strategy.

Installation is straightforward via PyPI (pip install monailabel), which also pulls in the pre‑built OHIF viewer. Comprehensive documentation, sample applications, and Docker images are provided to facilitate rapid adoption. The paper discusses future directions such as multi‑modal (CT + MRI) support, more sophisticated Bayesian uncertainty estimation, and large‑scale clinical validation.

In summary, MONAI Label unifies interactive segmentation, automatic model deployment, active learning, and DICOM integration into a single, extensible platform. By lowering the technical barrier for researchers to create and share AI‑assisted labeling tools, it promises to accelerate the generation of high‑quality annotated datasets—an essential step toward robust, clinically‑ready deep learning solutions in medical imaging.