Foundation models (FMs) are rapidly reshaping medical imaging, shifting the field from narrowly trained, task-specific networks toward large, general-purpose models that can be adapted across modalities, anatomies, and clinical tasks. In this review, we synthesize the emerging landscape of medical imaging FMs along three major axes: principles of FM design, applications of FMs, and forward-looking challenges and opportunities. Taken together, this review provides a technically grounded, clinically aware, and future-facing roadmap for developing FMs that are not only powerful and versatile but also trustworthy and ready for responsible translation into clinical practice.
rtificial intelligence (AI) for medical imaging is experiencing a transformative shift from task-specific models toward foundation models (FMs), which are large artificial neural networks pre-trained on vast, diverse datasets and adapted efficiently to a variety of downstream tasks. In medical imaging, where labels are scarce, heterogeneous, and expensive, FMs show a strong promise for rapid adaptation with minimal annotation, improved generalization across sites, scanners, and populations, and a plausible route to "generalist" medical imaging assistants that reason across contexts.
Recent overviews from both the radiology and computer vision communities document a surge of FM research, spanning 2D/3D segmentation, image-text representation learning through vision-language fusion, and generative models. Together, these developments motivate a new synthesis of principles, capabilities, and translational considerations tailored to the healthcare ecosystem [1].
To contextualize foundation models, we begin by exploring their relationship with the broader AI landscape, coupled with Figure 1.1 illustrating the relative timelines of the related areas along with some seminal publications. AI refers to non-human systems performing tasks that mimic human perception and reasoning, such as language understanding and image analysis. Machine learning, a subset of AI, trains models to detect patterns in data, evolving from simple statistical methods to more sophisticated tools like random forests and support vector machines. Deep learning uses multi-layer artificial neural networks to represent data in a data-driven fashion, leading to advanced architectures, like Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Graph Neural Networks (GNNs), and Transformers.
The term foundation model was coined by the Center for Research on Foundation Models at the Stanford Institute for Human-Centered Artificial Intelligence in August 2021 [19]. Foundation models are a class of deep learning models that are initially trained based on a diverse dataset for broad applications, and that can then be fine-tuned for specific downstream applications. Typically, they are initially trained in a self-supervised fashion. These pre-trained FMs then serve as the basis for developing task-specific models through transfer learning. The term foundation model is sometimes used loosely: a critical examination of the criteria for a model to qualify as a foundation model is given in [20].
Foundation models are characterized by enormous training data and parameter counts, which lead to emergent capabilities that do not present in smaller models. In other words, a foundation model serves as a general-purpose platform that, with minimal task-specific training, can achieve strong performance across a variety of tasks. Another hallmark of foundation models is scalability. Their performance improves predictably as model size, training data, and amount of compute increase, following empirical scaling laws. This scaling yields surprising capabilities, e.g., GPT-3 demonstrated in-context learning to solve tasks it was not explicitly trained for. Foundation models also exhibit strong generalization and transferability, meaning that the knowledge captured during pretraining on broad data can be transferred to unseen tasks. A single pretrained model can be fine-tuned to excel in applications ranging from natural language processing (NLP) to computer vision and robotics. This versatility has incentivized homogenization of AI research around a few architectures, especially the Transformer. However, this also means any defects or biases in a foundation model might propagate to its downstream uses.
We first introduce several previous review papers related to foundation models. A comprehensive survey of self-supervised A Turing [2], Deep Blue [3], MYCIN [4], SHRDLU [5] IBM [6], Linear Regression [7], SVMs [8], Random Forests [9] LeCun [10], AlexNet [11], ResNet [12], LSTM [13], GAN [14], U-Net [15] DeepMind AlphaFold [16], Google Med-PaLM [17], Bommasani [18]. learning (SSL) is provided by Liu et al in 2021 [21]. Two 2023 surveys by Kazerooni et al. and by Yang et al [22], [23] provide in-depth overviews of the rapidly evolving field of diffusion models, which are increasingly being integrated into foundation models. A more recent review of generative models is provided by Hein et al [24]. Longpre et al. [25] present a practical guide to support responsible and transparent development of FMs across text, vision, and speech modalities.
Large language models (LLMs) are the most popular type of FMs. Zhou et al. [26] trace the evolution from BERT to ChatGPT, emphasizing key advancements in architecture, training methods, and model capabilities. Zhao et al. [27] summarize LLMs and emerging trends like multi-agent collaboration and chain-of-thought reasoning. Ian A. Scott [28] introduces physicians to FMs and LLMs, explaining how they can perform div
This content is AI-processed based on open access ArXiv data.