EchoJEPA: A Latent Predictive Foundation Model for Echocardiography

Foundation models for echocardiography often struggle to disentangle anatomical signal from the stochastic speckle and acquisition artifacts inherent to ultrasound. We present EchoJEPA, a foundation model trained on 18 million echocardiograms across 300K patients, representing the largest pretraining corpus for this modality to date. By leveraging a latent predictive objective, EchoJEPA learns robust anatomical representations that ignore speckle noise. We validate this using a novel multi-view probing framework with frozen backbones, where EchoJEPA outperforms leading baselines by approximately 20% in left ventricular ejection fraction (LVEF) estimation and 17% in right ventricular systolic pressure (RVSP) estimation. The model also exhibits remarkable sample efficiency, reaching 79% view classification accuracy with only 1% of labeled data versus 42% for the best baseline trained on 100%. Crucially, EchoJEPA demonstrates superior generalization, degrading by only 2% under physics-informed acoustic perturbations compared to 17% for competitors. Most remarkably, its zero-shot performance on pediatric patients surpasses fully fine-tuned baselines, establishing latent prediction as a superior paradigm for robust, generalizable medical AI.

💡 Research Summary

EchoJEPA introduces a large‑scale self‑supervised foundation model for echocardiography that explicitly addresses the unique noise characteristics of ultrasound imaging. The authors pre‑train the model on an unprecedented corpus of 18 million video clips from 300 K patients, using a Joint‑Embedding Predictive Architecture (JEPA) adapted for video (V‑JEP​A2). Instead of reconstructing masked pixels, the model predicts the embeddings of masked spatio‑temporal tubelets generated by an exponential‑moving‑average (EMA) teacher network. This latent‑prediction objective down‑weights stochastic speckle and acoustic shadows while reinforcing temporally coherent anatomical structures such as chamber geometry and wall motion.

Key architectural choices include: (1) a high temporal resolution of 24 fps to capture rapid cardiac dynamics; (2) domain‑specific augmentations that limit aspect‑ratio distortion and enforce a minimum crop scale, preserving the fan‑shaped ultrasound sector; (3) a factorized multi‑view probing framework that adds lightweight view‑ and clip‑level embeddings to frozen video tokens, applies random view dropout, and aggregates information across all available views with a shallow transformer probe. The probing head is identical for every baseline, ensuring that performance differences reflect representation quality alone.

Two model scales are released: EchoJEPA‑G (ViT‑Giant, 1.1 B parameters) trained on the proprietary 18 M‑clip dataset, and EchoJEPA‑L (ViT‑Large, 300 M parameters) trained on the publicly available MIMIC‑IV‑Echo (525 K clips) for reproducibility. For a fair comparison, the authors also train a VideoMAE model (pixel‑level masked autoencoding) with the same architecture, data, augmentations, and compute budget.

The evaluation spans three axes: (i) latent prediction versus pixel reconstruction, (ii) sample efficiency and robustness under distribution shift, and (iii) generalization across patient populations and multi‑view reasoning tasks. Experiments are conducted on internal Toronto and Chicago datasets (150 K and 60 K studies respectively) and on public benchmarks EchoNet‑Dynamic (10 030 adult studies) and EchoNet‑Pediatric (3 316 pediatric studies).

Results show that EchoJEPA outperforms the pixel‑based VideoMAE by a substantial margin: left ventricular ejection fraction (LVEF) root‑mean‑square error improves from 0.07 to 0.05 and right ventricular systolic pressure (RVSP) mean absolute error drops from 2.3 mmHg to 1.8 mmHg. Sample efficiency is striking—using only 1 % of labeled data (≈1 k studies) the model reaches 79 % view‑classification accuracy, whereas the best baseline reaches 42 % even with the full label set. Robustness tests simulate depth attenuation and acoustic shadows with physics‑informed perturbations; EchoJEPA’s performance degrades by only 2 % compared with a 17 % drop for VideoMAE and other baselines. Zero‑shot transfer to the pediatric cohort yields an LVEF R² of 0.78, surpassing fully fine‑tuned supervised EchoNet‑Dynamic (R² = 0.71).

The paper contributes a standardized multi‑view probing protocol, a comprehensive robustness benchmark tailored to ultrasound, and open‑source releases of EchoJEPA‑L and the evaluation code. Limitations include reliance on predominantly internal data (potential vendor and demographic bias) and a lack of sensitivity analysis for EMA decay rate and masking ratio. Future work should explore multimodal extensions (video‑report alignment), real‑time inference for bedside deployment, and broader cross‑institutional validation.

In summary, EchoJEPA demonstrates that latent‑prediction self‑supervision is a superior paradigm for ultrasound video representation learning. By suppressing stochastic speckle and emphasizing anatomically meaningful dynamics, the model achieves state‑of‑the‑art performance on clinically relevant tasks, exhibits remarkable sample efficiency, and maintains robustness and generalization across diverse acquisition conditions and patient groups. This work paves the way for more reliable, scalable AI assistance in cardiac ultrasound diagnostics.