Bridging Performance Gaps for ECG Foundation Models: A Post-Training Strategy

ECG foundation models are increasingly popular due to their adaptability across various tasks. However, their clinical applicability is often limited by performance gaps compared to task-specific models, even after pre-training on large ECG datasets and fine-tuning on target data. This limitation is likely due to the lack of an effective post-training strategy. In this paper, we propose a simple yet effective post-training approach to enhance ECG foundation models. We evaluate it on a publicly available Transformer-based foundation model. Experiments across multiple ECG tasks show that our method consistently outperforms baseline fine-tuning. On the PTB-XL benchmarks, it improves macro AUROC by 0.7%-8.9% and macro AUPRC by 23.3%-77.9%, also outperforming several recent state-of-the-art approaches, including task-specific and advanced architectures. Further analyses demonstrate improved training dynamics and data efficiency, with only 30% of the training data outperforming the baseline trained on the full dataset. Ablation studies highlight the importance of stochastic depth and preview linear probing. These findings underscore the potential of post-training strategies to improve ECG foundation models, and we hope this work will contribute to the continued development of foundation models in the ECG domain.

💡 Research Summary

The paper addresses a persistent gap between the performance of ECG foundation models and that of task‑specific architectures when transferred to clinical downstream tasks. Although large‑scale pre‑training endows these models with generalizable cardiac representations, standard fine‑tuning often leaves them trailing behind specialized networks. To bridge this gap, the authors propose a simple yet powerful two‑stage post‑training strategy that can be applied to any transformer‑based ECG foundation model (and, by extension, to CNN‑based models). In the first “initialization” stage, the pre‑trained backbone is frozen, the original classification head is replaced with a task‑specific linear head, and a brief linear‑probing phase is used to initialize this head. This step aligns the new head with the rich features already learned during pre‑training while providing a well‑conditioned starting point for downstream learning. In the second “regularization” stage, the entire network is fine‑tuned with three complementary regularization techniques: stochastic depth applied to all residual connections to reduce redundancy and mitigate vanishing‑gradient issues; dropout on the feed‑forward layers to curb over‑fitting; and a cosine‑annealed learning‑rate schedule together with AdamW weight decay to ensure stable convergence. The authors evaluate the approach on the PTB‑XL benchmark, using the publicly available transformer‑FM ST‑MEM as a representative backbone. Across multiple ECG classification tasks, the proposed method yields consistent gains: macro‑AUROC improvements ranging from 0.7 % to 8.9 % points and macro‑AUPRC boosts from 23.3 % to 77.9 % points relative to the baseline fine‑tuning used in the original ST‑MEM paper. Remarkably, training with only 30 % of the available data surpasses the baseline trained on the full dataset, highlighting substantial data‑efficiency benefits. Compared against recent state‑of‑the‑art task‑specific models and other foundation models, the post‑training strategy remains competitive or superior. Ablation studies confirm that stochastic depth and the preview linear‑probing step are the primary contributors to the observed performance gains; removing either component leads to noticeable degradation. The authors also note that the method requires minimal architectural changes and can be readily integrated into existing pipelines. In conclusion, the work demonstrates that an effective post‑training regimen can dramatically improve the clinical utility of ECG foundation models, offering a practical path toward more robust and data‑efficient cardiac AI systems.