Leveraging Foundational Models and Simple Fusion for Multi-modal Physiological Signal Analysis

📝 Original Info

• Title: Leveraging Foundational Models and Simple Fusion for Multi-modal Physiological Signal Analysis
• ArXiv ID: 2512.15250
• Date: 2025-12-17
• Authors: Youssef Ghallab, Omar Iraqy, Mohamed Kandil, Mohamed Ashraf, Saadeldine Eletter, Morougue Ghazal, Ayman Khalafallah, Nagwa El-Makky

📝 Abstract

Physiological signals such as electrocardiograms (ECG) and electroencephalograms (EEG) provide complementary insights into human health and cognition, yet multi-modal integration is challenging due to limited multi-modal labeled data, and modality-specific differences . In this work, we adapt the CBraMod encoder [1] for large-scale self-supervised ECG pretraining, introducing a dual-masking strategy to capture intra-and inter-lead dependencies. To overcome the above challenges, we utilize a pre-trained CBraMod encoder [1] for EEG and pre-train a symmetric ECG encoder, equipping each modality with a rich foundational representation. These representations are then fused via simple embedding concatenation, allowing the classification head to learn cross-modal interactions, together enabling effective downstream learning despite limited multi-modal supervision. Evaluated on emotion recognition, our approach achieves near state-of-the-art performance, demonstrating that carefully designed physiological encoders, even with straightforward fusion, substantially improve downstream performance. These results highlight the potential of ...

📄 Full Content

...(본문 내용이 길어 생략되었습니다. 사이트에서 전문을 확인해 주세요.)