Learning Reconfigurable Representations for Multimodal Federated Learning with Missing Data

📝 Original Info

• Title: Learning Reconfigurable Representations for Multimodal Federated Learning with Missing Data
• ArXiv ID: 2510.22880
• Date: 2025-10-27
• Authors: ** 제공된 정보에 저자 명단이 포함되어 있지 않아 “정보 없음” 으로 표기합니다. (논문 원문 혹은 DOI를 확인하면 정확한 저자 정보를 얻을 수 있습니다.) **

📝 Abstract

Multimodal federated learning in real-world settings often encounters incomplete and heterogeneous data across clients. This results in misaligned local feature representations that limit the effectiveness of model aggregation. Unlike prior work that assumes either differing modality sets without missing input features or a shared modality set with missing features across clients, we consider a more general and realistic setting where each client observes a different subset of modalities and might also have missing input features within each modality. To address the resulting misalignment in learned representations, we propose a new federated learning framework featuring locally adaptive representations based on learnable client-side embedding controls that encode each client's data-missing patterns. These embeddings serve as reconfiguration signals that align the globally aggregated representation with each client's local context, enabling more effective use of shared information. Furthermore, the embedding controls can be algorithmically aggregated across clients with similar data-missing patterns to enhance the robustness of reconfiguration signals in adapting the global representation. Empirical results on multiple federated multimodal benchmarks with diverse data-missing patterns across clients demonstrate the efficacy of the proposed method, achieving up to 36.45\% performance improvement under severe data incompleteness. The method is also supported by a theoretical analysis with an explicit performance bound that matches our empirical observations. Our source codes are provided at https://github.com/nmduonggg/PEPSY

💡 Deep Analysis

📄 Full Content

📸 Image Gallery

Reference