FedSCAM (Federated Sharpness-Aware Minimization with Clustered Aggregation and Modulation): Scam-resistant SAM for Robust Federated Optimization in Heterogeneous Environments

Federated Learning (FL) enables collaborative model training across decentralized edge devices while preserving data privacy. However, statistical heterogeneity among clients, often manifested as non-IID label distributions, poses significant challenges to convergence and generalization. While Sharpness-Aware Minimization (SAM) has been introduced to FL to seek flatter, more robust minima, existing approaches typically apply a uniform perturbation radius across all clients, ignoring client-specific heterogeneity. In this work, we propose FedSCAM (Federated Sharpness-Aware Minimization with Clustered Aggregation and Modulation), a novel algorithm that dynamically adjusts the SAM perturbation radius and aggregation weights based on client-specific heterogeneity scores. By calculating a heterogeneity metric for each client and modulating the perturbation radius inversely to this score, FedSCAM prevents clients with high variance from destabilizing the global model. Furthermore, we introduce a heterogeneity-aware weighted aggregation mechanism that prioritizes updates from clients that align with the global optimization direction. Extensive experiments on CIFAR-10 and Fashion-MNIST under various degrees of Dirichlet-based label skew demonstrate that FedSCAM achieves competitive performance among state-of-the-art baselines, including FedSAM, FedLESAM, etc. in terms of convergence speed and final test accuracy. Code and artifacts available at: Github Repository.

💡 Research Summary

Federated learning (FL) enables multiple edge devices to collaboratively train a shared model while keeping raw data locally, but statistical heterogeneity—especially non‑IID label distributions—remains a major obstacle to fast convergence and strong generalization. Sharpness‑Aware Minimization (SAM) has been introduced to FL (e.g., FedSAM, FedLESAM) to seek flatter minima that are more robust to perturbations. However, all existing SAM‑based FL methods use a single, uniform perturbation radius (ε) for every client, ignoring the fact that some clients possess highly skewed data and can destabilize the global model when subjected to the same magnitude of adversarial perturbation. Moreover, conventional aggregation (FedAvg) treats all client updates equally or weights them only by data volume, which can give disproportionate influence to high‑heterogeneity clients whose updates may be misaligned with the global descent direction.

The paper proposes FedSCAM (Federated Sharpness‑Aware Minimization with Clustered Aggregation and Modulation), a novel framework that (1) quantifies each client’s heterogeneity, (2) modulates the SAM perturbation radius inversely to this heterogeneity score, and (3) performs a heterogeneity‑aware weighted aggregation that emphasizes updates aligned with the global gradient.

Heterogeneity Scoring

For each client c, the method computes a heterogeneity score HS₍c₎ by measuring the divergence between the client’s label distribution p_c and the global label distribution p_g. The authors use either KL‑divergence D_KL(p_c‖p_g) or a Dirichlet‑based statistic derived from the concentration parameter α used to generate label skew. A larger HS indicates a more “out‑of‑distribution” client.

Dynamic SAM Radius

The SAM perturbation radius for client c is defined as
\