DFCA: Decentralized Federated Clustering Algorithm

Clustered Federated Learning has emerged as an effective approach for handling heterogeneous data across clients by partitioning them into clusters with similar or identical data distributions. However, most existing methods, including the Iterative Federated Clustering Algorithm (IFCA), rely on a central server to coordinate model updates, which creates a bottleneck and a single point of failure, limiting their applicability in more realistic decentralized learning settings. In this work, we introduce DFCA, a fully decentralized clustered FL algorithm that enables clients to collaboratively train cluster-specific models without central coordination. DFCA uses a sequential running average to aggregate models from neighbors as updates arrive, providing a communication-efficient alternative to batch aggregation while maintaining clustering performance. Our experiments on various datasets demonstrate that DFCA outperforms other decentralized algorithms and performs comparably to centralized IFCA, even under sparse connectivity, highlighting its robustness and practicality for dynamic real-world decentralized networks.

💡 Research Summary

The paper addresses a fundamental limitation of existing clustered federated learning (FL) methods: their reliance on a central server for model aggregation and client clustering. While the Iterative Federated Clustering Algorithm (IFCA) achieves strong performance in heterogeneous data settings, it assumes stable, synchronous communication and a globally reachable coordinator—assumptions that break down in realistic Internet‑of‑Things (IoT) or edge scenarios where devices have intermittent, sparse, or peer‑to‑peer connectivity.

To overcome this, the authors propose DFCA (Decentralized Federated Clustering Algorithm), a fully decentralized clustered FL framework that operates without any central orchestrator. The key ideas are:

1. Loss‑based client assignment – each client stores k model copies (one per potential cluster). At the beginning of every round it evaluates all k models on its local data and assigns itself to the cluster whose model yields the smallest empirical loss. This mirrors IFCA’s loss‑based clustering but is performed locally and repeatedly, allowing dynamic reassignment as training progresses.

2. Local SGD on the assigned cluster – after assignment, a client performs τ stochastic gradient descent steps (or any local optimizer) on the model belonging to its chosen cluster, using its private dataset. Only this model is updated; the other k‑1 models remain unchanged.

3. Decentralized parameter exchange – after local training, each client exchanges model parameters with its immediate neighbors in the communication graph. The novel contribution is a sequential running‑average aggregation rule: when a neighbor’s model for cluster j arrives, the client updates its own copy of that cluster’s parameters as
   \