FedSUM Family: Efficient Federated Learning Methods under Arbitrary Client Participation

Federated Learning (FL) methods are often designed for specific client participation patterns, limiting their applicability in practical deployments. We introduce the FedSUM family of algorithms, which supports arbitrary client participation without additional assumptions on data heterogeneity. Our framework models participation variability with two delay metrics, the maximum delay $τ_{\max}$ and the average delay $τ_{\text{avg}}$. The FedSUM family comprises three variants: FedSUM-B (basic version), FedSUM (standard version), and FedSUM-CR (communication-reduced version). We provide unified convergence guarantees demonstrating the effectiveness of our approach across diverse participation patterns, thereby broadening the applicability of FL in real-world scenarios.

💡 Research Summary

Federated learning (FL) enables large‑scale model training while keeping raw data on distributed devices such as smartphones, sensors, or hospitals. A major obstacle in real‑world deployments is the irregular availability of clients: network outages, battery constraints, or computational load cause only a subset of clients to participate in any given round. Existing FL algorithms typically assume a specific participation pattern—uniform random sampling, fixed probability models, or deterministic cyclic schedules—and often rely on additional assumptions about data heterogeneity (e.g., bounded gradient divergence) or require multiple local updates that can introduce bias under non‑i.i.d. data. Consequently, these methods are fragile when faced with truly arbitrary client participation.

The paper introduces the FedSUM family, a set of FL algorithms designed to operate under arbitrary client participation without imposing extra heterogeneity constraints. The authors first formalize participation variability using two delay metrics:

• Maximum delay τ_max – the largest gap (in rounds) between the current round and the most recent round any client was active.
• Average delay τ_avg – the mean of per‑round delays across the entire training horizon.

These metrics capture both worst‑case and typical participation frequencies, and they can be computed for any deterministic or stochastic schedule, including the four canonical patterns discussed in the literature (uniform random, independent probability‑based, deterministic cyclic, reshuffled cyclic).

The core algorithmic innovation is the Stochastic Uplink‑Merge technique. In each round t, every active client i computes a mini‑batch average stochastic gradient at the current global model x(t). Rather than sending the full gradient, the client transmits only the difference δ_i(t) between this freshly computed average and the gradient it sent the last time it participated. If the client is participating for the first time, δ_i(t) equals the full average gradient. The server aggregates all received differences, forming a global control variable y(t) that effectively represents the sum of the most recent gradients from every client that has ever been active. Because y(t) is built from delayed but still recent information, more frequent client participation (smaller τ_max, τ_avg) keeps y(t) close to the true current global gradient, mitigating the drift caused by data heterogeneity.

Three algorithmic variants are presented:

1. FedSUM‑B (Basic) – uses a single scalar for both uplink and downlink communication and omits local model updates entirely. This yields communication and memory costs identical to the classic FedAvg while still achieving robustness to arbitrary participation.

2. FedSUM (Standard) – adds a per‑client control variable h_i(t) on the client side and a global control variable y(t) on the server, mirroring the variance‑reduction strategy of SCAFFOLD. Despite this extra bookkeeping, the communication cost remains that of a single variable per direction, matching SCAFFOLD’s convergence speed without its higher bandwidth demand.

3. FedSUM‑CR (Communication‑Reduced) – further compresses downlink traffic by also sending only differences for the server‑to‑client broadcast, achieving up to ~30 % reduction in total transmitted bytes while preserving convergence guarantees.

The theoretical analysis assumes only L‑smoothness of each local loss and a bounded variance σ² of stochastic gradients. No additional constraints on the divergence between local and global objectives are required. The main convergence theorem shows that the error after T rounds scales as O( (τ_max + τ_avg)/√T ) for non‑convex objectives and as O( (τ_max + τ_avg)/T ) for strongly convex objectives. In other words, the delay metrics appear linearly in the convergence bound, confirming the intuition that more frequent participation accelerates learning. Moreover, by plugging in specific values of τ_max and τ_avg for the four canonical participation patterns, the authors recover the known rates of FedAvg, SCAFFOLD, and other specialized algorithms, demonstrating that FedSUM is a unified framework that subsumes existing methods as special cases.

Empirical evaluation covers four participation patterns (uniform random, independent probability, deterministic cyclic, reshuffled cyclic) on heterogeneous datasets (CIFAR‑10 with label‑skew, FEMNIST with user‑skew, and the Shakespeare language modeling task). Across all settings, FedSUM‑B, FedSUM, and FedSUM‑CR consistently outperform baseline algorithms in terms of final test accuracy and number of communication rounds needed to reach a target loss. Notably, even when τ_max is as high as 20 rounds, FedSUM variants lose less than 2 % accuracy compared to the ideal case of τ_max = 0, whereas FedAvg and SCAFFOLD experience larger degradations. The communication‑reduced variant (FedSUM‑CR) achieves comparable accuracy to FedSUM while transmitting roughly one‑third fewer bytes, confirming its practicality for bandwidth‑limited environments.

In summary, the paper makes three major contributions:

• A universal quantification of client participation variability via τ_max and τ_avg, enabling a single analytical treatment of arbitrary schedules.
• The Stochastic Uplink‑Merge mechanism, which leverages delayed gradient information to correct for data heterogeneity without extra communication overhead.
• A family of algorithms (FedSUM‑B, FedSUM, FedSUM‑CR) that attain state‑of‑the‑art convergence rates under minimal assumptions, while matching or improving the communication efficiency of FedAvg and SCAFFOLD.

The work opens several promising directions: adaptive estimation of τ_max/τ_avg during training, integration with differential privacy mechanisms, and extension to fully asynchronous server updates. Overall, FedSUM provides a robust, communication‑efficient foundation for deploying federated learning in the highly variable, heterogeneous environments encountered in practice.