Personalized Federated Learning with Bidirectional Communication Compression via One-Bit Random Sketching

Federated Learning (FL) enables collaborative training across decentralized data, but faces key challenges of bidirectional communication overhead and client-side data heterogeneity. To address communication costs while embracing data heterogeneity, we propose pFed1BS, a novel personalized federated learning framework that achieves extreme communication compression through one-bit random sketching. In personalized FL, the goal shifts from training a single global model to creating tailored models for each client. In our framework, clients transmit highly compressed one-bit sketches, and the server aggregates and broadcasts a global one-bit consensus. To enable effective personalization, we introduce a sign-based regularizer that guides local models to align with the global consensus while preserving local data characteristics. To mitigate the computational burden of random sketching, we employ the Fast Hadamard Transform for efficient projection. Theoretical analysis guarantees that our algorithm converges to a stationary neighborhood of the global potential function. Numerical simulations demonstrate that pFed1BS substantially reduces communication costs while achieving competitive performance compared to advanced communication-efficient FL algorithms.

💡 Research Summary

The paper introduces pFed1BS, a personalized federated learning (PFL) framework that simultaneously tackles two fundamental challenges in federated learning: data heterogeneity across clients and the prohibitive communication cost of transmitting high‑dimensional model parameters. The key innovation is the use of one‑bit random sketching for both uplink and downlink communication. Each client projects its local model wₖ onto a low‑dimensional random subspace using a matrix Φ and then transmits only the sign of the projected vector, i.e., sign(Φwₖ). The server aggregates these one‑bit sketches by a weighted majority vote (weights pₖ reflect the relative data size of each client) and broadcasts the resulting one‑bit consensus vector v back to all clients.

To make this extreme compression compatible with personalization, the authors design a sign‑based regularizer. For a given global consensus v, the regularizer
g(v, Φwₖ) = ½(‖Φwₖ‖₁ − ⟨v, Φwₖ⟩)
penalizes disagreement between the signs of the projected local model and the global consensus. Because the ℓ₁ term is non‑smooth, they replace it with a smooth approximation hγ(z) = (1/γ)∑ᵢ log cosh(γzᵢ). The resulting smoothed client objective is
\tilde Fₖ(wₖ; v) = fₖ(wₖ) + λ