Decentralized Domain Generalization with Style Sharing: Formal Model and Convergence Analysis

Much of federated learning (FL) focuses on settings where local dataset statistics remain the same between training and testing. However, this assumption often does not hold in practice due to distribution shifts, motivating the development of domain generalization (DG) approaches that leverage source domain data to train models capable of generalizing to unseen target domains. In this paper, we are motivated by two major gaps in existing work on FL and DG: (1) the lack of formal mathematical analysis of DG objectives; and (2) DG research in FL being limited to the star-topology architecture. We develop Decentralized Federated Domain Generalization with Style Sharing ($\textit{StyleDDG}$), a decentralized DG algorithm which allows devices in a peer-to-peer network to achieve DG based on sharing style information inferred from their datasets. Additionally, we provide the first systematic approach to analyzing style-based DG training in decentralized networks. We cast existing centralized DG algorithms within our framework, and employ their formalisms to model $\textit{StyleDDG}$. We then obtain analytical conditions under which convergence of $\textit{StyleDDG}$ can be guaranteed. Through experiments on popular DG datasets, we demonstrate that $\textit{StyleDDG}$ can obtain significant improvements in accuracy across target domains with minimal communication overhead compared to baseline decentralized gradient methods.

💡 Research Summary

The paper addresses a critical gap at the intersection of federated learning (FL) and domain generalization (DG). While most FL research assumes that training and test data share the same distribution, real‑world deployments (e.g., autonomous vehicles, edge cameras) often encounter distribution shifts, making DG essential. Existing DG work in FL is limited to star‑topology settings and lacks a rigorous mathematical formulation of the DG objective. To fill these voids, the authors propose StyleDDG, a fully decentralized DG algorithm that operates over a peer‑to‑peer (P2P) network.

Key technical contributions

1. Formalization of style‑based DG objectives – The authors start by rigorously defining the AdaIN operation and higher‑order style statistics (mean µ, variance σ, and second‑order variances Σ²µ, Σ²σ). Using these, they express the loss of popular style‑based DG methods (e.g., MixStyle, DSU) as a weighted sum of (i) the standard empirical loss on raw samples and (ii) the loss on style‑shifted or style‑mixed synthetic samples. This yields a unified objective that is L‑smooth and μ‑strongly convex under standard assumptions.

2. Decentralized algorithm design – In a connected graph G=(M,E), each device i maintains local parameters θ_i and a local dataset D_i that includes a domain label d_i. At each round, devices: (a) perform a local SGD step on their raw mini‑batch, (b) exchange model parameters and the batch‑level style statistics (µ,σ,Σ²µ,Σ²σ) with one‑hop neighbors, (c) apply neighbor‑provided statistics via AdaIN to a randomly selected subset of the batch (style‑shift), (d) concatenate original and shifted samples, then randomly select a portion for MixStyle extrapolation, and (e) compute the unified loss and update θ_i. The communication overhead is limited to the low‑dimensional style statistics plus occasional model averaging.

3. Convergence analysis – The update rule can be written as
   θ_i^{t+1}=∑{j∈N_i∪{i}} w{ij}θ_j^{t}−η∇θ L_i^{style}(θ_i^{t}),
   where w{ij} form a doubly‑stochastic matrix reflecting the consensus protocol. Under the assumptions that (i) the graph is connected, (ii) each local objective is L‑smooth and μ‑strongly convex, and (iii) the variance introduced by style‑mixing is bounded (≤σ²), the authors prove that the average model θ̄^{t}= (1/m)∑_i θ_i^{t} converges in expectation to the global optimum θ* with rate
   E