Federated Gaussian Process Learning via Pseudo-Representations for Large-Scale Multi-Robot Systems

Multi-robot systems require scalable and federated methods to model complex environments under computational and communication constraints. Gaussian Processes (GPs) offer robust probabilistic modeling, but suffer from cubic computational complexity, limiting their applicability in large-scale deployments. To address this challenge, we introduce the pxpGP, a novel distributed GP framework tailored for both centralized and decentralized large-scale multi-robot networks. Our approach leverages sparse variational inference to generate a local compact pseudo-representation. We introduce a sparse variational optimization scheme that bounds local pseudo-datasets and formulate a global scaled proximal-inexact consensus alternating direction method of multipliers (ADMM) with adaptive parameter updates and warm-start initialization. Experiments on synthetic and real-world datasets demonstrate that pxpGP and its decentralized variant, dec-pxpGP, outperform existing distributed GP methods in hyperparameter estimation and prediction accuracy, particularly in large-scale networks.

💡 Research Summary

The paper tackles the long‑standing scalability and privacy challenges of Gaussian Process (GP) learning in large multi‑robot teams. Exact GP inference scales cubically with the total number of observations, which quickly becomes infeasible when dozens or hundreds of robots each collect data. Moreover, many distributed GP approaches require sharing raw measurements, violating privacy constraints and overloading limited communication links.

To overcome these issues, the authors propose pxpGP, a federated GP framework that works for both centralized and fully decentralized networks, and its decentralized variant dec‑pxpGP. The key idea is to replace raw data exchange with the exchange of compact pseudo‑representations that are generated locally via sparse variational inference. Each robot i builds a sparse GP on its own dataset D_i, optimizes a set of P inducing points X_P^i together with variational parameters (μ_P^i, A_P^i), and then extracts a pseudo‑dataset D_i^* (the inducing inputs, their variational means, and covariances). This step reduces the per‑robot computational cost from O(N_i³) to O(N_i P²) and the storage from O(N_i²) to O(P²).

Because unconstrained inducing points can drift outside the region covered by local data or collapse into clusters, the authors augment the standard Evidence Lower Bound (ELBO) with two regularizers:

1. Boundary penalty L_b – a quadratic ReLU term that penalizes any inducing point that lies beyond the min/max bounds of the robot’s local inputs.
2. Repulsive penalty L_r – another ReLU term that enforces a minimum Euclidean distance d_min between any pair of inducing points, encouraging a well‑spread configuration.

The combined objective (ELBO + L_b + L_r) yields a compact, well‑conditioned pseudo‑dataset that preserves privacy (no raw measurements are ever transmitted).

All robots then broadcast their pseudo‑datasets to a central node (centralized mode) or to their one‑hop neighbors (decentralized mode). The union of all pseudo‑datasets, D_c^, is merged with each robot’s original data to form a pseudo‑augmented set D_i^+ = D_i ∪ D_c^*. This set is used for the final GP hyper‑parameter estimation.

Hyper‑parameter consensus is enforced via a scaled proximal‑inexact ADMM (pxADMM). The augmented Lagrangian is linearized around a stationary point v_i = z + u_i, yielding simple closed‑form updates for the local hyper‑parameters θ_i, the global consensus variable z, and the scaled dual variables u_i. Two practical tricks dramatically improve convergence:

• Adaptive penalty updates – the ADMM penalty ρ_i is adjusted on‑the‑fly using residual balancing, preventing stagnation when the primal and dual residuals diverge.
• Warm‑start initialization – each robot’s variational hyper‑parameters (θ_i^*) from the sparse GP are used as the initial guess for the ADMM iterations, cutting the number of required rounds.

In the decentralized version, the same update rules are applied on each edge of the communication graph G = (V,E). Each robot i maintains auxiliary variables z_ij for every neighbor j, and exchanges only θ_i and z_ij with its neighbors, preserving the federated constraint of no raw data sharing.

Experimental evaluation spans synthetic 2‑D functions with non‑stationary behavior and real‑world sensor data (e.g., temperature, gas concentration) collected by up to 200 robots. Baselines include centralized cGP, apxGP, gapxGP, and decentralized counterparts (dec‑cGP, dec‑apxGP, dec‑gapxGP). The metrics are: (a) hyper‑parameter estimation error (relative to ground‑truth MLE), (b) prediction RMSE on held‑out test points, (c) number of ADMM communication rounds to reach a tolerance, and (d) total transmitted data volume.

Key findings:

• Accuracy – pxpGP and dec‑pxpGP achieve 30‑50 % lower hyper‑parameter error than the best baseline and match or improve RMSE, especially as the number of robots exceeds 40 where baseline methods deteriorate.
• Communication efficiency – thanks to adaptive ρ and warm‑starts, convergence is reached in 2–3 × fewer ADMM rounds. The pseudo‑datasets are typically only 5–10 % of the original data size, leading to >95 % reduction in transmitted bytes.
• Robustness – the boundary and repulsive penalties prevent numerical failures (e.g., Cholesky breakdown) that occur in baseline methods when inducing points drift or cluster.
• Scalability – while baseline methods start to fail or require excessive communication beyond ~40 agents, pxpGP scales smoothly up to 200 agents with modest per‑robot computation.

The authors acknowledge some limitations: the ELBO optimization still depends on a good K‑means initialization, and the penalty hyper‑parameters (λ_b, λ_r, d_min) must be tuned for each domain. Future work is suggested on fully asynchronous ADMM, dynamic network topologies, multi‑kernel extensions, and online updating of pseudo‑representations.

Overall, the paper delivers a well‑grounded, practically implementable federated GP learning framework that reconciles the competing demands of computational tractability, communication efficiency, and data privacy in large‑scale multi‑robot systems.