Edge Collaborative Gaussian Splatting with Integrated Rendering and Communication

Gaussian splatting (GS) struggles with degraded rendering quality on low-cost devices. To address this issue, we present edge collaborative GS (ECO-GS), where each user can switch between a local small GS model to guarantee timeliness and a remote large GS model to guarantee fidelity. However, deciding how to engage the large GS model is nontrivial, due to the interdependency between rendering requirements and resource conditions. To this end, we propose integrated rendering and communication (IRAC), which jointly optimizes collaboration status (i.e., deciding whether to engage large GS) and edge power allocation (i.e., enabling remote rendering) under communication constraints across different users by minimizing a newly-derived GS switching function. Despite the nonconvexity of the problem, we propose an efficient penalty majorization minimization (PMM) algorithm to obtain the critical point solution. Furthermore, we develop an imitation learning optimization (ILO) algorithm, which reduces the computational time by over 100x compared to PMM. Experiments demonstrate the superiority of PMM and the real-time execution capability of ILO.

💡 Research Summary

This paper tackles the well‑known degradation of Gaussian splatting (GS) rendering quality on low‑cost mobile devices. While GS offers state‑of‑the‑art 3D reconstruction, its large model size and high computational demand force practitioners to compress or prune the model for deployment on resource‑constrained hardware, which inevitably harms visual fidelity (lower PSNR, SSIM, and higher L1 loss). To mitigate this, the authors propose an Edge Collaborative GS (ECO‑GS) framework in which each user can dynamically switch between a lightweight local GS model (ensuring low latency) and a heavyweight remote GS model hosted on a proximal edge server (ensuring high fidelity).

The central technical challenge is deciding, for each user, whether to engage the remote model. This decision depends on two intertwined factors: (i) the rendering requirement, i.e., how much the local model’s output deviates from the ground‑truth image for the current pose, and (ii) the communication and power resources available to download the remote image within a strict end‑to‑end latency budget. Existing edge‑collaboration schemes either maximize sum‑rate or adopt simple heuristics, ignoring the non‑uniform discrepancy across poses and the coupling between rendering and communication resources.

To address this, the authors formulate an Integrated Rendering and Communication (IRAC) optimization problem. They first derive a novel “GS switching function” that quantifies the expected quality gain of remote rendering without requiring ground‑truth images. By applying the triangle inequality and assuming a small weighting λ for the SSIM term, they upper‑bound the original rendering loss L(v_k, b v_k) with an expression that depends only on the L1 distance between the local and remote GS models for the same pose. Consequently, the objective becomes a sum over users of (1 − x_k)·L(Φ_edge(s_k), Φ_k(s_k)) plus a constant, where x_k∈{0,1} indicates remote participation.

The IRAC problem includes realistic constraints: (a) each user’s downlink rate must support downloading the remote image within the remaining latency budget, (b) the edge server’s total transmit power cannot exceed P, (c) at most S users may be served remotely simultaneously, and (d) binary collaboration variables. This yields a mixed‑integer non‑convex program.

The authors relax the binary variables to the continuous interval