Decoupling and Damping: Structurally-Regularized Gradient Matching for Multimodal Graph Condensation

In multimodal graph learning, graph structures that integrate information from multiple sources, such as vision and text, can more comprehensively model complex entity relationships. However, the continuous growth of their data scale poses a significant computational bottleneck for training. Graph condensation methods provide a feasible path forward by synthesizing compact and representative datasets. Nevertheless, existing condensation approaches generally suffer from performance limitations in multimodal scenarios, mainly due to two reasons: (1) semantic misalignment between different modalities leads to gradient conflicts; (2) the message-passing mechanism of graph neural networks further structurally amplifies such gradient noise. Based on this, we propose Structural Regularized Gradient Matching (SR-GM), a condensation framework for multimodal graphs. This method alleviates gradient conflicts between modalities through a gradient decoupling mechanism and introduces a structural damping regularizer to suppress the propagation of gradient noise in the topology, thereby transforming the graph structure from a noise amplifier into a training stabilizer. Extensive experiments on four multimodal graph datasets demonstrate the effectiveness of SR-GM, highlighting its state-of-the-art performance and cross-architecture generalization capabilities in multimodal graph dataset condensation.

💡 Research Summary

The paper tackles the pressing problem of scaling multimodal graph learning, where nodes carry heterogeneous features such as text and images. While Graph Neural Networks (GNNs) achieve state‑of‑the‑art performance on large graphs, the sheer size of multimodal datasets makes training prohibitively expensive. Graph condensation—synthesizing a tiny, information‑dense surrogate graph—offers a promising remedy, but existing condensation methods falter on multimodal data. The authors identify two root causes: (1) semantic misalignment between modalities creates conflicting gradients, and (2) the message‑passing mechanism of GNNs amplifies these conflicts across the graph, turning the topology into a noise amplifier.

To address these challenges, the authors propose Structural Regularized Gradient Matching (SR‑GM), a two‑component framework that modifies the classic gradient‑matching condensation pipeline. First, gradient decoupling is introduced. For each synthetic node, the text‑gradient and image‑gradient are orthogonally projected against each other, effectively removing the component of one that lies in the direction of the other. This operation yields two mutually orthogonal gradient vectors, eliminating intra‑node modality conflict while preserving the contribution of each modality to the overall parameter update.

Second, structural damping is added as a regularizer on the gradient field itself. By incorporating a term proportional to the squared Frobenius norm of the graph Laplacian applied to the node‑gradient matrix (½ γ ‖L_S R‖_F²), the method forces neighboring nodes to have similar gradient directions. The authors prove (Theorem 2.1) that the amplification factor of modal‑mixing noise is bounded by the Dirichlet energy of the gradient field, showing that the Laplacian regularizer directly curtails the spectral amplification inherent in the graph structure.

The overall optimization proceeds in a bi‑level fashion: an inner loop trains a GNN on the current synthetic graph for a few steps, yielding parameters θ_t; an outer loop updates the synthetic node features X_S and adjacency A_S by minimizing a loss composed of (i) the distance between real‑graph and synthetic‑graph parameter gradients (the classic gradient‑matching term) and (ii) the structural damping term. The gradient decoupling step is applied before computing the outer‑loop loss, ensuring that the matching objective sees conflict‑free modality gradients.

Extensive experiments on four multimodal benchmark datasets (Flickr‑30K, MM‑Cite, OGB‑MolTex, Visual‑Text‑Social) and six GNN architectures (GCN, GraphSAGE, GAT, GIN, APPNP, JK‑Net) demonstrate the efficacy of SR‑GM. At compression ratios as low as 1 % of the original graph size, SR‑GM consistently outperforms prior condensation methods (PCG, GCond, GraphMatch, etc.) by an average of 4.2 percentage points and up to 9.7 points in node classification accuracy. Moreover, the Concentration Ratio (CR) metric—measuring intra‑modal feature collapse—drops dramatically, confirming that the multimodal richness is preserved. Cross‑architecture tests reveal that a single condensed graph generated by SR‑GM transfers well across all tested GNNs, with performance gaps under 1 %.

Ablation studies isolate the contributions of each component. Removing gradient decoupling while keeping structural damping leads to residual modality conflict and modest performance loss; removing structural damping while keeping decoupling leaves the graph’s noise‑amplifying nature unchecked, again hurting accuracy and raising CR. Omitting both reverts the method to vanilla gradient matching, which performs the worst.

The authors acknowledge a limitation: computing the full Laplacian L_S scales quadratically with the number of synthetic nodes, which could become a bottleneck for larger condensates. They suggest future work on spectral approximations (e.g., Lanczos, Chebyshev) or stochastic Laplacian estimators, as well as exploring non‑linear damping functions and adaptive γ schedules.

In summary, SR‑GM offers a theoretically grounded and empirically validated solution to multimodal graph condensation. By decoupling conflicting modality gradients and damping the graph‑induced propagation of gradient noise, it transforms the graph from a source of instability into a stabilizing scaffold, enabling compact synthetic graphs that retain the expressive power of their massive multimodal originals. This work paves the way for more scalable, data‑centric multimodal graph learning in real‑world applications.