Graph Contrastive Learning via Spectral Graph Alignment

Given augmented views of each input graph, contrastive learning methods (e.g., InfoNCE) optimize pairwise alignment of graph embeddings across views while providing no mechanism to control the global structure of the view specific graph-of-graphs built from these embeddings. We introduce SpecMatch-CL, a novel loss function that aligns the view specific graph-of-graphs by minimizing the difference between their normalized Laplacians. Theoretically, we show that under certain assumptions, the difference between normalized Laplacians provides an upper bound not only for the difference between the ideal Perfect Alignment contrastive loss and the current loss, but also for the Uniformly loss. Empirically, SpecMatch-CL establishes new state of the art on eight TU benchmarks under unsupervised learning and semi-supervised learning at low label rates, and yields consistent gains in transfer learning on PPI-306K and ZINC 2M datasets.

💡 Research Summary

Graph contrastive learning (GCL) has become a dominant paradigm for learning graph‑level representations without labels. Existing methods such as GraphCL, JOAO, and their variants rely on the InfoNCE loss to pull together embeddings of two augmented views of the same graph (Alignment) while implicitly encouraging a uniform distribution of all embeddings (Uniformity). However, InfoNCE operates only at the instance level: it does not constrain the global geometry of the similarity graph that each view induces over the whole dataset. Consequently, two views can achieve identical pairwise contrastive scores yet generate very different neighborhood structures, which may hurt downstream generalization and transferability.

SpecMatch‑CL addresses this gap by introducing a spectral regularizer that aligns the view‑specific graph‑of‑graphs. For each view v∈{1,2}, the method builds a similarity matrix S^{(v)} where S^{(v)}_{ij}=⟨z^{(v)}_i, z^{(v)}_j⟩ using the normalized embeddings z^{(v)}_i. An adjacency matrix A^{(v)} is obtained by thresholding S^{(v)}; the threshold can be a fixed value or an adaptive percentile of the similarity distribution. From A^{(v)} the degree matrix D^{(v)} is computed and the normalized Laplacian L^{(v)} = I – D^{(v)−1/2} A^{(v)} D^{(v)−1/2} is formed. The spectral loss is the Frobenius norm of the Laplacian difference:

 L_G = ‖L^{(1)} – L^{(2)}‖_F².

The final training objective combines the standard InfoNCE loss L_C with the spectral term:

 L = L_C + β·L_G,

where β balances alignment and spectral consistency.

Theoretical contributions.
Two theorems connect L_G to the classic Alignment and Uniformity objectives introduced by Wang & Isola. Under Assumption 4.1 (the embedding discrepancy is bounded by a constant‑scaled diffusion distance between the two view‑wise heat kernels), Theorem 4.2 shows that the absolute gap between the realized contrastive loss and its “Perfect Alignment” counterpart is bounded by (t_d)²·c·τ·L_G. Here t_d is a diffusion time, τ the InfoNCE temperature, and c a data‑dependent constant. This establishes that reducing the spectral loss directly tightens the contrastive loss toward its ideal value.

Theorem 4.3 addresses Uniformity. Assuming each view‑specific similarity graph is connected and that the two Laplacians are i.i.d. given the data, the theorem derives an upper bound on the uniformity loss L_unif that grows linearly with the expected spectral loss E