HyReaL: Clustering Attributed Graph via Hyper-Complex Space Representation Learning

Clustering complex data in the form of attributed graphs has attracted increasing attention, where powerful graph representation is a critical prerequisite. However, the well-known Over-Smoothing (OS) effect makes Graph Convolutional Networks tend to homogenize the representation of graph nodes, while the existing OS solutions focus on alleviating the homogeneity of nodes’ embeddings from the aspect of graph topology information, which is inconsistent with the attributed graph clustering objective. Therefore, we introduce hyper-complex space with powerful quaternion feature transformation to enhance the representation learning of the attributes. A generalized \textbf{Hy}per-complex space \textbf{Re}present\textbf{a}tion \textbf{L}earning (\textbf{HyReaL}) model is designed to: 1) bridge arbitrary dimensional attributes to the well-developed quaternion algebra with four parts, and 2) connect the learned representations to more generalized clustering objective without being restricted to a given number of clusters $k$. The novel introduction of quaternion benefits attributed graph clustering from two aspects: 1) enhanced attribute coupling learning capability allows complex attribute information to be sufficiently exploited in clustering, and 2) stronger learning capability makes it unnecessary to stack too many graph convolution layers, naturally alleviating the OS problem. It turns out that the node representations learned by HyReaL are more discriminative and widely suit downstream clustering with different $k$s. Extensive experiments including significance tests, ablation studies, qualitative results, etc., show the superiority of HyReaL.

💡 Research Summary

The paper addresses two intertwined challenges in attributed graph clustering: Over‑Smoothing (OS), which causes node embeddings to become indistinguishable when many GCN layers are stacked, and Over‑Dominating (OD), where the topology‑driven learning overshadows attribute information that is crucial for clustering. Existing OS mitigation techniques (residual connections, dynamic hopping, contrastive learning, etc.) focus on preserving topological distinctions for supervised tasks and therefore do not resolve OD in an unsupervised clustering setting.

To simultaneously alleviate OS and OD, the authors propose HyReaL, a hyper‑complex (quaternion) based representation learning framework. First, arbitrary‑dimensional node attributes X∈ℝ^{n×d} are projected into four separate views using four independent MLPs, producing F_r, F_x, F_y, and F_z. These four views are combined into a quaternion feature F = F_r + F_x i + F_y j + F_z k, effectively lifting the data into a four‑axis hyper‑complex space ℍ. A learnable quaternion weight matrix W_Q∈ℍ^{4d×4\hat d} then interacts with F via the Hamilton product, enabling rich feature coupling with only a modest number of parameters. The authors argue that quaternion transformations provide a four‑fold higher degree of freedom (DoF) compared with real‑valued linear maps, allowing stronger learning capacity without deepening the network.

The quaternion‑encoded features are fed into a Quaternion Graph Encoder (QGE). Each encoder layer performs a graph convolution in the hyper‑complex domain: H^{(l)} = φ^{(l)}( \tilde A·H^{(l‑1)} ⊗ W_Q^{(l)} ), where ⊗ denotes the Hamilton product and φ is a non‑linear activation. Stacking m such layers aggregates multi‑hop neighborhood information while preserving the high‑DoF quaternion transformations. The final embedding Γ is obtained by averaging the real and three imaginary parts of the last quaternion representation, yielding a real‑valued matrix suitable for downstream tasks.

Training is driven by a joint loss: (1) a graph reconstruction loss L_kl based on the Kullback‑Leibler divergence between the original adjacency matrix A and its reconstruction \hat A = ΓΓᵀ, ensuring that structural information is retained; (2) a clustering‑oriented loss L_sc that does not require a pre‑specified number of clusters k, thereby producing embeddings that are “k‑free” and can be clustered with any desired granularity after training. The total loss L = L_kl + λ·L_sc balances structure preservation and clustering friendliness.

Extensive experiments on seven benchmark datasets (Cora, Citeseer, Pubmed, DBLP, ACM, etc.) demonstrate that HyReaL consistently outperforms state‑of‑the‑art methods such as GAE, VGAE, DAEGC, ARGA, and SDCN. Metrics (NMI, ARI, ACC) improve by 4–6 % on average. Ablation studies confirm that both the Four‑View Projection (FVP) and Quaternion Graph Encoders (QGE) are essential: removing either component degrades performance dramatically. Significance tests (p < 0.01) validate the statistical robustness of the gains. Visualizations (t‑SNE) show clearer, more separated clusters compared with baselines.

The paper’s contributions are fourfold: (i) introducing quaternion algebra to unsupervised graph clustering, bridging arbitrary‑dimensional attributes to a four‑part hyper‑complex representation; (ii) formally defining the Over‑Dominating effect and demonstrating its impact on clustering performance; (iii) leveraging the high DoF of quaternion transformations to mitigate both OS and OD without deep networks; (iv) providing a k‑free clustering loss that yields reusable embeddings for varying cluster numbers, reducing the need for retraining.

Limitations include modest additional computational overhead due to quaternion operations and the need for careful design of the MLPs in FVP for very high‑dimensional attributes. The current formulation assumes undirected graphs; extending to directed or heterogeneous graphs is left for future work.

In summary, HyReaL offers a novel hyper‑complex representation learning paradigm that balances attribute and structural information, alleviates over‑smoothing, and produces versatile embeddings for attributed graph clustering, opening new avenues for applications in social network analysis, bioinformatics, recommendation systems, and beyond.