Enhancing Imbalanced Node Classification via Curriculum-Guided Feature Learning and Three-Stage Attention Network

Imbalanced node classification in graph neural networks (GNNs) happens when some labels are much more common than others, which causes the model to learn unfairly and perform badly on the less common classes. To solve this problem, we propose a Curriculum-Guided Feature Learning and Three-Stage Attention Network (CL3AN-GNN), a learning network that uses a three-step attention system (Engage, Enact, Embed) similar to how humans learn. The model begins by engaging with structurally simpler features, defined as (1) local neighbourhood patterns (1-hop), (2) low-degree node attributes, and (3) class-separable node pairs identified via initial graph convolutional networks and graph attention networks (GCN and GAT) embeddings. This foundation enables stable early learning despite label skew. The Enact stage then addresses complicated aspects: (1) connections that require multiple steps, (2) edges that connect different types of nodes, and (3) nodes at the edges of minority classes by using adjustable attention weights. Finally, Embed consolidates these features via iterative message passing and curriculum-aligned loss weighting. We evaluate CL3AN-GNN on eight Open Graph Benchmark datasets spanning social, biological, and citation networks. Experiments show consistent improvements across all datasets in accuracy, F1-score, and AUC over recent state-of-the-art methods. The model’s step-by-step method works well with different types of graph datasets, showing quicker results than training everything at once, better performance on new, imbalanced graphs, and clear explanations of each step using gradient stability and attention correlation learning curves. This work provides both a theoretically grounded framework for curriculum learning in GNNs and practical evidence of its effectiveness against imbalances, validated through metrics, convergence speeds, and generalisation tests.

💡 Research Summary

The paper tackles the pervasive problem of label imbalance in graph neural networks (GNNs) for node classification. When some classes dominate the label distribution, standard GNNs tend to over‑fit the majority and neglect minority nodes, leading to poor overall performance. Existing remedies—oversampling, class‑weight rebalancing, hybrid ensembles, or graph‑specific techniques such as GraphSMOTE—either introduce noise, lose structural information, or fail to exploit the hierarchical nature of GNNs. Moreover, prior curriculum learning (CL) approaches for graphs have not been integrated with the model architecture and often rely on simplistic difficulty metrics (e.g., node degree) that do not reflect true learning complexity.

To address these gaps, the authors propose Curriculum‑Guided Feature Learning and Three‑Stage Attention Network (CL3AN‑GNN). The core idea is to mimic human learning by structuring the training process into three progressive stages—Engage, Enact, Embed—each equipped with its own attention mechanism and loss weighting.

1. Engage focuses on “easy” graph signals: (i) 1‑hop neighbourhood patterns, (ii) low‑degree node attributes, and (iii) class‑separable node pairs identified from preliminary GCN and GAT embeddings. By training on these simple cues first, the model builds a stable foundation and avoids early loss explosion despite severe label skew.

2. Enact introduces higher‑order complexity. It expands the receptive field to multi‑hop connections, attends to heterogeneous edge types, and specifically targets minority‑class boundary nodes. Adaptive attention weights (α for nodes, β for edges) are learned jointly with a curriculum‑aware class‑weight function w_c(y, t), ensuring that as training progresses, harder samples receive proportionally larger gradients.

3. Embed consolidates the representations. Multi‑head attention and iterative message passing fuse the refined node and edge embeddings into deep, discriminative vectors. A time‑dependent curriculum loss L_curr(t) combines cross‑entropy and KL‑divergence terms, modulated by λ_c(t) and λ_e(t) that gradually shift emphasis from easy to hard examples.

The architecture is modular: each stage outputs a representation that directly feeds the next stage, allowing independent analysis and clear dependency mapping. The authors also define formal difficulty functions D_v (node) and D_{u,v} (edge) and a threshold θ_t to construct progressive sub‑graphs G_t, thereby grounding the curriculum in graph topology rather than heuristic degree alone.

Experimental evaluation spans eight Open Graph Benchmark (OGB) datasets covering social, citation, and biological networks, with imbalance ratios ranging from 1:10 to 1:100. CL3AN‑GNN is benchmarked against state‑of‑the‑art baselines such as GraphSMOTE, GraphENS, ReWeight‑GNN, and standard GCN/GAT models. Results show consistent improvements: average gains of +3.2% in accuracy, +4.5% in macro‑F1, and +2.8% in AUC. Notably, minority‑class F1 scores increase by about 7%, confirming the method’s effectiveness in addressing imbalance. Training converges faster—approximately 15% fewer epochs—thanks to the early “Engage” phase that stabilizes gradients. The model also generalizes well to unseen imbalanced graphs (e.g., a 1:50 label ratio) without retraining, outperforming baselines by +2.3% accuracy.

Interpretability analyses plot the evolution of attention weights and curriculum loss coefficients over time, revealing a smooth transition from easy to hard focus that aligns with the designed curriculum schedule. Ablation studies demonstrate that removing any stage degrades performance, underscoring the necessity of the three‑stage progression.

Theoretical contributions include a formal curriculum framework for graphs, a phase‑gated loss modulation scheme, and a proof‑of‑concept that curriculum‑guided attention can mitigate gradient starvation for minority classes. Limitations are acknowledged: the difficulty functions rely on initial GCN/GAT embeddings, so poor initializations could affect curriculum quality; dynamic graphs and meta‑learning scenarios remain unexplored.

In conclusion, CL3AN‑GNN offers a principled, curriculum‑driven architecture that harmonizes progressive feature learning with adaptive attention, delivering superior performance on imbalanced node classification tasks while improving training efficiency, generalization, and interpretability. The work opens avenues for extending curriculum learning to dynamic graphs, meta‑learning, and other graph‑structured domains.