Graph-based Semi-Supervised Learning via Maximum Discrimination

Semi-supervised learning (SSL) addresses the critical challenge of training accurate models when labeled data is scarce but unlabeled data is abundant. Graph-based SSL (GSSL) has emerged as a popular framework that captures data structure through graph representations. Classic graph SSL methods, such as Label Propagation and Label Spreading, aim to compute low-dimensional representations where points with the same labels are close in representation space. Although often effective, these methods can be suboptimal on data with complex label distributions. In our work, we develop AUC-spec, a graph approach that computes a low-dimensional representation that maximizes class separation. We compute this representation by optimizing the Area Under the ROC Curve (AUC) as estimated via the labeled points. We provide a detailed analysis of our approach under a product-of-manifold model, and show that the required number of labeled points for AUC-spec is polynomial in the model parameters. Empirically, we show that AUC-spec balances class separation with graph smoothness. It demonstrates competitive results on synthetic and real-world datasets while maintaining computational efficiency comparable to the field’s classic and state-of-the-art methods.

💡 Research Summary

The paper addresses the problem of semi‑supervised learning (SSL) when labeled data are scarce but unlabeled data are abundant, focusing on graph‑based approaches. Classical graph‑based SSL methods such as Label Propagation (LP) and Laplacian eigenvector embeddings aim to enforce smoothness on the graph while fixing the values of labeled nodes. These methods work well when class boundaries align with well‑separated clusters, but they often fail on more complex label distributions. For example, in a “ring of Gaussians” dataset where clusters alternate labels around a circle, LP produces a nearly constant solution (spike phenomenon) and low‑dimensional eigenvectors capture global structure but not the discriminative direction needed for classification.

To overcome these limitations, the authors propose AUC‑spec, a novel graph‑based SSL algorithm that directly maximizes class separation by optimizing an Area Under the ROC Curve (AUC) objective on the labeled points while preserving graph smoothness. The objective function combines a Laplacian quadratic form (vᵀLv) with a negative AUC term weighted by a hyper‑parameter γ:
 J(v) = vᵀLv – γ·AUĈ(y_L, v_L).
AUĈ is approximated using a sigmoid surrogate σ(z)=1/(1+e⁻ᶻ) over all positive‑negative labeled pairs, making the objective differentiable.

Because J(v) is non‑convex, the authors adopt a simple iterative scheme reminiscent of power iteration but augmented with an AUC gradient step:
 v^{t+1} = v^{t} + γ (L_rw v^{t} + ∇AUC),
where L_rw is the random‑walk normalized Laplacian and ∇AUC is computed as σ_{ij}(1–σ_{ij}) for each pair (i∈P, j∈N). After each update the vector is normalized to avoid scale drift. This procedure yields a low‑dimensional representation v that simultaneously respects the graph topology and pushes positive labeled points above negatives in the ranking induced by v.

For multi‑class problems the method extends via a One‑vs‑Rest scheme: each class c is treated as a binary problem with its own positive set P_c and negative set N_c, a separate v^{(c)} is learned using the same iteration, and the final label for an unlabeled node i is arg max_c v^{(c)}_i. The computational cost per iteration is O(C·|E|) for sparse graphs plus O(C·|P||N|) for the AUC gradient, which remains modest when the labeling rate is low.

Theoretical analysis is conducted under a “product‑of‑manifold” generative model, where data lie on a Cartesian product of low‑dimensional manifolds, each associated with a class. The authors prove that, under this model, the number of labeled points required for AUC‑spec to achieve a non‑trivial predictor grows only polynomially with the model parameters (e.g., number of manifolds, intrinsic dimensions). Moreover, they show that the solution remains smooth with respect to the graph Laplacian, avoiding the spike phenomenon observed in LP for large unlabeled sets.

Empirical evaluation includes three major settings: (1) the synthetic ring‑of‑Gaussians dataset, where AUC‑spec correctly identifies the 7th eigenvector‑like direction that separates alternating labels, achieving >90 % accuracy versus <60 % for LP; (2) a 2‑D Gaussian mixture with overlapping components, where AUC‑spec outperforms LP and label spreading in both accuracy and robustness to label imbalance; (3) several real‑world benchmarks (e.g., MNIST subsets, text classification, image classification) where AUC‑spec matches or slightly exceeds state‑of‑the‑art graph‑based SSL methods while maintaining comparable or faster runtimes. Notably, even with only 1 % of the data labeled, AUC‑spec produces stable predictions without the degradation seen in LP.

In summary, AUC‑spec introduces a principled way to incorporate discriminative ranking (AUC) into graph‑based SSL, thereby relaxing the overly restrictive constraint of fixing labeled node values. The method provides both theoretical guarantees on label complexity and practical advantages in terms of accuracy and efficiency, representing a significant step forward for semi‑supervised learning on complex, graph‑structured data.