Consistent Supervised-Unsupervised Alignment for Generalized Category Discovery

Generalized Category Discovery (GCD) focuses on classifying known categories while simultaneously discovering novel categories from unlabeled data. However, previous GCD methods face challenges due to inconsistent optimization objectives and category confusion. This leads to feature overlap and ultimately hinders performance on novel categories. To address these issues, we propose the Neural Collapse-inspired Generalized Category Discovery (NC-GCD) framework. By pre-assigning and fixing Equiangular Tight Frame (ETF) prototypes, our method ensures an optimal geometric structure and a consistent optimization objective for both known and novel categories. We introduce a Consistent ETF Alignment Loss that unifies supervised and unsupervised ETF alignment and enhances category separability. Additionally, a Semantic Consistency Matcher (SCM) is designed to maintain stable and consistent label assignments across clustering iterations. Our method achieves strong performance on multiple GCD benchmarks, significantly enhancing novel category accuracy and demonstrating its effectiveness.

💡 Research Summary

Generalized Category Discovery (GCD) tackles the open‑world scenario where a model must simultaneously classify samples from a set of known categories (with limited labeled data) and discover entirely unlabeled novel categories. Existing GCD approaches typically fall into two families: contrastive‑learning methods that try to separate known from unknown samples, and clustering‑or‑distribution‑estimation methods that infer the structure of the unknown classes. Both families rely on dynamically learned class prototypes (e.g., classifier weights, cluster centroids) which leads to two fundamental problems. First, the supervised loss (for known classes) and the unsupervised loss (for novel classes) optimize different objectives, causing the model to over‑fit the known categories and to neglect proper decision boundaries for the novel ones. Second, clustering is inherently unstable; pseudo‑labels can change dramatically between iterations, resulting in category confusion and degraded feature alignment.

The authors propose to resolve these issues by leveraging the Neural Collapse (NC) phenomenon. NC describes a geometric configuration that emerges in well‑trained deep classifiers: the last‑layer features of each class collapse onto their class mean, and the class means (or classifier weights) form a Simplex Equiangular Tight Frame (ETF). In an ETF, K class vectors in a d‑dimensional space are unit‑norm and have equal pairwise cosine similarity, which maximizes inter‑class separation while minimizing intra‑class variance. This configuration is provably optimal for classification under balanced settings.

Building on this theory, the paper introduces the Neural Collapse‑inspired GCD framework (NC‑GCD). The key idea is to pre‑assign a fixed set of ETF prototypes for all categories (both known and novel) before training begins. By fixing these prototypes, the model receives a single, consistent geometric target throughout training, eliminating the mismatch between supervised and unsupervised objectives.

Three main components constitute NC‑GCD:

1. Pre‑assigned ETF prototypes – Given an estimate of the total number of categories K (known + novel), the authors construct K unit‑norm vectors that satisfy the ETF inner‑product constraints. These vectors remain unchanged during training and serve as the anchors for feature alignment.

2. Consistent ETF Alignment Loss – Two alignment terms are defined:

   • Unsupervised ETF alignment (L_uETF): Every T epochs the model performs a periodic clustering on all embeddings. For each cluster, the top α % most confident samples (those with highest cosine similarity to the cluster centroid) are pulled toward the corresponding ETF prototype using a dot‑regression (L2) loss.
   • Supervised ETF alignment (L_sETF): Labeled samples are directly aligned with the ETF prototype that corresponds to their ground‑truth class (after a mapping step described below). The two terms are combined as L_ETF = (1 − γ) L_uETF + γ L_sETF, where γ balances the contribution of the supervised and unsupervised parts. This unified loss ensures that both known and novel categories are driven toward the same optimal geometric arrangement.

3. Semantic Consistency Matcher (SCM) – Clustering can produce inconsistent pseudo‑labels across iterations, and the direct mapping from true labels to ETF prototypes may be mismatched. SCM addresses both issues by enforcing a one‑to‑one correspondence between clusters of consecutive iterations. It solves an optimal assignment problem (e.g., Hungarian algorithm) to align current cluster IDs with previous ones, thereby stabilizing pseudo‑labels. In the supervised branch, SCM also maps true labels to ETF prototypes in a way that respects the current cluster‑to‑prototype alignment, preventing mismatches that would otherwise destabilize training.

The overall training pipeline proceeds as follows: a pretrained visual encoder extracts embeddings for each image; every T epochs a clustering step groups embeddings; high‑confidence samples are aligned unsupervisedly, labeled samples are aligned supervisedly, and SCM guarantees label consistency across clustering rounds. The model is trained end‑to‑end with the combined loss.

Experimental evaluation is conducted on six public GCD benchmarks, including CIFAR‑100, ImageNet‑LT, and Stanford‑Cars, each split into known and novel subsets with only a small fraction of known samples labeled. Baselines comprise recent state‑of‑the‑art methods such as DCCL, PromptCAL, SimGCD, ProtoGCD, UNO, and ORCA. Two metrics are reported: overall accuracy (Acc_All) and novel‑category accuracy (Acc_N). NC‑GCD consistently outperforms all baselines, achieving notable gains in Acc_N (average improvement of ~4.8 percentage points, with some datasets exceeding 7 pp). Ablation studies reveal that (i) fixing the ETF alone already yields substantial benefits, and (ii) removing SCM leads to severe performance drops due to pseudo‑label instability. Sensitivity analysis shows that setting γ≈0.5 and α≈30 % works well across datasets.

Significance and limitations: This work is the first to bring Neural Collapse theory into the GCD setting, demonstrating that a pre‑defined optimal geometric structure can unify supervised and unsupervised learning objectives. By fixing ETF prototypes, the method eliminates the drifting decision boundaries that plague dynamic prototype approaches. SCM further stabilizes the learning dynamics by mitigating clustering noise. However, the approach assumes that the total number of categories K can be estimated beforehand; in truly open‑world environments K may be unknown or may change over time. Moreover, the dimensionality of the ETF (the choice of d) is fixed, and scaling to extremely high‑dimensional embeddings or to modalities beyond images (e.g., text, multimodal data) remains an open question.

Future directions suggested by the authors include: (a) developing dynamic K estimation techniques (e.g., Bayesian non‑parametrics) to integrate with the fixed‑ETF framework; (b) extending the method to other data modalities and to multimodal GCD; (c) exploring alternative frame structures (non‑equiangular or data‑adaptive frames) that might better match real‑world data distributions; and (d) incorporating online or streaming clustering mechanisms together with SCM for continual GCD scenarios.

In summary, NC‑GCD introduces a principled, geometry‑driven solution to the core challenges of Generalized Category Discovery. By anchoring all categories to a pre‑computed Simplex ETF and enforcing semantic consistency across clustering iterations, the method achieves superior discovery of novel categories while maintaining strong performance on known classes, setting a new benchmark for open‑world classification tasks.