Domain-Invariant Representation Learning of Bird Sounds

Passive acoustic monitoring (PAM) is crucial for bioacoustic research, enabling non-invasive species tracking and biodiversity monitoring. Citizen science platforms provide large annotated datasets from focal recordings, where the target species is intentionally recorded. However, PAM requires monitoring in passive soundscapes, creating a domain shift between focal and passive recordings, challenging deep learning models trained on focal recordings. To address domain generalization, we leverage supervised contrastive learning by enforcing domain invariance across same-class examples from different domains. Additionally, we propose ProtoCLR, an alternative to SupCon loss which reduces the computational complexity by comparing examples to class prototypes instead of pairwise comparisons. We conduct few-shot classification based on BIRB, a large-scale bird sound benchmark to assess pre-trained bioacoustic models. Our findings suggest that ProtoCLR is a better alternative to SupCon.

💡 Research Summary

Passive acoustic monitoring (PAM) is a cornerstone technology for non‑invasive wildlife observation, allowing researchers to collect long‑term data on animal behavior, migration, and population trends. Large citizen‑science platforms such as Xeno‑Canto (XC) have amassed millions of annotated bird vocalizations, providing a rich source of labeled data for training deep learning models. However, recordings from XC are “focal”: the recorder is deliberately aimed at the target species, resulting in clean, high‑signal‑to‑noise samples. In contrast, PAM devices are deployed in natural environments and capture “soundscape” recordings that contain a mixture of multiple species, background environmental noise, and varying recording conditions. This discrepancy creates a domain shift that severely hampers the ability of models trained solely on focal data to generalize to real‑world soundscape recordings.

The paper addresses two intertwined challenges: (1) learning domain‑invariant representations that remain robust when the training data (focal) and test data (soundscape) come from different distributions, and (2) reducing the computational burden of the widely used supervised contrastive loss (SupCon), which scales quadratically with batch size (O(N²)). To this end, the authors propose ProtoCLR, a prototypical contrastive loss that replaces pairwise sample‑wise comparisons with comparisons against class‑level prototypes (centroids). ProtoCLR retains the essential objective of SupCon—pulling same‑class examples together and pushing different‑class examples apart—while achieving linear complexity O(N·C), where C is the number of classes in the batch (typically far smaller than N).

Methodological Details
SupCon defines, for each anchor sample i, a set of positive indices P(i) (other samples sharing the same label) and a set of all other indices A(i). The loss aggregates log‑softmax terms over all positives, normalizing by the sum over all negatives. Its gradient consists of a positive term that pulls the anchor toward each positive example and a negative term that pushes it away from every other sample. This formulation requires computing dot products for every pair of samples in the batch, leading to O(N²) operations.

ProtoCLR first computes a prototype c_y for each class y present in the batch: c_y = (1/|C(y)|) Σ_{i∈C(y)} z_i, where z_i is the L2‑normalized embedding of sample i. The loss for an anchor i becomes a log‑softmax over the similarity between its embedding and all class prototypes, with the positive term involving its own class prototype c_{y_i}. The gradient mirrors SupCon’s positive term but replaces the negative term with a weighted average over prototypes rather than individual embeddings. This change reduces variance because prototypes aggregate information from many samples, leading to more stable gradients and faster convergence.

The authors analytically show that, near convergence, both SupCon and ProtoCLR encourage embeddings to collapse onto their class centroids, establishing a theoretical equivalence despite the different computational pathways.

Experimental Setup
The evaluation uses the BIRB benchmark, which provides a large focal training set (XC: 684,744 recordings, 10,127 species) and six soundscape test sets (PER, NES, UHH, HSN, SSW, SNE) covering 132 species. The authors adopt a few‑shot classification protocol: for each class, k examples (k = 1 or 5) are randomly sampled to form a class prototype; the remaining examples serve as queries. SimpleShot is employed for inference: both query embeddings and class prototypes are mean‑subtracted, L2‑normalized, and the nearest prototype determines the predicted label. Each experiment is repeated ten times with different random seeds, and mean accuracy plus standard deviation are reported.

All models share the same backbone: a 2‑D ConvVision Transformer (CvT‑13) with ~20 M parameters, trained for 300 epochs on XC with a batch size of 256. Data augmentations include circular time shift, SpecAugment, and spectrogram mixing (the latter excluded for CE and ProtoCLR because it hindered convergence). Learning rates are 5 × 10⁻⁴ for CE and ProtoCLR, and 1 × 10⁻⁴ for SupCon and SimCLR. Hyper‑parameters are tuned by monitoring k‑NN accuracy on a held‑out portion of the POW validation set.

Baseline and comparison methods include:

• Cross‑entropy (CE) supervised training.
• SimCLR (self‑supervised contrastive learning).
• SupCon (supervised contrastive learning).
• ProtoCLR (the proposed loss).
• State‑of‑the‑art large‑scale bioacoustic models: BirdAVES‑biox‑base (12‑layer transformer, 768 units), BirdAVES‑bioxn‑large (24‑layer, 1024 units), BioLingual (audio‑text contrastive pre‑training), and Perch (EfficientNet trained on XC with taxonomic multitask heads).

Results
One‑Shot Classification: ProtoCLR achieves an average top‑1 accuracy of 21.4 % across all soundscape test sets, surpassing SupCon (20.5 %) and CE (9.55 %). The gain is most pronounced on the PER and NES datasets, which contain highly variable background noise. SimCLR trails at 15.4 %, confirming that label‑free contrastive learning struggles without carefully crafted augmentations.

Five‑Shot Classification: ProtoCLR’s average accuracy rises to 42.4 %, again outpacing SupCon (39.5 %) and CE (21.4 %). The gap widens on the more challenging SNE and SSW sets, indicating that the prototypical approach better leverages the additional examples per class. Large‑scale models such as BirdAVES‑bioxn‑large and Perch achieve higher absolute numbers (≈48‑60 %) but are trained on or fine‑tuned with soundscape data, whereas ProtoCLR is trained exclusively on focal recordings and still attains competitive performance.

Computational Efficiency: A single epoch on XC (batch size 256) requires 80.4 billion multiply‑accumulate operations for SupCon versus 28.3 billion for ProtoCLR, confirming the theoretical O(N·C) advantage. Memory consumption follows the same trend, enabling training on modest GPU resources.

Discussion and Implications
ProtoCLR demonstrates that incorporating class prototypes into supervised contrastive learning yields a dual benefit: (i) substantial reduction in computational cost, making contrastive pre‑training feasible for large bioacoustic datasets, and (ii) improved domain generalization when the target domain lacks labeled data. The method is straightforward to implement—only a per‑batch prototype computation and a modified loss term are required—yet it delivers measurable gains across diverse soundscape environments.

The authors acknowledge potential limitations: prototype quality depends on class balance within each batch, so highly imbalanced batches could produce biased centroids. Moreover, the study focuses on few‑shot classification; extending ProtoCLR to continuous detection, segmentation, or multi‑label scenarios remains future work. Nonetheless, the approach offers a practical pathway for leveraging abundant citizen‑science focal recordings to build robust models for real‑world passive monitoring.

Conclusion
The paper introduces ProtoCLR, a prototypical supervised contrastive loss that replaces pairwise sample comparisons with class‑level prototype comparisons. This redesign reduces training complexity from quadratic to linear while preserving the core objective of supervised contrastive learning. Empirical evaluation on the BIRB benchmark shows that ProtoCLR consistently outperforms SupCon, CE, and SimCLR in both one‑shot and five‑shot bird‑sound classification across multiple soundscape datasets, and does so with roughly one‑third of the computational overhead. The work provides a scalable, effective solution for domain‑invariant representation learning in bioacoustics, bridging the gap between richly labeled focal recordings and the noisy, unlabeled soundscapes encountered in passive acoustic monitoring.