An Augmentation Overlap Theory of Contrastive Learning

📝 Original Info

• Title: An Augmentation Overlap Theory of Contrastive Learning
• ArXiv ID: 2511.03114
• Date: 2025-11-05
• Authors: ** 논문에 명시된 저자 정보가 제공되지 않았습니다. **

📝 Abstract

Recently, self-supervised contrastive learning has achieved great success on various tasks. However, its underlying working mechanism is yet unclear. In this paper, we first provide the tightest bounds based on the widely adopted assumption of conditional independence. Further, we relax the conditional independence assumption to a more practical assumption of augmentation overlap and derive the asymptotically closed bounds for the downstream performance. Our proposed augmentation overlap theory hinges on the insight that the support of different intra-class samples will become more overlapped under aggressive data augmentations, thus simply aligning the positive samples (augmented views of the same sample) could make contrastive learning cluster intra-class samples together. Moreover, from the newly derived augmentation overlap perspective, we develop an unsupervised metric for the representation evaluation of contrastive learning, which aligns well with the downstream performance almost without relying on additional modules. Code is available at https://github.com/PKU-ML/GARC.

💡 Deep Analysis

📄 Full Content

Reference