Semantic Bridging Domains: Pseudo-Source as Test-Time Connector

Distribution shifts between training and testing data are a critical bottleneck limiting the practical utility of models, especially in real-world test-time scenarios. To adapt models when the source domain is unknown and the target domain is unlabeled, previous works constructed pseudo-source domains via data generation and translation, then aligned the target domain with them. However, significant discrepancies exist between the pseudo-source and the original source domain, leading to potential divergence when correcting the target directly. From this perspective, we propose a Stepwise Semantic Alignment (SSA) method, viewing the pseudo-source as a semantic bridge connecting the source and target, rather than a direct substitute for the source. Specifically, we leverage easily accessible universal semantics to rectify the semantic features of the pseudo-source, and then align the target domain using the corrected pseudo-source semantics. Additionally, we introduce a Hierarchical Feature Aggregation (HFA) module and a Confidence-Aware Complementary Learning (CACL) strategy to enhance the semantic quality of the SSA process in the absence of source and ground truth of target domains. We evaluated our approach on tasks like semantic segmentation and image classification, achieving a 5.2% performance boost on GTA2Cityscapes over the state-of-the-art.

💡 Research Summary

The paper tackles the challenging setting of Test‑Time Adaptation (TTA), where a model trained on a source domain must be adapted at inference time using only unlabeled target data, with no access to the original source samples. Existing approaches address this “source‑free” scenario by constructing a pseudo‑source domain from the target data via generation, style transfer, or memory selection, and then aligning the target directly with this pseudo‑source. However, the pseudo‑source inevitably diverges from the true source distribution, and direct alignment can therefore degrade performance or even cause divergence.

To overcome this limitation, the authors propose Stepwise Semantic Alignment (SSA), a paradigm that treats the pseudo‑source not as a substitute for the source but as a semantic bridge between the inaccessible source and the target. SSA proceeds in two main stages:

1. Data Selection and Pseudo‑Source Construction – For each target sample, the model’s output probability distribution is used to compute entropy. An exponential moving average (EMA) smooths entropy over training epochs, yielding a stable uncertainty estimate. Samples with low entropy (high confidence) are assigned to the pseudo‑source set (D_{ps}); the remaining samples form the “remaining‑target” set (D_{rt}). This split creates a high‑confidence subset that can serve as a reliable intermediate domain.

2. Semantic Correction of the Pseudo‑Source – The method leverages universal semantic features extracted from a frozen, pre‑trained visual backbone (e.g., ImageNet‑trained ResNet). For any input (x), the current model’s feature map (f(x)) is aligned with the backbone’s feature map (f_{pre}(x)) via a cosine‑similarity loss: \