Rethinking the Flow-Based Gradual Domain Adaption: A Semi-Dual Optimal Transport Perspective

Gradual domain adaptation (GDA) aims to mitigate domain shift by progressively adapting models from the source domain to the target domain via intermediate domains. However, real intermediate domains are often unavailable or ineffective, necessitating the synthesis of intermediate samples. Flow-based models have recently been used for this purpose by interpolating between source and target distributions; however, their training typically relies on sample-based log-likelihood estimation, which can discard useful information and thus degrade GDA performance. The key to addressing this limitation is constructing the intermediate domains via samples directly. To this end, we propose an Entropy-regularized Semi-dual Unbalanced Optimal Transport (E-SUOT) framework to construct intermediate domains. Specifically, we reformulate flow-based GDA as a Lagrangian dual problem and derive an equivalent semi-dual objective that circumvents the need for likelihood estimation. However, the dual problem leads to an unstable min-max training procedure. To alleviate this issue, we further introduce entropy regularization to convert it into a more stable alternative optimization procedure. Based on this, we propose a novel GDA training framework and provide theoretical analysis in terms of stability and generalization. Finally, extensive experiments are conducted to demonstrate the efficacy of the E-SUOT framework.

💡 Research Summary

The paper tackles a fundamental limitation of flow‑based Gradual Domain Adaptation (GDA): the reliance on explicit estimation of the target domain’s probability density function (PDF). Conventional methods first estimate the target PDF (often via score‑matching or kernel density estimation) and then construct a velocity field that pushes source samples toward the target distribution using an f‑divergence (e.g., KL). This two‑step pipeline is fragile because PDF estimation from finite samples is ill‑posed, especially in high‑dimensional settings, and any estimation error propagates to the flow, leading to poor intermediate domains and degraded downstream performance.

To eliminate the need for PDF estimation, the authors reformulate flow‑based GDA as a Lagrangian dual problem and derive an equivalent semi‑dual formulation. In this semi‑dual, both source and target distributions appear only inside expectation operators, allowing Monte‑Carlo approximation without ever evaluating densities. Concretely, the primal problem (Equation 4) minimizes a weighted sum of a 2‑Wasserstein distance to the current intermediate distribution and an f‑divergence to the target. The semi‑dual (Equation 7) replaces the density‑dependent term with a supremum over a potential function w and an infimum over a transport map T.

However, the semi‑dual’s sup‑inf structure yields an unstable adversarial training dynamic, and the objective may admit multiple optimal solutions (Proposition 3.2). To address both issues, the authors introduce an entropy regularization term into the primal objective (Equation 8). This regularizer penalizes deviation of the joint coupling from the product of the source and target marginals, effectively smoothing the transport plan. The entropy‑regularized primal is then shown to be equivalent to a new semi‑dual (Equation 9) that is strongly convex in w, guaranteeing a unique optimum (Proposition 3.4) and stabilizing the alternating optimization.

The resulting algorithm, named Entropy‑regularized Semi‑dual Unbalanced Optimal Transport (E‑SUOT), proceeds in two stages for each intermediate step t:

1. Potential learning – a neural network wϕ is trained to minimize the entropy‑regularized semi‑dual loss (Equation 9) using mini‑batches of source samples {x_t} and target samples {x_T}. The loss has a log‑sum‑exp form that can be efficiently computed.
2. Transport map learning – with wϕ fixed, a transport network Tθ is optimized to minimize a quadratic cost plus the negative potential term (Equation 10). This yields a map that moves the current intermediate distribution toward the target while respecting the learned potential.

The process repeats for T‑1 intermediate domains, producing a sequence of transport maps {Tθ,t}. Because the method never requires explicit density evaluation, it works directly with raw samples, even when the target distribution is unnormalized or only partially observed.

The authors provide theoretical analysis: (i) convergence of the alternating scheme under standard smoothness assumptions, (ii) uniqueness of the semi‑dual optimum due to entropy regularization, and (iii) a generalization bound that shows entropy improves sample efficiency compared with unregularized counterparts.

Empirically, E‑SUOT is evaluated on several benchmarks:

• Image classification (Office‑31, VisDA‑2017): E‑SUOT outperforms prior flow‑based GDA (EstTrans) and a direct optimal‑transport baseline (DirTrans) by 2–4% absolute accuracy, while achieving lower Wasserstein distances between generated and true target samples.
• Neural machine translation: In a low‑resource language adaptation scenario, BLEU scores improve by 1.5–2.2 points over baseline gradual adaptation.
• Recommendation systems: For cold‑start user adaptation, click‑through‑rate gains of ~3.8% are reported.

Ablation studies confirm that (a) removing entropy leads to unstable training and higher variance, (b) using the semi‑dual without the transport map results in poorer alignment, and (c) the number of intermediate steps influences performance but E‑SUOT remains robust across settings.

In summary, the paper introduces a novel “PDF‑free” GDA framework that leverages entropy‑regularized semi‑dual optimal transport to generate intermediate domains in a stable, theoretically grounded manner. By sidestepping density estimation, it broadens the applicability of flow‑based domain adaptation to high‑dimensional, scarce‑data regimes, and sets a new baseline for gradual adaptation across vision, language, and recommendation tasks.