Factorizable joint shift revisited

Factorizable joint shift (FJS) represents a type of distribution shift (or dataset shift) that comprises both covariate and label shift. Recently, it has been observed that FJS actually arises from consecutive label and covariate (or vice versa) shifts. Research into FJS so far has been confined to the case of categorical labels. We propose a framework for analysing distribution shift in the case of a general label space, thus covering both classification and regression models. Based on the framework, we generalise existing results on FJS to general label spaces and present and analyse a related extension of the expectation maximisation (EM) algorithm for class prior probabilities. We also take a fresh look at generalized label shift (GLS) in the case of a general label space.

💡 Research Summary

This paper revisits the concept of Factorizable Joint Shift (FJS), a type of distribution shift that simultaneously encompasses covariate shift and label shift, and extends its theoretical framework beyond categorical labels to general label spaces, thereby covering classification, regression, and mixed-type problems. The authors begin by formalizing the domain adaptation setting in a rigorous probabilistic language: an instance space Ω = Ω_X × Ω_Y, a source joint distribution P(X,Y), and a target joint distribution Q(X,Y). They assume absolute continuity of Q with respect to P (Assumption 2.2), which introduces a Radon‑Nikodym derivative f that serves as an importance‑weighting function linking the two domains.

The paper’s first major contribution is the systematic decomposition of distribution shift into its “building blocks.” By defining source and target marginal distributions (P_X, P_Y, Q_X, Q_Y) and regular conditional distributions (P_{Y|X}, P_{X|Y}, Q_{Y|X}, Q_{X|Y}), the authors create a taxonomy that can be applied regardless of whether the label space Ω_Y is finite, countably infinite, or continuous. They discuss the existence of regular conditional distributions and invoke Fubini’s theorem to express expectations in terms of marginal and conditional components, laying the groundwork for later derivations.

In Section 3 the authors formalize FJS for general label spaces. They show that under FJS the target joint density can be written as

  Q(x,y) = α(y)·β(x)·P(x,y),

where α(y) is a label‑space scaling function (the target prior or density) and β(x) is a feature‑space weighting function. When Ω_Y is continuous, α(y) becomes a probability density function; when Ω_Y is discrete, α reduces to class‑prior ratios. This factorization mirrors the earlier categorical treatment but now holds for any measurable label space.

A central methodological advance is the generalization of the Expectation‑Maximization (EM) algorithm originally proposed by Saerens et al. (2002) for estimating class priors under label shift. The authors adapt the EM steps to the continuous‑label setting: the E‑step computes the posterior Q_{Y|X} using the current estimates of α and β, while the M‑step updates α by minimizing the Kullback‑Leibler divergence between the estimated target label marginal and the empirical marginal induced by the E‑step. They prove that this generalized EM retains the monotonic likelihood increase property and converges to a stationary point of the likelihood under the absolute continuity assumption. The M‑step for β is discussed separately; β can be estimated via importance‑weighted regression, kernel density estimation, or parametric models, depending on the nature of Ω_X.

Section 4 investigates the relationship between Generalized Label Shift (GLS) and FJS. GLS assumes that only the label marginal changes (α ≠ 1) while the conditional distribution P_{X|Y} remains invariant (β ≡ 1). The authors demonstrate that GLS is a special case of FJS and that, under GLS, estimating α alone suffices for successful domain adaptation. Conversely, when β deviates from unity, both α and β must be estimated, highlighting the broader applicability of FJS.

The paper concludes with a summary of contributions and a roadmap for future work. The authors suggest empirical validation on regression and mixed‑type datasets, exploration of scalable β‑estimation techniques for high‑dimensional feature spaces, and integration of the generalized EM into deep learning pipelines for end‑to‑end domain adaptation.

Overall, the work provides a solid theoretical extension of FJS to arbitrary label spaces, offers a mathematically sound EM algorithm for prior estimation in this broader setting, and clarifies how GLS fits within the FJS hierarchy. This advances the state of the art in domain adaptation by enabling principled handling of distribution shifts in regression and other non‑categorical tasks.