Towards regularized learning from functional data with covariate shift

This paper investigates a general regularization framework for unsupervised domain adaptation in vector-valued regression under the covariate shift assumption, utilizing vector-valued reproducing kernel Hilbert spaces (vRKHS). Covariate shift occurs when the input distributions of the training and test data differ, introducing significant challenges for reliable learning. By restricting the hypothesis space, we develop a practical operator learning algorithm capable of handling functional outputs. We establish optimal convergence rates for the proposed framework under a general source condition, providing a theoretical foundation for regularized learning in this setting. We also propose an aggregation-based approach that forms a linear combination of estimators corresponding to different regularization parameters and different kernels. The proposed approach addresses the challenge of selecting appropriate tuning parameters, which is crucial for constructing a good estimator, and we provide a theoretical justification for its effectiveness. Furthermore, we illustrate the proposed method on a real-world face image dataset, demonstrating robustness and effectiveness in mitigating distributional discrepancies under covariate shift.

💡 Research Summary

This paper addresses the problem of unsupervised domain adaptation for vector‑valued regression when both inputs and outputs are functional (i.e., infinite‑dimensional) and the input distribution changes between source and target domains—a setting known as covariate shift. The authors build a rigorous regularization framework within vector‑valued reproducing kernel Hilbert spaces (vRKHS).

The key methodological steps are as follows. First, the covariate shift assumption is formalized as p(x,y)=p(y|x)p_X(x) and q(x,y)=p(y|x)q_X(x), with the conditional distribution p(y|x) shared across domains. Consequently, the regression function f* is identical for source and target, and the learning goal is to approximate f* in the L²(q_X,Y) norm despite the lack of target labels.

To handle the distribution mismatch, an importance‑weighting function β(x)=dq_X/dp_X is introduced. Using β, the empirical risk over source samples becomes a weighted least‑squares objective. The authors rewrite this objective in operator form by defining a sampling operator S_XS and its adjoint, together with a diagonal matrix B^{1/2} containing √β(x_i). The resulting normal equation is (S_XS* B S_XS)f = S_XS* B y.

Because directly solving this equation leads to overfitting, a broad class of regularization families {g_λ} is employed. The regularized estimator is f_{z,λ}=g_λ(S_XS* B S_XS) S_XS* B y. The framework encompasses Tikhonov, iterated Tikhonov, and spectral cut‑off regularization, each characterized by a qualification ν that quantifies the smoothness of the regularizer.

A central theoretical contribution is the derivation of optimal convergence rates under a general source condition. Assuming the true operator C* (the Hilbert‑Schmidt representation of f*) satisfies φ(T)C*∈S₂(H,Y) for an index function φ, the authors prove that both the L²(q_X,Y) error and the vRKHS norm error decay as O(λ^{ν}) (up to logarithmic factors). This result extends classical scalar‑valued rates to the vector‑valued, functional‑output setting and holds for any regularization family with qualification ν.

Recognizing that selecting λ and the kernel k is notoriously difficult in practice, the paper proposes an aggregation strategy. Multiple estimators are computed for a grid of (λ,k) pairs, and a convex combination of these estimators is formed. The authors show that this aggregated estimator inherits the same error bounds without requiring data‑driven tuning of λ or k, thereby alleviating a major practical obstacle.

Empirically, the method is evaluated on a face‑image dataset where source and target domains differ in lighting and pose, creating a realistic covariate shift. Importance weights β are estimated via the KuLSIF algorithm. The proposed operator‑learning plus aggregation approach achieves lower mean‑squared error and visibly better reconstructed faces compared with baseline Tikhonov regression that uses a single λ and kernel.

In summary, the paper makes four principal contributions: (1) it formulates covariate‑shift learning for functional data within the vRKHS framework; (2) it introduces a weighted operator‑learning algorithm and establishes optimal rates under general source conditions; (3) it devises an aggregation scheme that mitigates the tuning of regularization and kernel parameters; and (4) it validates the theory with real‑world image‑domain adaptation experiments. These results provide a solid foundation for high‑dimensional multi‑task regression, functional data analysis, and related applications where distributional shifts are unavoidable.