Gradual Fine-Tuning for Flow Matching Models

Fine-tuning flow matching models is a central challenge in settings with limited data, evolving distributions, or strict efficiency demands, where unconstrained fine-tuning can erode the accuracy and efficiency gains learned during pretraining. Prior work has produced theoretical guarantees and empirical advances for reward-based fine-tuning formulations, but these methods often impose restrictions on permissible drift structure or training techniques. In this work, we propose Gradual Fine-Tuning (GFT), a principled framework for fine-tuning flow-based generative models when samples from the target distribution are available. For stochastic flows, GFT defines a temperature-controlled sequence of intermediate objectives that smoothly interpolate between the pretrained and target drifts, approaching the true target as the temperature approaches zero. We prove convergence results for both marginal and conditional GFT objectives, enabling the use of suitable (e.g., optimal transport) couplings during GFT while preserving correctness. Empirically, GFT improves convergence stability and shortens probability paths, resulting in faster inference, while maintaining generation quality comparable to standard fine-tuning. Our results position GFT as a theoretically grounded and practically effective alternative for scalable adaptation of flow matching models under distribution shift.

💡 Research Summary

The paper addresses the problem of adapting pretrained flow‑matching generative models to new target distributions when only samples from the target are available. Existing fine‑tuning approaches for flow models largely rely on external reward functions or impose restrictive linear control structures, which limits their applicability, prevents the use of optimal‑transport (OT) couplings, and is incompatible with popular nonlinear adaptation techniques such as LoRA.

To overcome these limitations, the authors propose Gradual Fine‑Tuning (GFT), a principled framework that interpolates between the pretrained dynamics and the target dynamics via a temperature‑controlled regularization term. Formally, GFT minimizes a composite objective
\