A Unified Density Operator View of Flow Control and Merging

Recent progress in large-scale flow and diffusion models raised two fundamental algorithmic challenges: (i) control-based reward adaptation of pre-trained flows, and (ii) integration of multiple models, i.e., flow merging. While current approaches address them separately, we introduce a unifying probability-space framework that subsumes both as limit cases, and enables reward-guided flow merging, allowing principled, task-aware combination of multiple pre-trained flows (e.g., merging priors while maximizing drug-discovery utilities). Our formulation renders possible to express a rich family of operators over generative models densities, including intersection (e.g., to enforce safety), union (e.g., to compose diverse models), interpolation (e.g., for discovery), their reward-guided counterparts, as well as complex logical expressions via generative circuits. Next, we introduce Reward-Guided Flow Merging (RFM), a mirror-descent scheme that reduces reward-guided flow merging to a sequence of standard fine-tuning problems. Then, we provide first-of-their-kind theoretical guarantees for reward-guided and pure flow merging via RFM. Ultimately, we showcase the capabilities of the proposed method on illustrative settings providing visually interpretable insights, and apply our method to high-dimensional de-novo molecular design and low-energy conformer generation.

💡 Research Summary

The paper tackles two central challenges that have emerged as generative flow and diffusion models move from impressive laboratory results to real‑world scientific applications: (i) adapting a pretrained flow to maximize a downstream utility (reward‑guided fine‑tuning) and (ii) merging several pretrained flows into a single model (flow merging). Existing work treats these problems separately, using distinct mathematical formulations and algorithms. The authors propose a unified probability‑space optimization framework that subsumes both as special cases and enables a new class of “reward‑guided flow merging” where multiple priors are combined while steering the resulting density toward high‑reward regions.

The core of the framework is a density operator O that takes n pretrained policies π_pre,i (and associated marginal densities p_pre,i,1) and returns a merged policy π*. The merged policy is defined as the solution of

 max_π  E_{x∼p_{π,1}}