DreamVAR: Taming Reinforced Visual Autoregressive Model for High-Fidelity Subject-Driven Image Generation

Recent advances in subject-driven image generation using diffusion models have attracted considerable attention for their remarkable capabilities in producing high-quality images. Nevertheless, the potential of Visual Autoregressive (VAR) models, despite their unified architecture and efficient inference, remains underexplored. In this work, we present DreamVAR, a novel framework for subject-driven image synthesis built upon a VAR model that employs next-scale prediction. Technically, multi-scale features of the reference subject are first extracted by a visual tokenizer. Instead of interleaving these conditional features with target image tokens across scales, our DreamVAR pre-fills the full subject feature sequence prior to predicting target image tokens. This design simplifies autoregressive dependencies and mitigates the train-test discrepancy in multi-scale conditioning scenario within the VAR paradigm. DreamVAR further incorporates reinforcement learning to jointly enhance semantic alignment and subject consistency. Extensive experiments demonstrate that DreamVAR achieves superior appearance preservation compared to leading diffusion-based methods.

💡 Research Summary

DreamVAR introduces a novel subject‑driven image generation framework built on Visual Autoregressive (VAR) models, which predict image tokens scale‑by‑scale rather than pixel‑by‑pixel. While diffusion models dominate the field, VAR offers a unified architecture, fast inference, and natural multimodal conditioning. The authors identify a critical train‑test discrepancy in existing multi‑scale conditioning approaches: during training, models receive ground‑truth history (teacher‑forcing), but at inference they must rely on their own generated tokens. This gap is amplified when subject features are interleaved with target image tokens across scales, leading to poor subject fidelity.

To close this gap, DreamVAR adopts a “pre‑filling” strategy. A visual tokenizer extracts multi‑scale features of the reference subject (I_s1 … I_sK). These features are left‑padded before the textual prompt tokens, forming a sequence (I_s1,…,I_sK, C_t, I_1,…,I_K). The VAR model then predicts each target scale I_k conditioned only on the previously generated lower‑scale image tokens and the full, fixed subject‑prompt context. This eliminates the need for joint control‑image modeling and removes the train‑test distribution mismatch.

Beyond architectural changes, the paper leverages reinforcement learning to jointly optimize two objectives: (1) Subject Consistency Reward (R_I), measured by cosine similarity between the generated image and the reference subject in a visual feature space (e.g., DINO or CLIP‑V) after segmentation; (2) Semantic Alignment Reward (R_S), measured by CLIP similarity between the generated image and the textual prompt. A weighted sum R = α R_I + γ R_S serves as the final reward. The authors employ Group Relative Policy Optimization (GRPO), sampling a batch of G images per iteration, normalizing advantages across the group, and adding a KL‑regularization term to keep the updated policy close to a frozen reference model. Empirically, α = 1.0 and γ = 2.0 provide the best trade‑off.

Training proceeds in three stages. Stage 1 (Task Adaptation) fine‑tunes the pre‑trained Infinity‑2B VAR model on the Subject‑200K dataset to acquire basic subject‑driven capabilities. Stage 2 (Supervised Fine‑Tuning) refines visual fidelity using a curated high‑quality DreamSubject‑14K dataset, which is built by generating diverse captions for Object365 categories, synthesizing images with a text‑to‑image model, filtering low‑quality outputs, and performing precise object detection and segmentation. Stage 3 applies GRPO to further improve both subject preservation and prompt alignment.

Evaluation on the DreamBench benchmark reports three metrics: DINO (visual similarity to reference subject), CLIP‑I (image‑subject similarity), and CLIP‑T (image‑prompt similarity). DreamVAR, with only 2 B parameters, achieves DINO = 0.764, CLIP‑I = 0.838, and CLIP‑T = 0.310, surpassing larger diffusion‑based baselines such as OminiControl and UNO (12 B parameters). It also demonstrates a 1.75× speedup in training and reduces inference time from 17 s to 2 s per image.

Ablation studies confirm: (i) using only R_I boosts subject detail but harms prompt alignment (reward hacking); adding R_S restores balance; (ii) multi‑scale subject features outperform single‑scale features, confirming richer detail capture; (iii) the pre‑filling conditioning outperforms the interleaved approach, validating the mitigation of train‑test discrepancy; (iv) the chosen α/γ values yield the best combined scores.

In summary, DreamVAR shows that VAR models, when equipped with multi‑scale pre‑filled subject conditioning and reinforced with multi‑reward policy optimization, can achieve state‑of‑the‑art subject fidelity and competitive prompt alignment while remaining computationally efficient. This work opens a new direction for subject‑driven generation, suggesting that autoregressive paradigms, traditionally considered slower, can be competitive with diffusion when carefully engineered.