TeleBoost: A Systematic Alignment Framework for High-Fidelity, Controllable, and Robust Video Generation

Post-training is the decisive step for converting a pretrained video generator into a production-oriented model that is instruction-following, controllable, and robust over long temporal horizons. This report presents a systematical post-training framework that organizes supervised policy shaping, reward-driven reinforcement learning, and preference-based refinement into a single stability-constrained optimization stack. The framework is designed around practical video-generation constraints, including high rollout cost, temporally compounding failure modes, and feedback that is heterogeneous, uncertain, and often weakly discriminative. By treating optimization as a staged, diagnostic-driven process rather than a collection of isolated tricks, the report summarizes a cohesive recipe for improving perceptual fidelity, temporal coherence, and prompt adherence while preserving the controllability established at initialization. The resulting framework provides a clear blueprint for building scalable post-training pipelines that remain stable, extensible, and effective in real-world deployment settings.

💡 Research Summary

TeleBoost presents a systematic post‑training framework that transforms a pretrained video diffusion model into a production‑ready system capable of following instructions, offering fine‑grained control, and remaining robust over long temporal horizons. The authors argue that simply adding reinforcement learning (RL) to a video generator is insufficient due to three core challenges: (1) the high computational cost of video rollouts, (2) temporally compounding errors that cause artifacts to cascade across frames, and (3) heterogeneous, uncertain, and often weakly discriminative feedback signals. To address these, the paper organizes post‑training into three sequential stages, each guided by three design principles—feedback reliability, structural alignment of learning signals, and adaptivity over training time.

Stage I – Supervised Fine‑Tuning (SFT).
Starting from a frozen large‑scale video diffusion backbone, the decoder is fine‑tuned with a unified loss that combines (a) instruction and control supervision (e.g., temporal ordering, camera motion, compositional edits), (b) spatial‑structure awareness (3‑D consistency across frames, evaluated via automatic geometric stability metrics derived from real, simulated, and synthetic videos), and (c) physics‑aware motion supervision (auxiliary optical‑flow prediction trained on real fluid videos and physics simulations). The goal is not to maximize perceptual quality yet, but to shape a stable policy that respects geometric and physical constraints, thereby providing a reliable reference for downstream optimization.

Stage II – Group‑based Reinforcement Learning with Relative Optimization (GRPO).
Using the SFT policy π_ref as a baseline, the system generates multiple samples per prompt group and derives relative rewards from pairwise or groupwise comparisons, avoiding the need for absolute value critics that are noisy in the video domain. The relative objective is optimized with PPO‑style policy updates, while dynamically re‑weighting loss terms based on feedback confidence (e.g., variance of CLIP‑like scores) and structural error signals (e.g., depth inconsistency). This stage directly improves measurable objectives such as perceptual fidelity (measured by video‑CLIP scores), temporal coherence (measured by motion smoothness and frame‑to‑frame consistency), and adherence to the instruction set, all under explicit stability constraints to prevent policy collapse.

Stage III – Direct Preference Optimization (DPO).
After the model exhibits strong low‑level quality and stability, human preference data—collected as pairwise judgments on generated videos—is used to fine‑tune the policy with a log‑likelihood loss that aligns the model with holistic judgments that are difficult to encode as explicit rewards (e.g., storytelling flow, aesthetic appeal). The DPO step respects the structural constraints learned earlier, ensuring that the final model does not revert to earlier failure modes while achieving higher human‑rated quality.

Across all stages, the framework incorporates cross‑stage diagnostics and slice‑based evaluation to monitor rollout stability, reward signal saturation, and alignment drift. Ablation studies demonstrate that each component—spatial‑structure supervision, physics‑aware loss, relative RL, and human preference fine‑tuning—contributes significantly to the final performance. Compared to baselines that apply a single RL fine‑tuning or only supervised adaptation, TeleBoost achieves (i) reduced prompt sensitivity, (ii) markedly better temporal consistency on videos longer than 30 seconds, and (iii) higher human preference scores in blind evaluations.

The authors conclude that a staged, diagnostic‑driven optimization pipeline is essential for scaling video generation to real‑world deployment. The approach balances the need for high‑quality perceptual output with the practical constraints of rollout cost and noisy feedback, offering a blueprint that can be extended to other high‑cost generative domains such as robotics simulation or 3D scene synthesis. Future work includes scaling human preference collection via automated preference models, integrating online feedback loops for continual improvement, and exploring more efficient rollout strategies for real‑time applications.