AutoRefiner: Improving Autoregressive Video Diffusion Models via Reflective Refinement Over the Stochastic Sampling Path

Autoregressive video diffusion models (AR-VDMs) show strong promise as scalable alternatives to bidirectional VDMs, enabling real-time and interactive applications. Yet there remains room for improvement in their sample fidelity. A promising solution is inference-time alignment, which optimizes the noise space to improve sample fidelity without updating model parameters. Yet, optimization- or search-based methods are computationally impractical for AR-VDMs. Recent text-to-image (T2I) works address this via feedforward noise refiners that modulate sampled noises in a single forward pass. Can such noise refiners be extended to AR-VDMs? We identify the failure of naively extending T2I noise refiners to AR-VDMs and propose AutoRefiner-a noise refiner tailored for AR-VDMs, with two key designs: pathwise noise refinement and a reflective KV-cache. Experiments demonstrate that AutoRefiner serves as an efficient plug-in for AR-VDMs, effectively enhancing sample fidelity by refining noise along stochastic denoising paths.

💡 Research Summary

The paper “AutoRefiner: Improving Autoregressive Video Diffusion Models via Reflective Refinement Over the Stochastic Sampling Path” addresses a critical bottleneck in Autoregressive Video Diffusion Models (AR-VDMs): the trade-off between computational efficiency and sample fidelity. While AR-VDMs are highly scalable and suitable for real-time, interactive video generation, they often struggle to produce high-fidelity samples compared to their bidirectional counterparts.

The authors identify that while inference-time alignment—optimizing the noise space to improve quality without retraining—is a promising direction, traditional optimization-based or search-based methods are computationally prohibitive for the autoregressive structure of AR-VDMs. Although feedforward noise refiners have successfully enhanced Text-to-Image (T2I) models, the authors demonstrate that a naive extension of these T2I refiners to AR-VDMs fails. This failure stems from the temporal dependencies inherent in AR-VDMs; modifying noise in a single forward pass without considering the temporal context leads to temporal inconsistencies and artifacts across frames.

To overcome this, the paper introduces “AutoRefiner,” a specialized noise refiner designed specifically for the autoregressive paradigm. The architecture relies on two fundamental innovations:

1. Pathwise Noise Refinement: Instead of treating noise refinement as a single-step correction, AutoRefiner refines the noise along the stochastic denoising trajectory. By considering the entire sampling path, the model ensures that the refinements are consistent with the diffusion process, leading to more stable and high-quality sample generation.
2. Reflective KV-cache: To maintain temporal continuity, the authors propose a reflective mechanism for the Key-Value (KV) cache. This allows the refined noise information to be integrated into the autoregressive context, ensuring that the improvements in one frame are reflected in subsequent frames without disrupting the established temporal flow.

Experimental results validate that AutoRefiner acts as an efficient, “plug-and-play” module that can be integrated into existing AR-VDMs without any parameter updates. The method significantly enhances sample fidelity and visual quality while maintaining the computational advantages of the autoregressive framework. This work provides a scalable and efficient paradigm for enhancing the quality of generative video models, paving the way for more realistic and interactive video synthesis technologies.