D$^2$-VR: Degradation-Robust and Distilled Video Restoration with Synergistic Optimization Strategy

The integration of diffusion priors with temporal alignment has emerged as a transformative paradigm for video restoration, delivering fantastic perceptual quality, yet the practical deployment of such frameworks is severely constrained by prohibitive inference latency and temporal instability when confronted with complex real-world degradations. To address these limitations, we propose \textbf{D$^2$-VR}, a single-image diffusion-based video-restoration framework with low-step inference. To obtain precise temporal guidance under severe degradation, we first design a Degradation-Robust Flow Alignment (DRFA) module that leverages confidence-aware attention to filter unreliable motion cues. We then incorporate an adversarial distillation paradigm to compress the diffusion sampling trajectory into a rapid few-step regime. Finally, a synergistic optimization strategy is devised to harmonize perceptual quality with rigorous temporal consistency. Extensive experiments demonstrate that D$^2$-VR achieves state-of-the-art performance while accelerating the sampling process by \textbf{12$\times$}

💡 Research Summary

The paper introduces D²‑VR, a novel video restoration framework that leverages a single‑image diffusion prior while addressing two major bottlenecks of existing diffusion‑based video restoration methods: high inference latency due to multi‑step sampling and temporal instability caused by unreliable optical flow under severe degradations.
The architecture consists of three key components. First, the Degradation‑Robust Flow Alignment (DRFA) module augments a global motion aggregation (GMA) backbone with a confidence‑aware attention mechanism. A lightweight confidence estimator Φ processes contextual features to produce a per‑pixel confidence map C, which is log‑transformed into a bias term M and added to the standard attention score. This bias suppresses contributions from low‑confidence regions, effectively filtering out noisy flow cues and yielding more accurate motion‑compensated features even in heavily degraded frames.
Second, an Adversarial Distillation scheme compresses the diffusion sampling trajectory from dozens of steps to only four (timesteps 750, 500, 250, 0). A frozen teacher diffusion model (Stable Diffusion 2.1 x4 Upscaler) provides score‑distillation supervision (SDS) to the student model. To counteract the typical over‑smoothing of few‑step diffusion, the authors employ a feature‑based spatial adversarial loss: a pre‑trained UNet encoder acts as a discriminator, encouraging the student to generate high‑frequency textures that are indistinguishable from real latent data at the selected timesteps.
Third, to preserve temporal coherence, a Temporal‑LPIPS (T‑LPIPS) loss is introduced. Unlike simple frame‑wise L2 or perceptual losses, T‑LPIPS aligns the magnitude of perceptual changes between consecutive generated frames with those of the ground‑truth high‑quality video, thereby directly penalizing flickering and inconsistent motion. The total training objective combines the distillation loss, the spatial adversarial loss (weighted by λ₁), and the temporal consistency loss (weighted by λ₂).
Extensive experiments on synthetic REDS30 and real‑world VideoLQ datasets demonstrate that D²‑VR outperforms state‑of‑the‑art methods such as Real‑ESRGAN, StableVSR, DO‑VE, and others across a comprehensive suite of metrics: PSNR, SSIM, LPIPS, DISTS, MUSIQ, MANIQA, CLIP‑IQA, NIQE, as well as temporal metrics tOF and tLPIPS. Notably, D²‑VR achieves a 12× speedup in inference while maintaining or improving perceptual quality, and it operates with a lightweight model suitable for consumer‑grade GPUs. Ablation studies confirm that each component—DRFA, adversarial distillation, and the synergistic combination of spatial adversarial and T‑LPIPS losses—contributes significantly to the final performance.
In summary, D²‑VR presents the first diffusion‑based video restoration system that simultaneously delivers high‑fidelity texture reconstruction, robust motion alignment under severe degradations, and ultra‑fast few‑step inference, making it a strong candidate for real‑world deployment in applications such as smartphone video enhancement, live streaming upscaling, and archival video restoration.