EasyV2V: A High-quality Instruction-based Video Editing Framework

EasyV2V: A High-quality Instruction-based Video Editing Framework Jinjie Mai1,2∗ Chaoyang Wang2 Guocheng Gordon Qian2 Willi Menapace2 Sergey Tulyakov2 Bernard Ghanem1 Peter Wonka2,1 Ashkan Mirzaei2∗† 1 KAUST 2 Snap Inc. ∗Core contributors † Project lead Project Page: https://snap-research.github.io/easyv2v/ Make them smile, shake hands, and kiss Turn her into a robot Input Mask (via Image Editing) A random frame transformed via image editing EasyV2V Make it into an anime style and change the sign to say “EasyV2V” Input EasyV2V Edit Here! Input Reference Image EasyV2V The lamp smoothly turns on lighting up the room. Input EasyV2V w/ Text Instruction + Mask Instruction + Temporal Control + Reference Figure 1. EasyV2V unifies data processing, architecture, and control for high-quality, instruction-based video editing with flexible inputs (text, masks, edit timing). The figure illustrates its versatility across diverse input types and editing tasks. Abstract While image editing has advanced rapidly, video editing re- mains less explored, facing challenges in consistency, con- trol, and generalization. We study the design space of data, architecture, and control, and introduce EasyV2V, a simple and effective framework for instruction-based video edit- ing. On the data side, we compose existing experts with fast inverses to build diverse video pairs, lift image edit pairs into videos via single-frame supervision and pseudo pairs with shared affine motion, mine dense-captioned clips for video pairs, and add transition supervision to teach how edits unfold. On the model side, we observe that pretrained text-to-video models possess editing capability, motivating a simplified design. Simple sequence concate- nation for conditioning with light LoRA fine-tuning suf- fices to train a strong model. For control, we unify spa- tiotemporal control via a single mask mechanism and sup- port optional reference images. Overall, EasyV2V works with flexible inputs, e.g., video+text, video+mask+text, video+mask+reference+text, and achieves state-of-the-art video editing results, surpassing concurrent and commer- cial systems. 1. Introduction What makes a good instruction-based video editor? We ar- gue that three components govern performance: data, archi- tecture, and control. This paper analyzes the design space of these components and distills a recipe that works great in practice. The result is a lightweight model that reaches state-of-the-art quality while accepting flexible inputs. Training-free video editors adapt pretrained generators but are fragile and slow [10, 26]. Training-based ap- proaches improve stability, yet many target narrow tasks such as ControlNet-style conditioning [1, 20], video in- painting [72], or reenactment [8]. General instruction-based video editors handle a wider range of edits [9, 53, 64, 73], yet still lag behind image-based counterparts in visual fi- delity and control. We set out to narrow this gap. Figure 2 motivates our design philosophy: modern video models al- ready know how to transform videos. To unlock this emerg- ing capability with minimal adaptation, we conduct a com- prehensive investigation into data curation, architectural de- 1 arXiv:2512.16920v1 [cs.CV] 18 Dec 2025 A man playing the piano gradually transforms into a panda A woman wearing a red tshirt that slowly turns blue Figure 2. A pretrained text-to-video model can mimic common editing effects without finetuning. This suggests that much of the “how” of video editing already lives inside modern backbones. sign, and instruction-control. Data. We see three data strategies. (A) One general- ist model renders all edits and is used to self-train an ed- itor [9, 53, 64]. This essentially requires a single teacher model that already solves the problem in high quality. (B) Design and train new experts for specific edit types, then synthesize pairs at scale [73]. This yields higher per-task fidelity and better coverage of hard skills, but training and maintaining many specialists is expensive and slows iter- ation and adaptation to future base models. We propose a new strategy: (C) Select existing experts and compose them. We focus on experts with a fast inverse (e.g., edge↔video, depth↔video) and compose more complex experts from them. This makes supervision easy to obtain, and experts are readily available as off-the-shelf models. It keeps costs low and diversity high by standing on strong off-the-shelf modules; the drawback is heterogeneous artifacts across ex- perts, which we mitigate through filtering and by favoring experts with reliable inverses. Concretely, we combine off- the-shelf video/image experts for stylization, local edits, in- sertion/removal, and human animation; we also convert im- age edit pairs into supervision via two routes: single-frame training and pseudo video-to-video (V2V) pairs created by applying the same smooth camera transforms to source and edited images. Lifting image-to-image (I2I) data increases sca

This content is AI-processed based on open access ArXiv data.