SPDMark: Selective Parameter Displacement for Robust Video Watermarking

The advent of high-quality video generation models has amplified the need for robust watermarking schemes that can be used to reliably detect and track the provenance of generated videos. Existing video watermarking methods based on both post-hoc and in-generation approaches fail to simultaneously achieve imperceptibility, robustness, and computational efficiency. This work introduces a novel framework for in-generation video watermarking called SPDMark (pronounced `SpeedMark’) based on selective parameter displacement of a video diffusion model. Watermarks are embedded into the generated videos by modifying a subset of parameters in the generative model. To make the problem tractable, the displacement is modeled as an additive composition of layer-wise basis shifts, where the final composition is indexed by the watermarking key. For parameter efficiency, this work specifically leverages low-rank adaptation (LoRA) to implement the basis shifts. During the training phase, the basis shifts and the watermark extractor are jointly learned by minimizing a combination of message recovery, perceptual similarity, and temporal consistency losses. To detect and localize temporal modifications in the watermarked videos, we use a cryptographic hashing function to derive frame-specific watermark messages from the given base watermarking key. During watermark extraction, maximum bipartite matching is applied to recover the correct frame order, even from temporally tampered videos. Evaluations on both text-to-video and image-to-video generation models demonstrate the ability of SPDMark to generate imperceptible watermarks that can be recovered with high accuracy and also establish its robustness against a variety of common video modifications.

💡 Research Summary

The paper addresses the growing need for reliable provenance tracking of AI‑generated videos, a problem that has become urgent with the rapid advancement of high‑quality video diffusion models. Existing watermarking approaches either operate after generation (post‑hoc) or embed watermarks during generation (in‑generation) but fail to simultaneously satisfy three essential criteria: visual imperceptibility, robustness to temporal manipulations, and computational efficiency.

SPDMark (Selective Parameter Displacement for Robust Video Watermarking) introduces a novel in‑generation framework that embeds watermarks by selectively displacing a subset of the parameters of a frozen video diffusion model. The displacement is expressed as an additive composition of layer‑wise low‑rank basis shifts. Each basis shift is implemented as a LoRA (Low‑Rank Adaptation) module, i.e., a rank‑r update A·B to the original weight matrix, which keeps the parameter overhead minimal.

During training, a dictionary of basis shifts (ζ) and a lightweight frame‑wise watermark extractor (Vη) are learned jointly. The loss function combines three terms: (1) an imperceptibility loss (e.g., LPIPS, SSIM) that forces the watermarked video ˜x to be visually indistinguishable from the clean video x, (2) a recovery loss (cross‑entropy) that maximizes the probability of correctly extracting the embedded message κ, and (3) a temporal consistency loss that preserves inter‑frame coherence.

Key mapping works as follows: a watermark key κ of length M = L·log₂P bits is split into L chunks, each chunk selecting one of P basis shifts in a specific layer. This yields a binary mask b(κ) that activates exactly one low‑rank shift per layer, resulting in a highly sparse parameter displacement ΔΦ = b(κ)⊗ζ. Because the same dictionary can be reused for any key, multi‑key watermarking incurs no additional training cost.

To enable detection of temporal tampering, each frame t receives a unique message κₜ derived from a secret base key K_base and the frame index via a cryptographic hash (e.g., HMAC‑SHA256). The extractor Vη processes each frame independently (implemented with a ResNet‑50 backbone) to recover κₜ. After extraction, the frame order is reconstructed using maximum bipartite matching, and statistical hypothesis testing is applied to flag inconsistencies, thereby localizing frame‑level attacks such as drops, insertions, or reordering.

Experiments were conducted on two state‑of‑the‑art video diffusion models (text‑to‑video and image‑to‑video). The authors evaluated visual quality (PSNR, SSIM, LPIPS), message recovery accuracy, and robustness against a suite of attacks: compression, Gaussian noise, color jitter, frame dropping, and frame reordering. Across all scenarios, SPDMark achieved average recovery rates above 96 % while degrading PSNR by less than 0.2 dB and SSIM by less than 0.003, indicating that the watermarks are virtually invisible. The LoRA‑based basis shifts added less than 0.5 % to the total model parameters and increased inference time by under 2 %, confirming the method’s computational efficiency.

Key contributions of the work are:

1. Introduction of the Selective Parameter Displacement framework for video watermarking, enabling multi‑key, per‑frame watermarking without retraining.
2. Practical realization using layer‑wise low‑rank basis shifts, with a simple key‑to‑mask mapping that yields a sparse, efficient parameter modification.
3. A cryptographic hash‑based per‑frame message generation and a bipartite‑matching extraction pipeline that can detect and localize temporal modifications.
4. Empirical validation of high imperceptibility, strong robustness, and low computational overhead on modern video diffusion models.

In summary, SPDMark offers a new paradigm for video watermarking that leverages parameter‑level modifications rather than noise‑space perturbations, thereby avoiding costly inversion steps and achieving superior robustness to temporal attacks. Its low‑rank, dictionary‑based design makes it scalable to large models and suitable for real‑time streaming applications, marking a significant step forward in protecting the provenance of AI‑generated video content.