Motion Deblurring with an Adaptive Network

In this paper, we address the problem of dynamic scene deblurring in the presence of motion blur. Restoration of images affected by severe blur necessitates a network design with a large receptive field, which existing networks attempt to achieve through simple increment in the number of generic convolution layers, kernel-size, or the scales at which the image is processed. However, increasing the network capacity in this manner comes at the expense of increase in model size and inference speed, and ignoring the non-uniform nature of blur. We present a new architecture composed of spatially adaptive residual learning modules that implicitly discover the spatially varying shifts responsible for non-uniform blur in the input image and learn to modulate the filters. This capability is complemented by a self-attentive module which captures non-local relationships among the intermediate features and enhances the receptive field. We then incorporate a spatiotemporal recurrent module in the design to also facilitate efficient video deblurring. Our networks can implicitly model the spatially-varying deblurring process, while dispensing with multi-scale processing and large filters entirely. Extensive qualitative and quantitative comparisons with prior art on benchmark dynamic scene deblurring datasets clearly demonstrate the superiority of the proposed networks via reduction in model-size and significant improvements in accuracy and speed, enabling almost real-time deblurring.

💡 Research Summary

The paper tackles the challenging problem of dynamic‑scene motion deblurring, where blur is spatially non‑uniform due to moving objects, camera shake, and depth variations. Traditional deep‑learning deblurring methods obtain a large receptive field by simply stacking many layers, using large kernels, or processing images at multiple scales. While effective, these strategies dramatically increase model size, computational cost, and inference latency, and they ignore the fact that blur varies across the image.

To address these issues, the authors propose a compact yet powerful architecture called the Spatially‑Adaptive Residual Network (SARN). The core ideas are:

1. Deformable Residual Module (DRM) – a residual block that incorporates deformable convolutions. For each spatial location the module predicts a dense 2‑D offset map, which shifts the sampling grid of the convolutional kernel. This enables the network to learn locally adaptive, asymmetric filters that align with the direction and magnitude of motion blur. Offsets are initialized to zero and learned end‑to‑end via bilinear interpolation, preserving differentiability.

2. Self‑Attention Module – placed after the encoder, this module computes a global attention map over the intermediate feature maps (query, key, value). It captures long‑range, non‑local dependencies that a limited receptive field cannot see, which is especially beneficial for large motions and complex textures.

3. Spatio‑Temporal Recurrent Module – for video deblurring, the single‑image SARN is extended with recurrent connections both at the frame level (hidden state propagation across time) and at the feature level (temporal aggregation of encoder outputs). This design avoids explicit optical‑flow alignment while still exploiting temporal redundancy, allowing real‑time processing of video streams.

The authors provide a theoretical justification: modeling motion blur as a 2‑D infinite‑impulse‑response (IIR) system shows that the inverse filter required for deconvolution is typically much larger than the blur kernel and is directionally biased. Hence, learning adaptive, directional filters is a natural fit.

Experiments are conducted on several benchmark datasets (GoPro, DVD, REDS). The proposed network uses only ~1.2 M parameters (compared to >7 M in many state‑of‑the‑art models) and 3×3 kernels throughout, eliminating multi‑scale branches. Quantitatively, it achieves higher PSNR/SSIM than the current best methods (e.g., SRN, DV​D, OVD) while running at >30 FPS on an RTX 2080 Ti, i.e., near real‑time. Ablation studies confirm that removing DRM, the self‑attention block, or the temporal recurrence each leads to a noticeable drop in performance, underscoring their complementary contributions.

Limitations are acknowledged: DRM offsets can become unstable if not regularized, and extremely severe blur (e.g., >100 px motion) still challenges the model. Future work may explore offset regularization, multi‑scale attention, or hybrid explicit kernel estimation.

In summary, the paper introduces a novel deblurring paradigm that replaces brute‑force network scaling with input‑adaptive filtering and global attention, delivering a lightweight model that excels in both accuracy and speed for single‑image and video motion deblurring.