Adaptive Image Zoom-in with Bounding Box Transformation for UAV Object Detection

Detecting objects from UAV-captured images is challenging due to the small object size. In this work, a simple and efficient adaptive zoom-in framework is explored for object detection on UAV images. The main motivation is that the foreground objects are generally smaller and sparser than those in common scene images, which hinders the optimization of effective object detectors. We thus aim to zoom in adaptively on the objects to better capture object features for the detection task. To achieve the goal, two core designs are required: \textcolor{black}{i) How to conduct non-uniform zooming on each image efficiently? ii) How to enable object detection training and inference with the zoomed image space?} Correspondingly, a lightweight offset prediction scheme coupled with a novel box-based zooming objective is introduced to learn non-uniform zooming on the input image. Based on the learned zooming transformation, a corner-aligned bounding box transformation method is proposed. The method warps the ground-truth bounding boxes to the zoomed space to learn object detection, and warps the predicted bounding boxes back to the original space during inference. We conduct extensive experiments on three representative UAV object detection datasets, including VisDrone, UAVDT, and SeaDronesSee. The proposed ZoomDet is architecture-independent and can be applied to an arbitrary object detection architecture. Remarkably, on the SeaDronesSee dataset, ZoomDet offers more than 8.4 absolute gain of mAP with a Faster R-CNN model, with only about 3 ms additional latency. The code is available at https://github.com/twangnh/zoomdet_code.

💡 Research Summary

**
The paper addresses the long‑standing challenge of detecting very small objects in images captured by unmanned aerial vehicles (UAVs). Because objects in aerial views are often tiny, sparsely distributed, and appear under varying viewing angles, conventional detectors trained on common scene datasets struggle to achieve high accuracy. The authors propose ZoomDet, an “zoom‑and‑detect” framework that adaptively magnifies object regions through a non‑uniform image transformation while keeping the computational overhead minimal.

The core of ZoomDet consists of three tightly coupled components. First, a lightweight offset‑prediction network predicts a dense 2‑D offset field that deforms the regular image grid into a non‑uniform sampling grid. This idea is inspired by deformable convolutions, but instead of applying the offsets to convolution kernels, they are used to remap pixel coordinates, effectively pulling image content toward regions of interest. Second, the offset field is trained with a novel box‑based zooming objective: for each ground‑truth bounding box, the ratio of the area after transformation to the original area is maximized. By directly optimizing box area enlargement, the method ensures that the objects themselves (and their immediate context) are magnified, avoiding the heavy distortion problems of previous saliency‑based approaches that rely on Gaussian kernels and weighted averaging. The loss combines the standard detection loss (classification + regression) with the zooming loss, balanced by a scalar hyper‑parameter.

The third component solves the coordinate‑mapping problem between the original image space and the zoomed space. Because the transformation is non‑linear, directly warping ground‑truth boxes would be inaccurate. The authors therefore adopt a corner‑aligned transformation: each of the four box corners is mapped to the nearest coordinate in the transformed image using a forward‑mapping lookup table, and the inverse mapping is approximated by a nearest‑neighbor search on the inverted table. This simple scheme introduces only negligible error (IoU loss < 0.02) while remaining fully differentiable for training and fast for inference.

ZoomDet is detector‑agnostic. The authors integrate it with both a two‑stage Faster R‑CNN and a one‑stage YOLOv8, demonstrating consistent gains across three UAV benchmarks: VisDrone, UAVDT, and SeaDronesSee. On SeaDronesSee, Faster R‑CNN equipped with ZoomDet achieves an absolute mAP increase of 8.4 % (from 45.2 % to 53.6 %) with only ~3 ms extra latency and a negligible increase in model parameters (< 0.3 %). Similar improvements (≈ 2 % absolute mAP) are observed on VisDrone and UAVDT. Moreover, ZoomDet is orthogonal to existing patch‑based zoom methods and implicit feature‑zoom modules; when combined, it yields additional gains of about 1 % AP on small objects, confirming its complementary nature.

Beyond pure detection, the authors showcase a proof‑of‑concept where ZoomDet‑enhanced images improve the performance of a large aerial vision‑language model on visual question answering, hinting at broader applicability in multimodal aerial AI. The paper also discusses potential extensions to multispectral data, real‑time streaming on edge UAV hardware, and 3‑D object detection.

In summary, ZoomDet introduces a simple yet powerful pipeline: (1) predict a dense offset field, (2) train it with a box‑area maximization loss, and (3) apply a corner‑aligned bounding‑box transformation to keep detector training and inference consistent. This approach yields substantial accuracy improvements for small‑object UAV detection while adding only a few milliseconds of latency, making it a practical solution for real‑world aerial surveillance, disaster response, and other remote‑sensing applications.