Low-Resolution Action Recognition for Tiny Actions Challenge

Tiny Actions Challenge focuses on understanding human activities in real-world surveillance. Basically, there are two main difficulties for activity recognition in this scenario. First, human activities are often recorded at a distance, and appear in a small resolution without much discriminative clue. Second, these activities are naturally distributed in a long-tailed way. It is hard to alleviate data bias for such heavy category imbalance. To tackle these problems, we propose a comprehensive recognition solution in this paper. First, we train video backbones with data balance, in order to alleviate overfitting in the challenge benchmark. Second, we design a dual-resolution distillation framework, which can effectively guide low-resolution action recognition by super-resolution knowledge. Finally, we apply model en-semble with post-processing, which can further boost per-formance on the long-tailed categories. Our solution ranks Top-1 on the leaderboard.

💡 Research Summary

The paper addresses the challenging problem of recognizing human actions in low‑resolution surveillance videos, as posed by the Tiny Actions Challenge. Two fundamental difficulties are identified: (1) the visual details of actions are severely degraded because subjects are captured from a distance, resulting in small, blurry frames; and (2) the dataset exhibits a long‑tailed distribution, with a few categories containing many examples while most categories have very few, leading to severe class imbalance. To overcome these issues, the authors propose a three‑stage solution: (i) training robust video backbones with explicit data‑balancing techniques, (ii) a dual‑resolution knowledge distillation framework that transfers fine‑grained cues from super‑resolution (SR) videos to low‑resolution (LR) models, and (iii) a large‑scale model ensemble combined with post‑processing that adapts decision thresholds per class and applies group‑wise filtering.

Backbone selection and data balancing: The authors choose two strong video encoders—ir‑CSN‑ResNet152 and UniFormer‑Base—both pretrained on Kinetics‑400. For LR videos they uniformly split each clip into 16 segments and randomly sample one frame per segment, preserving long‑range temporal context while limiting computational cost. To mitigate the long‑tail distribution, they augment minority‑class videos with horizontal flips, effectively doubling the training samples for those categories. This simple balancing yields noticeable gains: the F1 score of ir‑CSN improves from 0.403 (uniform sampling) to 0.469 after data balancing, and UniFormer rises from 0.404 to 0.452.

Dual‑resolution distillation: Since no ground‑truth high‑resolution counterpart exists for each LR video, the authors employ RealBasicVSR, a state‑of‑the‑art video super‑resolution model, to generate SR versions (224 × 224) of all training clips. The SR videos inherit the original LR labels, allowing the authors to train a “teacher” network on SR data. For each LR sample, the corresponding SR video is passed through the teacher to obtain a soft prediction vector k (knowledge). The LR “student” network is then trained with a combined loss: binary cross‑entropy (L_bce) against the ground‑truth label and a mean‑squared‑error knowledge‑distillation term (L_kd) against k. The total loss L_total = α L_bce + (1 − α) L_kd balances label supervision and teacher guidance. Empirically, adding knowledge distillation (SR + KD) raises ir‑CSN’s F1 from 0.484 (SR only) to 0.492, confirming that SR‑derived cues improve LR recognition.

Model ensemble and post‑processing: The authors collect 12 checkpoints from different training epochs and architectures (e.g., four ir‑CSN models, two UniFormer models, and several SR/KD variants). They fuse predictions via weighted averaging. Recognizing that long‑tailed categories require different operating points, they set class‑specific thresholds: high‑frequency classes receive higher thresholds to reduce false positives, while low‑frequency classes receive lower thresholds to boost recall. Additionally, they exploit prior knowledge of activity groups; within each group, only the class with the highest score is retained, reducing inter‑class confusion. This ensemble plus post‑processing pipeline achieves a final F1 score of 0.883, securing the top position on the challenge leaderboard.

Training details: The multi‑label nature of the task leads the authors to use binary cross‑entropy loss, augmented with Asymmetric Loss (a focal‑type loss) to further address imbalance. Optimization is performed with AdamW, employing a warm‑up phase followed by cosine annealing with restarts. Learning rates are set to 2 × 10⁻⁴ for UniFormer and 1 × 10⁻⁴ for ir‑CSN; dropout rates of 0.5 (ir‑CSN) and a drop‑path rate of 0.4 (UniFormer) are applied to curb overfitting.

In conclusion, the paper presents a comprehensive, well‑engineered pipeline that synergistically combines data balancing, super‑resolution knowledge distillation, and sophisticated ensemble/post‑processing to tackle low‑resolution, long‑tailed action recognition. The reported gains demonstrate that each component contributes meaningfully, and the overall framework offers a practical blueprint for future low‑resolution video understanding tasks.