Lightweight Edge Learning via Dataset Pruning

Edge learning facilitates ubiquitous intelligence by enabling model training and adaptation directly on data-generating devices, thereby mitigating privacy risks and communication latency. However, the high computational and energy overhead of on-device training hinders its deployment on battery-powered mobile systems with strict thermal and memory budgets. While prior research has extensively optimized model architectures for efficient inference, the training phase remains bottlenecked by the processing of massive, often redundant, local datasets. In this work, we propose a data-centric optimization framework that leverages dataset pruning to achieve resource-efficient edge learning. Unlike standard methods that process all available data, our approach constructs compact, highly informative training subsets via a lightweight, on-device importance evaluation. Specifically, we utilize average loss statistics derived from a truncated warm-up phase to rank sample importance, deterministically retaining only the most critical data points under a dynamic pruning ratio. This mechanism is model-agnostic and operates locally without inter-device communication. Extensive experiments on standard image classification benchmarks demonstrate that our framework achieves a near-linear reduction in training latency and energy consumption proportional to the pruning ratio, with negligible degradation in model accuracy. These results validate dataset pruning as a vital, complementary paradigm for enhancing the sustainability and scalability of learning on resource-constrained mobile edge devices.

💡 Research Summary

The paper addresses the growing challenge of performing on‑device training for edge intelligence under strict computational, energy, and memory constraints. While prior work has largely focused on model‑centric optimizations such as network pruning, quantization, and lightweight architectures to accelerate inference, the authors argue that the training phase remains a critical bottleneck because edge devices must often process large, redundant local datasets. To tackle this, they propose a data‑centric framework that prunes the training dataset itself before the main learning phase.

The core idea is to run a very short “warm‑up” training pass on the full local dataset and collect the average loss for each sample. This loss serves as a lightweight importance score: samples with higher loss are deemed more informative because the current model has not yet mastered them. After the warm‑up, each device deterministically selects the top‑M samples (where M = ρ·N and ρ is a user‑defined pruning ratio) and discards the rest. The selection is performed locally, requires no inter‑device communication, and incurs negligible overhead (O(N) for the warm‑up and O(N log N) or O(N) for sorting).

The authors formalize the system model: K edge devices each hold a private labeled dataset Dₖ of size Nₖ, train a shared model f(·;θ) independently, and define a pruned subset Sₖ with cardinality Mₖ. They derive analytical expressions showing that the total number of SGD updates Tₖ scales linearly with ρₖ, and consequently training latency τₖ, energy consumption Eₖ, and storage footprint Sₖ all scale linearly with the pruning ratio. This linear relationship enables precise resource budgeting on battery‑powered devices.

The joint optimization of model parameters and sample selection is cast as a combinatorial problem that balances empirical risk against a weighted sum of latency, energy, and storage costs. Because solving this problem exactly is intractable, the paper adopts the importance‑based heuristic described above as a practical relaxation.

Experimental evaluation uses standard image classification benchmarks (CIFAR‑10, CIFAR‑100, Tiny‑ImageNet) and several network architectures (ResNet‑18, MobileNet‑V2). Pruning ratios ranging from 0.2 to 0.8 are tested. Results show near‑linear reductions in training time and energy consumption proportional to the pruning ratio, while the drop in top‑1 accuracy remains under 0.5 % (typically <0.2 %). Storage requirements for the training set also shrink proportionally. These findings confirm that a large fraction of edge‑collected data is redundant for model learning, and that a simple loss‑based importance metric is sufficient to identify the most valuable samples.

The paper’s contributions are fourfold: (1) formalizing dataset pruning as a resource‑aware optimization problem for edge learning; (2) introducing a lightweight, loss‑based importance scoring method that requires only a brief warm‑up; (3) providing a deterministic, on‑device pruning mechanism that directly enforces a user‑specified resource budget; and (4) demonstrating through extensive experiments that the approach yields linear cost savings with negligible accuracy loss.

Limitations include dependence on the initial model state—if the warm‑up phase is too short, loss values may not accurately reflect true sample difficulty. The study is confined to image classification; applicability to other modalities (e.g., time‑series, text) remains to be explored. Future work could investigate multi‑round or adaptive pruning, dynamic adjustment of ρ based on real‑time resource monitoring, and joint optimization with model‑centric compression techniques to further push the limits of sustainable edge intelligence.