Automated Model Design using Gated Neuron Selection in Telecom

The telecommunications industry is experiencing rapid growth in adopting deep learning for critical tasks such as traffic prediction, signal strength prediction, and quality of service optimisation. However, designing neural network architectures for these applications remains challenging and time-consuming, particularly when targeting compact models suitable for resource-constrained network environments. Therefore, there is a need for automating the model design process to create high-performing models efficiently. This paper introduces TabGNS (Tabular Gated Neuron Selection), a novel gradient-based Neural Architecture Search (NAS) method specifically tailored for tabular data in telecommunications networks. We evaluate TabGNS across multiple telecommunications and generic tabular datasets, demonstrating improvements in prediction performance while reducing the architecture size by 51-82% and reducing the search time by up to 36x compared to state-of-the-art tabular NAS methods. Integrating TabGNS into the model lifecycle management enables automated design of neural networks throughout the lifecycle, accelerating deployment of ML solutions in telecommunications networks.

💡 Research Summary

The paper addresses the growing need for automated neural‑network design in telecommunications, where most machine‑learning tasks (traffic forecasting, signal‑strength prediction, QoS optimisation) rely on large tabular datasets. Manual architecture engineering is time‑consuming, error‑prone, and ill‑suited for the continuous retraining cycles required by modern model‑life‑cycle‑management (LCM) pipelines. To solve this, the authors introduce Tabular Gated Neuron Selection (TabGNS), a gradient‑based neural‑architecture‑search (NAS) method that works at the neuron level rather than the traditional layer level.

TabGNS builds a “SuperNet” that contains every candidate neuron up to a predefined maximum number of layers L and width W. Each neuron i in layer j is equipped with a learnable gate parameter g(i,j). During training, the gate is transformed into an activation probability p_open(i,j) using a Gumbel‑Softmax distribution combined with a Straight‑Through Estimator (GS‑STE). The probability starts low, keeping most neurons inactive, and is updated jointly with the network weights by back‑propagation. A progressive‑growth schedule is employed: the search begins with a very small network, and as training proceeds neurons whose p_open exceeds a threshold (commonly 0.5) are gradually “opened”. At the end of the search, the binary gates define a compact sub‑network A, which is then fine‑tuned as a standard fully‑connected MLP. Because all candidate neurons share weights inside the SuperNet, the method enjoys massive weight‑sharing benefits that are difficult to achieve with conventional DARTS‑style NAS for fully‑connected layers.

The authors evaluate TabGNS on four real‑world telecom datasets (Vodafone KPI, 5G traffic, LTE RAN logs, signal‑strength measurements) and three generic tabular benchmarks (Adult, Higgs, Click‑Through). Baselines include TabNAS, AgE (the aging‑evolution NAS for tabular data), and DARTS‑based variants. Three metrics are reported: predictive error (MSE or RMSE), model size (number of parameters), and search time (wall‑clock on a single GPU). TabGNS consistently matches or improves predictive performance (0.5 %–3 % lower error) while reducing model size by 51 %–82 % and cutting search time by a factor of 12–36. The most pronounced gains appear on telecom datasets, where the ability to prune unnecessary neurons translates directly into lower latency and memory footprints suitable for edge devices such as base‑station controllers or user equipment.

Key technical contributions are: (1) a neuron‑level gating mechanism that enables fine‑grained, differentiable architecture selection; (2) the use of GS‑STE to keep the search fully gradient‑based, avoiding costly reinforcement‑learning or evolutionary loops; (3) a progressive‑growth strategy that limits early‑stage computation and steers the search toward compact yet high‑capacity models; and (4) demonstration that weight sharing across a SuperNet of fully‑connected layers is feasible and highly efficient.

Limitations acknowledged by the authors include the focus on MLPs only—tree ensembles, gradient‑boosted decision trees, and transformer‑style tabular models are not compared—and the static gate‑threshold used for binarisation, which may not be optimal for multi‑objective constraints (e.g., latency, power). Future work is outlined as extending the gating concept to other layer types, integrating dynamic thresholding or Pareto‑front optimisation for latency‑size trade‑offs, and embedding TabGNS into standardized 3GPP/O‑RAN LCM frameworks for continuous on‑device retraining.

In summary, TabGNS offers a practical, scalable solution for automatically designing compact, high‑performing neural networks on tabular telecom data. By dramatically reducing both model size and NAS search time while preserving or improving accuracy, it paves the way for fully automated model lifecycle management and efficient AI deployment across resource‑constrained telecom infrastructure.