A Multi-objective Evolutionary Algorithm Based on Bi-population with Uniform Sampling for Neural Architecture Search

Neural architecture search (NAS) automates neural network design, improving efficiency over manual approaches. However, efficiently discovering high-performance neural network architectures that simultaneously optimize multiple objectives remains a significant challenge in NAS. Existing methods often suffer from limited population diversity and inadequate exploration of the search space, particularly in regions with extreme complexity values. To address these challenges, we propose MOEA-BUS, an innovative multi-objective evolutionary algorithm based on bi-population with uniform sampling for neural architecture search, aimed at simultaneously optimizing both accuracy and network complexity. In MOEA-BUS, a novel uniform sampling method is proposed to initialize the population, ensuring that architectures are distributed uniformly across the objective space. Furthermore, to enhance exploration, we deploy a bi-population framework where two populations evolve synergistically, facilitating comprehensive search space coverage. Experiments on CIFAR-10 and ImageNet demonstrate MOEA-BUS’s superiority, achieving top-1 accuracies of 98.39% on CIFAR-10, and 80.03% on ImageNet. Notably, it achieves 78.28% accuracy on ImageNet with only 446M MAdds. Ablation studies confirm that both uniform sampling and bi-population mechanisms enhance population diversity and performance. Additionally, in terms of the Kendall’s tau coefficient, the SVM achieves an improvement of at least 0.035 compared to the other three commonly used machine learning models, and uniform sampling provided an enhancement of approximately 0.07.

💡 Research Summary

The paper introduces MOEA‑BUS, a novel multi‑objective evolutionary algorithm designed specifically for neural architecture search (NAS) that simultaneously optimizes classification accuracy and computational complexity measured in multiply‑accumulate operations (MAdds). The authors identify two major shortcomings in existing NAS approaches: (1) insufficient population diversity caused by random initialization that tends to cluster architectures around medium‑sized models, leaving the extreme low‑ and high‑complexity regions under‑explored; and (2) limited exploration capability of single‑population or naïve multi‑population schemes, which can either converge prematurely or homogenize the gene pool when individuals migrate between populations.

To address these issues, MOEA‑BUS incorporates (a) a uniform sampling strategy for population initialization and (b) a bi‑population framework with controlled elite exchange. The uniform sampling method works in two stages. First, the objective space (accuracy vs. MAdds) is discretized into a grid, and at least one individual is placed in each cell, guaranteeing a roughly even spread across the entire complexity spectrum, including the extreme ends. Second, within each cell, random mutations are applied to generate diverse candidates while preserving the uniform distribution. This approach mitigates the bias of conventional random seeding and ensures that the evolutionary process starts with a well‑balanced representation of the search space.

The bi‑population mechanism maintains two sub‑populations that evolve largely independently. At predefined intervals, elite individuals from each sub‑population are exchanged based on a combined criterion of fitness and diversity, preventing the gene pool from collapsing into a single mode while still allowing high‑quality genetic material to propagate. This dual‑population design expands the explored region early on and accelerates convergence later, effectively balancing exploration and exploitation.

To further reduce the prohibitive cost of evaluating thousands of candidate architectures, the authors integrate a surrogate model (trained on a modest set of evaluated architectures) and weight inheritance from a super‑network. The surrogate predicts accuracy without full training, while weight inheritance initializes candidate networks with pre‑trained parameters, cutting down the required fine‑tuning epochs.

Experimental validation is performed on CIFAR‑10, CIFAR‑100, and ImageNet. On CIFAR‑10, MOEA‑BUS achieves a top‑1 accuracy of 98.39 %, surpassing state‑of‑the‑art NAS methods by roughly 0.5 %. On ImageNet, the algorithm reaches 80.03 % top‑1 accuracy, and notably attains 78.28 % accuracy with a lightweight model of only 446 M MAdds, demonstrating its ability to find high‑performing, low‑complexity architectures.

Ablation studies isolate the contributions of each component. Removing uniform sampling leads to a markedly less diverse initial population and a narrower Pareto front. Replacing the bi‑population with a single population slows convergence and increases the risk of premature stagnation. Excluding the surrogate model forces full training of each candidate, dramatically increasing runtime without improving final performance. Additionally, the authors evaluate four surrogate learning algorithms and report that a support vector machine (SVM) yields the highest Kendall’s τ correlation (at least 0.035 improvement over alternatives), while the uniform sampling itself contributes an extra ≈0.07 increase in τ, confirming its positive impact on surrogate reliability.

Overall, MOEA‑BUS presents a comprehensive solution to the dual challenges of diversity preservation and evaluation efficiency in multi‑objective NAS. By guaranteeing a uniform spread of initial solutions and fostering cooperative yet distinct evolutionary streams, the method uncovers a richer set of Pareto‑optimal architectures than prior approaches. The integration of surrogate prediction and weight inheritance further makes the framework practical for large‑scale datasets. The paper suggests future extensions such as incorporating additional objectives (energy consumption, memory footprint) and closed‑loop hardware‑in‑the‑loop optimization, indicating a promising direction for scalable, resource‑aware neural architecture design.