A Methodology for Effective Surrogate Learning in Complex Optimization

Solving complex problems requires continuous effort in developing theory and practice to cope with larger, more difficult scenarios. Working with surrogates is normal for creating a proxy that realistically models the problem into the computer. Thus, the question of how to best define and characterize such a surrogate model is of the utmost importance. In this paper, we introduce the PTME methodology to study deep learning surrogates by analyzing their Precision, Time, Memory, and Energy consumption. We argue that only a combination of numerical and physical performance can lead to a surrogate that is both a trusted scientific substitute for the real problem and an efficient experimental artifact for scalable studies. Here, we propose different surrogates for a real problem in optimally organizing the network of traffic lights in European cities and perform a PTME study on the surrogates’ sampling methods, dataset sizes, and resource consumption. We further use the built surrogates in new optimization metaheuristics for decision-making in real cities. We offer better techniques and conclude that the PTME methodology can be used as a guideline for other applications and solvers.

💡 Research Summary

The paper introduces PTME, a comprehensive evaluation framework for surrogate models that simultaneously measures four critical dimensions: Precision, Time, Memory, and Energy. While most existing surrogate studies focus only on predictive error or runtime, PTME insists that a surrogate must also be assessed for its physical resource consumption to be truly useful in real-world optimization pipelines.

The authors first define the PTME dimensions in detail. Precision is quantified using Mean Absolute Percentage Error (MAPE), Root Mean Squared Error (RMSE), and two variants of Kendall’s τ to capture both absolute prediction quality and ranking consistency—essential for meta‑heuristics that rely on ordering candidate solutions. Time is split into training time and inference time, reflecting the cost of building the surrogate and the speed at which it can evaluate new candidates. Memory is measured by model size, peak RAM usage, and activation footprint during inference, which is especially relevant for deep neural networks. Energy consumption is captured via Intel’s Running Average Power Limit (RAPL) counters, providing a direct measurement of power draw for both training and inference phases.

To ensure reproducibility, the paper presents a standardized measurement protocol (Algorithm 1). It repeatedly samples training data, trains the surrogate while logging CPU cycles, DRAM energy, elapsed time, and memory usage, then evaluates the trained model on a fixed test set, recording the same metrics. Statistics are aggregated over multiple runs (T = 10) to obtain robust estimates.

The methodology is demonstrated on a realistic case study: optimizing traffic‑light schedules in three European cities (Málaga, Stockholm, Paris). The underlying problem is a high‑dimensional (190–378 variables) multi‑objective optimization that combines vehicle throughput, travel time, waiting time, and green‑to‑red signal ratios into a single scalar objective. Evaluating a candidate solution requires running the SUMO microscopic traffic simulator, which can take from seconds to tens of minutes per run, making surrogate modeling highly attractive.

Two sampling strategies for generating training data are compared: Uniform Random Sampling (URS) and Latin Hypercube Sampling (LHS). Both are used to create datasets of 5 k, 10 k, and 20 k samples. The surrogate architecture is a two‑layer feed‑forward neural network whose hidden layers are sized proportionally to the input dimension (1.5× inputs for the first layer, inputs for the second). ReLU activation, Adam optimizer, 100 epochs, batch size 32, and a learning rate of 1e‑4 constitute the training regime.

Experimental results show that LHS consistently yields better precision: MAPE improves by roughly 12–18 % and RMSE shows similar gains compared to URS. Kendall’s τ also rises by 0.05–0.08, indicating more reliable ranking of solutions. The trade‑off is a modest increase (≈7 %) in training time and energy due to the more complex sampling process, while inference costs remain essentially unchanged. Scaling the dataset size improves accuracy but linearly increases memory footprint and power consumption; beyond 10 k samples the marginal precision gain diminishes, suggesting an optimal balance point for real‑time applications.

Finally, the authors embed the trained surrogates into meta‑heuristic solvers (genetic algorithms, particle swarm, etc.). The surrogate‑assisted optimization reduces total runtime by 65–78 % relative to direct simulation, while the final objective values deviate by only 3–5 % from the simulation‑only baseline. This demonstrates that PTME‑guided surrogate selection can achieve substantial computational savings without sacrificing solution quality.

The paper concludes that PTME provides a unified, reproducible framework applicable beyond traffic control to any domain where expensive simulations are used (e.g., power grid management, logistics, biomedical simulations). By jointly considering numerical accuracy and physical resource metrics, PTME helps researchers and practitioners design surrogates that are both scientifically trustworthy and operationally efficient.