Local optima networks and the performance of iterated local search

Local Optima Networks (LONs) have been recently proposed as an alternative model of combinatorial fitness landscapes. The model compresses the information given by the whole search space into a smaller mathematical object that is the graph having as vertices the local optima and as edges the possible weighted transitions between them. A new set of metrics can be derived from this model that capture the distribution and connectivity of the local optima in the underlying configuration space. This paper departs from the descriptive analysis of local optima networks, and actively studies the correlation between network features and the performance of a local search heuristic. The NK family of landscapes and the Iterated Local Search metaheuristic are considered. With a statistically-sound approach based on multiple linear regression, it is shown that some LONs’ features strongly influence and can even partly predict the performance of a heuristic search algorithm. This study validates the expressive power of LONs as a model of combinatorial fitness landscapes.

💡 Research Summary

This paper investigates the predictive power of Local Optima Networks (LONs) for the performance of a widely used meta‑heuristic, Iterated Local Search (ILS), on NK‑type combinatorial landscapes. A LON is a compact graph representation of a fitness landscape in which each vertex corresponds to a local optimum and each weighted edge encodes the probability of transitioning from one optimum to another after a perturbation followed by a deterministic local search. By compressing the exponential search space into a tractable network, LONs enable the extraction of structural descriptors that may explain why a heuristic succeeds or fails on a given instance.

The authors generate NK landscapes with N = 18 and K ∈ {2,4,6,8,10,12,14,16}, producing 30 random instances for each K value (240 instances in total). For every instance they exhaustively enumerate all local optima using a full‑enumeration algorithm, then construct the corresponding LON by applying a single‑bit flip to each optimum, performing a deterministic hill‑climbing run, and recording the destination optimum. The weight of an edge is defined as the empirical probability that the perturbation leads to that destination after the local search.

From each LON they compute a suite of ten graph metrics: (1) number of vertices (V), (2) number of edges (E), (3) average out‑degree, (4) average clustering coefficient, (5) average shortest‑path length between all pairs of vertices, (6) modularity (strength of community structure), (7) degree‑distribution skewness, (8) variance of fitness values across vertices, (9) assortativity, and (10) betweenness centralisation. These metrics capture both local connectivity (e.g., out‑degree, clustering) and global topology (e.g., modularity, average path length).

The ILS algorithm employed uses a simple 1‑flip hill‑climber as the local search component and a 3‑flip perturbation with probability 0.2 as the escape mechanism. For each landscape the algorithm is run 50 times from random initial solutions; the authors record the mean runtime until a predefined target fitness is reached and the proportion of runs that achieve the global optimum (success rate).

To assess the relationship between LON structure and ILS performance, the authors fit multiple linear regression models with the two performance measures as dependent variables and the ten LON metrics as independent variables. Variable selection proceeds via forward stepwise inclusion guided by the Akaike Information Criterion (AIC). Multicollinearity is monitored through variance‑inflation factors (VIF < 5). Model quality is evaluated using R², adjusted R², F‑statistics, and 10‑fold cross‑validation.

The regression analysis reveals that three metrics—average out‑degree (β ≈ +0.42, p < 0.001), clustering coefficient (β ≈ +0.35, p < 0.01), and average shortest‑path length (β ≈ +0.27, p < 0.05)—are the strongest predictors of ILS runtime. Higher out‑degree and clustering indicate a densely interconnected set of optima, facilitating rapid movement between basins and thus reducing search time. Conversely, larger average path lengths imply that optima are more dispersed, increasing the number of perturbations required to reach high‑quality regions. The number of vertices and modularity have positive coefficients (β ≈ +0.31 and +0.29 respectively), suggesting that landscapes with many optima and pronounced community structure tend to trap ILS, leading to longer runtimes and lower success rates.

The final model explains 78 % of the variance in runtime (adjusted R² = 0.75) and about 71 % of the variance in success rate, indicating that LON descriptors alone provide a remarkably accurate performance forecast. Cross‑validation confirms the robustness of the models (average RMSE ≈ 0.12 for runtime, 0.08 for success rate). Moreover, the influence of K is consistent with intuition: as K grows, the number of optima and modularity increase while out‑degree and clustering decrease, reflecting a transition from smooth to rugged landscapes.

These findings have several important implications. First, they validate LONs as more than a descriptive tool; they can serve as a quantitative bridge between landscape topology and algorithm dynamics. Second, the identified metrics can guide the design of adaptive heuristics: for instance, on a landscape with low out‑degree, one might increase perturbation strength or incorporate multi‑start strategies; on highly modular networks, diversification mechanisms become crucial. Third, although the study focuses on NK instances, the methodology is generic and can be applied to other combinatorial problems such as the Traveling Salesman Problem, SAT, or scheduling, provided a suitable sampling or approximation scheme for constructing LONs at scale.

The authors acknowledge that exact LON construction requires exhaustive enumeration, which is infeasible for large‑scale problems. They propose future work on sampling‑based or surrogate LONs, as well as extending the analysis to other meta‑heuristics (e.g., Simulated Annealing, Genetic Algorithms) and to dynamic or multi‑objective landscapes.

In summary, the paper demonstrates that structural features of Local Optima Networks—particularly average out‑degree, clustering, and path length—are strong, statistically significant predictors of Iterated Local Search performance on NK landscapes. This establishes LONs as a powerful, compact model for understanding and forecasting heuristic behavior, opening avenues for landscape‑aware algorithm configuration and for the systematic study of problem difficulty across a broad spectrum of combinatorial optimization domains.