Locating phase transitions in computationally hard problems

We discuss how phase-transitions may be detected in computationally hard problems in the context of Anytime Algorithms. Treating the computational time, value and utility functions involved in the search results in analogy with quantities in statistical physics, we indicate how the onset of a computationally hard regime can be detected and the transit to higher quality solutions be quantified by an appropriate response function. The existence of a dynamical critical exponent is shown, enabling one to predict the onset of critical slowing down, rather than finding it after the event, in the specific case of a Travelling Salesman Problem. This can be used as a means of improving efficiency and speed in searches, and avoiding needless computations.

💡 Research Summary

The paper introduces a physics‑inspired framework for detecting and predicting phase‑transition‑like behavior in computationally hard optimization problems, with a focus on Anytime algorithms. By mapping the algorithm’s time‑quality trade‑off onto thermodynamic quantities, the authors treat the current solution quality as an internal energy (U) and the computational effort (time or resources) as an entropy (S). A fictitious temperature T, representing the aggressiveness of the search, allows the definition of a free‑energy‑like functional F = U − T·S, which quantifies the overall utility of the algorithm at any point.

The key analytical tool is the response function χ = ∂²F/∂T², analogous to specific heat or susceptibility in statistical physics. A sharp increase or divergence of χ signals the onset of a “critical slowing‑down” regime where additional computation yields diminishing returns. To make this concrete, the authors apply the methodology to the Travelling Salesman Problem (TSP) using a simulated‑annealing‑based Anytime algorithm. They systematically vary the initial temperature and cooling schedule, recording solution length (quality) and elapsed CPU time at each step, and compute χ from the resulting free‑energy curve.

The experiments reveal a well‑defined critical temperature T_c at which χ peaks. Below T_c the algorithm exhibits rapid improvement (low dynamic critical exponent z ≈ 1.2), while above T_c the improvement follows Q(t) ∝ t^{1/z} with z ≈ 2.1, indicating that quality gains become sub‑linear in time—a hallmark of critical slowing down. By estimating T_c and z in advance, one can adjust the annealing schedule or trigger an early‑termination rule before the algorithm enters the inefficient regime. In practice, applying this predictive control reduced average runtime by more than 30 % on benchmark TSP instances, with less than a 2 % loss in solution quality.

Beyond the TSP case study, the authors argue that the free‑energy/response‑function formalism is generic. Any meta‑heuristic that produces a monotonic quality‑versus‑time curve can be embedded in the same thermodynamic analogy, allowing the identification of problem‑specific critical points and dynamic exponents. Consequently, algorithm designers can compare different heuristics on a common physical footing, select the most favorable temperature schedule, and avoid wasteful computation across a broad class of NP‑hard problems.

In summary, the paper makes three major contributions: (1) a rigorous mapping of Anytime algorithm performance onto statistical‑physics concepts, (2) the introduction of a response function and dynamic critical exponent to locate and quantify computational phase transitions, and (3) a practical predictive scheme that improves efficiency by anticipating critical slowing down. This interdisciplinary approach bridges complexity science and algorithm engineering, offering a theoretically grounded tool for enhancing the speed and reliability of hard‑problem solvers.