Graph Neural Networks are Heuristics

We demonstrate that a single training trajectory can transform a graph neural network into an unsupervised heuristic for combinatorial optimization. Focusing on the Travelling Salesman Problem, we show that encoding global structural constraints as an inductive bias enables a non-autoregressive model to generate solutions via direct forward passes, without search, supervision, or sequential decision-making. At inference time, dropout and snapshot ensembling allow a single model to act as an implicit ensemble, reducing optimality gaps through increased solution diversity. Our results establish that graph neural networks do not require supervised training nor explicit search to be effective. Instead, they can internalize global combinatorial structure and function as strong, learned heuristics. This reframes the role of learning in combinatorial optimization: from augmenting classical algorithms to directly instantiating new heuristics.

💡 Research Summary

The paper “Graph Neural Networks are Heuristics” puts forward a novel perspective on using graph neural networks (GNNs) for combinatorial optimization, specifically the Travelling Salesman Problem (TSP). The authors argue that a single training trajectory, without any supervision or explicit search, can turn a GNN into a powerful heuristic. Their approach hinges on three main ideas.

First, they encode the global structure of TSP directly into the learning objective. By representing a tour as a permutation matrix (P) and using a cyclic shift matrix (V), a Hamiltonian cycle can be expressed as (H = P V P^\top). The loss is defined as the inner product (\langle D, TVT^\top\rangle) where (D) is the distance matrix and (T) is a soft permutation obtained via the Gumbel‑Sinkhorn relaxation. This formulation forces the network to consider the entire tour simultaneously, rather than constructing it step‑by‑step as in autoregressive decoders.

Second, the authors introduce a symmetry‑aware feature extractor. Input coordinates are centered, and a data‑dependent canonical frame is built from the covariance eigenvectors. Intrinsic polar coordinates and a set of Fourier harmonics are then computed, yielding node features that are invariant to global translation and rotation (almost everywhere) and equivariant to permutations. This ensures that the GNN processes only the intrinsic structure of the instance.

Third, they propose two inexpensive ensemble mechanisms that require no extra training. During inference, dropout masks are kept active, allowing Monte‑Carlo sampling of multiple stochastic forward passes from a single trained model. In addition, they save several checkpoints (snapshots) along one training run and combine the predictions of these models at test time. Both techniques act as implicit ensembles, increasing solution diversity and reducing the optimality gap.

The network architecture, called a scattering‑attention GNN (SCT‑GNN), mixes multi‑scale diffusion operators (graph convolutions and scattering transforms) with learned attention weights. Dropout is applied inside the attention and feed‑forward layers, injecting stochasticity without altering the final discrete tour extraction.

Experimental results on Euclidean TSP instances of varying sizes show that the unsupervised GNN consistently outperforms classic greedy heuristics and prior GNN‑based methods that rely on supervision or reinforcement learning. The dropout‑only ensemble already narrows the optimality gap by a few percent; adding snapshot ensembling yields further improvements, achieving gaps that are often more than 10 % lower than the best greedy baselines. Moreover, the method requires far fewer training epochs and less memory than autoregressive reinforcement‑learning approaches.

The authors also critique recent works that claim GNNs cannot beat greedy heuristics, pointing out that those studies focus on locally favorable instances where greedy already performs well. By embedding the global Hamiltonian‑cycle constraint into the loss, the proposed model demonstrates that GNNs can indeed capture non‑local structure and act as genuine heuristics.

Limitations are acknowledged: the current formulation assumes symmetric Euclidean distances, and extending it to asymmetric or non‑metric TSP variants would require additional work. The snapshot ensemble does incur storage overhead, though it is modest compared to training multiple independent models.

In conclusion, the paper establishes that graph neural networks need not be paired with supervised signals or explicit search procedures to be effective. By leveraging global structural priors, symmetry‑aware features, and cheap stochastic ensembling, a single‑pass GNN can serve as a strong, learned heuristic for combinatorial problems, opening a new direction for research in learning‑driven optimization.