Learning to Execute Graph Algorithms Exactly with Graph Neural Networks

Understanding what graph neural networks can learn, especially their ability to learn to execute algorithms, remains a central theoretical challenge. In this work, we prove exact learnability results for graph algorithms under bounded-degree and finite-precision constraints. Our approach follows a two-step process. First, we train an ensemble of multi-layer perceptrons (MLPs) to execute the local instructions of a single node. Second, during inference, we use the trained MLP ensemble as the update function within a graph neural network (GNN). Leveraging Neural Tangent Kernel (NTK) theory, we show that local instructions can be learned from a small training set, enabling the complete graph algorithm to be executed during inference without error and with high probability. To illustrate the learning power of our setting, we establish a rigorous learnability result for the LOCAL model of distributed computation. We further demonstrate positive learnability results for widely studied algorithms such as message flooding, breadth-first and depth-first search, and Bellman-Ford.

💡 Research Summary

The paper “Learning to Execute Graph Algorithms Exactly with Graph Neural Networks” tackles a fundamental theoretical question: under what conditions can a graph neural network (GNN) learn to execute a graph algorithm without any approximation error? The authors answer this by combining two ideas—local instruction learning with multi‑layer perceptron (MLP) ensembles and a message‑passing GNN architecture that directly mirrors the LOCAL model of distributed computation.

First, they isolate the “local instruction” performed by a node in each round of a distributed algorithm: given the node’s current state and the messages received from its neighbors, the node computes a new state and a message to broadcast. They encode these instructions as binary, block‑structured vectors, separating a computation block from a communication block. An ensemble of K independently initialized MLPs is trained on this non‑graph data to predict the exact output of the instruction. Training uses mean‑squared error against the ground‑truth instruction outputs; because the data are purely local, the training set size grows only linearly with the size of the computation and communication blocks and quadratically with the maximum degree D of the graph.

Second, the trained ensemble is incorporated into a GNN. Input node features are first transformed by a deterministic encoder ΨEnc that orthogonalizes the blocks, then passed through the ensemble average ˆµ. The result is binarized by a Heaviside step function ΨH, yielding a binary vector that is split by two projection masks PC and PM. PC retains the computation part (local state) while PM extracts the communication part (message) that is sent to neighbors via standard message‑passing (multiplication by the adjacency matrix). This construction exactly reproduces the three‑step LOCAL round: (i) local computation, (ii) message broadcast, (iii) neighbor aggregation.

The theoretical core relies on Neural Tangent Kernel (NTK) theory. In the infinite‑width limit, gradient descent on a randomly initialized network follows the linear dynamics dictated by the NTK, and the predictor converges to the NTK regression solution. By choosing the ensemble size K polynomial in D and logarithmic in the number of rounds L and the number of vertices |V|, the ensemble average ˆµ approximates the NTK predictor arbitrarily well. Consequently, the GNN learns the exact mapping from any binary instruction to its correct output with high probability.

The main formal result (Theorem 5.1, informal) states that for any algorithm A expressible in the LOCAL model with bounded per‑node memory, message size, and a fixed number of rounds L on a graph of maximum degree D, there exists a training dataset of size O(state·message·D²) such that the proposed GNN learns to execute A exactly in O(L) GNN iterations. The probability of exact execution can be made arbitrarily close to 1 by scaling K appropriately.

To demonstrate the breadth of the framework, the authors instantiate the theory for several classic graph algorithms:

• Message Flooding – a trivial broadcast where each node forwards a bit to all neighbors.
• Breadth‑First Search (BFS) – nodes maintain distance labels; each round updates a node’s label to one plus the minimum label among its neighbors.
• Depth‑First Search (DFS) – simulated via a stack encoded in binary bits; local updates manipulate the stack pointer and push/pop operations.
• Bellman‑Ford (single‑source shortest paths) – nodes store tentative distances and relax edges by taking the minimum of their own distance and neighbor distance plus edge weight.

In each case the algorithm’s per‑round logic can be expressed as a finite set of binary templates, which the MLP ensemble learns exactly. During inference the GNN iteratively applies the learned local rule, reproducing the full algorithm without any accumulated error.

The paper also positions its contributions relative to prior work. Earlier results (e.g., Back de Luca et al., 2025) provided exact learning guarantees for feed‑forward networks but required encoding the entire graph into a fixed‑size vector, leading to parameter counts that scale linearly or quadratically with the graph size. By contrast, the GNN shares the same local MLP across all nodes, yielding a model whose size is independent of the number of nodes and only depends on the maximum degree and the chosen block dimensions. Moreover, while many recent works (Weiet et al., 2022; Malach, 2023) give probabilistic approximation guarantees for Turing‑complete functions, they do not ensure exact execution, which is crucial for iterative algorithms where errors compound.

Limitations are acknowledged. The approach assumes a known upper bound on node degree D and a bound on the total number of bits a node can store (the “local memory”). The finite‑precision assumption (binary features) is essential for the NTK analysis; extending the results to real‑valued features or to unbounded degree graphs would require new techniques. Additionally, the ensemble size K must be sufficiently large to approximate the NTK, which may increase computational cost during training.

In summary, the paper delivers a rigorous, constructive proof that GNNs can be trained to execute any bounded‑memory LOCAL algorithm exactly, by learning local instruction mappings with MLP ensembles and embedding them into a message‑passing architecture. This bridges a gap between expressive‑power results (showing that GNNs can simulate LOCAL) and learnability results (showing that standard gradient‑based training can actually find the required parameters). The work opens avenues for exact algorithmic learning on graphs, reliable graph‑based reasoning in safety‑critical systems, and principled design of GNNs that are provably correct for a wide class of combinatorial problems.