FloydNet: A Learning Paradigm for Global Relational Reasoning

Developing models capable of complex, multi-step reasoning is a central goal in artificial intelligence. While representing problems as graphs is a powerful approach, Graph Neural Networks (GNNs) are fundamentally constrained by their message-passing mechanism, which imposes a local bottleneck that limits global, holistic reasoning. We argue that dynamic programming (DP), which solves problems by iteratively refining a global state, offers a more powerful and suitable learning paradigm. We introduce FloydNet, a new architecture that embodies this principle. In contrast to local message passing, FloydNet maintains a global, all-pairs relationship tensor and learns a generalized DP operator to progressively refine it. This enables the model to develop a task-specific relational calculus, providing a principled framework for capturing long-range dependencies. Theoretically, we prove that FloydNet achieves 3-WL (2-FWL) expressive power, and its generalized form aligns with the k-FWL hierarchy. FloydNet demonstrates state-of-the-art performance across challenging domains: it achieves near-perfect scores (often >99%) on the CLRS-30 algorithmic benchmark, finds exact optimal solutions for the general Traveling Salesman Problem (TSP) at rates significantly exceeding strong heuristics, and empirically matches the 3-WL test on the BREC benchmark. Our results establish this learned, DP-style refinement as a powerful and practical alternative to message passing for high-level graph reasoning.

💡 Research Summary

FloydNet introduces a fundamentally different paradigm for graph reasoning by replacing the local message‑passing mechanism of traditional Graph Neural Networks (GNNs) with a global, dynamic‑programming‑style refinement of all‑pairs relationships. The model maintains a three‑dimensional tensor R ∈ ℝ^{N×N×d_r} that encodes a representation for every node pair (i, k). An initial projection combines node features, optional edge features, and optional graph‑level features via a multilayer perceptron to produce R^{(0)}. A learnable “SuperNode” is added to provide a unified interface for node‑level and graph‑level tasks: node embeddings are read from R_{i,SN} and the graph embedding from R_{SN,SN}.

The core computational block, the FloydBlock, updates R iteratively. Each block follows a pre‑LayerNorm transformer skeleton but replaces the standard self‑attention with a novel Pivotal Attention mechanism. For a target pair (i, k), a query vector q_{ik} is derived. For every possible pivot node j, the two‑hop path representations R_{ij} and R_{jk} are projected to key and value vectors (k_{ij}, v_{ij}) and (k_{jk}, v_{jk}). These are combined by a simple operation C (addition in most experiments) to form a joint key k_{ijk}=C(k_{ij},k_{jk}) and joint value v_{ijk}=C(v_{ij},v_{jk}). Multi‑head attention then aggregates over all pivots:  o_{ik}=∑{j} softmax(q{ik}·k_{ijk}/√d)·v_{ijk}. The result replaces R_{ik} via a residual connection, and a feed‑forward network follows. Stacking L such blocks yields R^{(L)} that integrates information from all relational paths of length up to 2L, providing exponential growth of the receptive field and mitigating the over‑squashing problem typical of deep GNNs.

The authors also generalize the architecture to k‑FloydNet, which operates on hyper‑edges connecting k vertices. By treating a hyper‑edge as a k‑tuple and using a pivot to replace one element, the same attention machinery captures higher‑order interactions. The computational cost scales as O(N^{k+1}·d_r + N^{k}·d_r²), but a custom kernel avoids storing O(N³·d_r) intermediate tensors, reducing memory to O(N²·d_r).

Theoretical analysis links FloydNet to the Weisfeiler–Lehman (WL) hierarchy. Theorem 1 proves that k‑FloydNet exactly implements the k‑FWL color refinement process, thus inheriting its distinguishing power. Consequently, 2‑FloydNet (the original model) is equivalent to 2‑FWL, which is known to be as expressive as the 3‑WL test. Theorem 2 shows that after L layers, each entry R^{(L)}_{ik} aggregates information from all paths of length ≤2L, establishing principled long‑range propagation.

Empirical evaluation covers three domains:

1. Expressive Power – On synthetic homomorphism‑counting tasks, FloydNet reduces mean absolute error to near zero, outperforming MPNNs, Subgraph GNNs, and local 2‑GNNs. On the BREC benchmark for graph isomorphism, FloydNet matches the 3‑WL heuristic (≈67.5 % accuracy). Higher‑order variants achieve 95 % (3‑FloydNet) and 99.8 % (4‑FloydNet), confirming the theoretical hierarchy.

2. Neural Algorithmic Reasoning – Using the CLRS‑30 suite, FloydNet attains near‑perfect scores across all 30 algorithms, especially excelling on dynamic‑programming problems such as shortest‑path and minimum‑spanning‑tree, where it outperforms Graphormer, PPGT, and other baselines.

3. Combinatorial Optimization – For the general Traveling Salesperson Problem (TSP) with 20–50 cities, FloydNet finds optimal tours at a rate that surpasses strong heuristics like 2‑opt and Lin‑Kernighan, while remaining computationally competitive.

Overall, FloydNet demonstrates that a learned, DP‑style global refinement can replace local message passing, delivering higher WL expressivity, permutation equivariance without positional encodings, and strong performance on both synthetic reasoning tasks and real‑world optimization problems. Limitations include cubic time complexity, which may hinder scalability to very large graphs, and sensitivity to the choice of the combination operator C for specific domains. Future work is suggested on efficient approximations, broader hypergraph applications, and automated design of the combination function.