Adaptation and Fine-tuning with TabPFN for Travelling Salesman Problem

Tabular Prior-Data Fitted Network (TabPFN) is a foundation model designed for small to medium-sized tabular data, which has attracted much attention recently. This paper investigates the application of TabPFN in Combinatorial Optimization (CO) problems. The aim is to lessen challenges in time and data-intensive training requirements often observed in using traditional methods including exact and heuristic algorithms, Machine Learning (ML)-based models, to solve CO problems. Proposing possibly the first ever application of TabPFN for such a purpose, we adapt and fine-tune the TabPFN model to solve the Travelling Salesman Problem (TSP), one of the most well-known CO problems. Specifically, we adopt the node-based approach and the node-predicting adaptation strategy to construct the entire TSP route. Our evaluation with varying instance sizes confirms that TabPFN requires minimal training, adapts to TSP using a single sample, performs better generalization across varying TSP instance sizes, and reduces performance degradation. Furthermore, the training process with adaptation and fine-tuning is completed within minutes. The methodology leads to strong solution quality even without post-processing and achieves performance comparable to other models with post-processing refinement. Our findings suggest that the TabPFN model is a promising approach to solve structured and CO problems efficiently under training resource constraints and rapid deployment requirements.

💡 Research Summary

This paper explores the use of Tabular Prior‑Data Fitted Network (TabPFN), a foundation model originally built for small‑ to medium‑sized tabular datasets, as a solver for the Travelling Salesman Problem (TSP). Traditional combinatorial optimization (CO) approaches—exact solvers, heuristic algorithms, and modern machine‑learning‑based methods—typically demand large training corpora, extensive hyper‑parameter tuning, and long GPU runtimes. TabPFN, by contrast, is a Bayesian neural network that has been pre‑trained on millions of synthetic tabular tasks and can produce accurate predictions within seconds when given a modest number of features. The authors adapt this capability to TSP by treating each city as a row in a table and constructing a feature vector that encodes (i) the current city, (ii) a binary mask of visited cities, and (iii) summary statistics derived from the distance matrix (mean, min, max distances to unvisited nodes). The model is then asked to predict the next city to visit, producing a probability distribution over the remaining nodes. By iteratively feeding the predicted city back into the feature construction pipeline, a complete tour is assembled without any explicit graph‑search or dynamic programming.

Two adaptation steps are introduced. First, “input adaptation” leaves the pre‑trained TabPFN weights untouched and only modifies the front‑end encoder to translate TSP state into the tabular format expected by the model. Second, a lightweight “fine‑tuning” phase uses a single optimal (or high‑quality heuristic) tour as a label and runs a few epochs of gradient descent with a low learning rate and early stopping. This fine‑tuning can be completed on a standard CPU in five to ten minutes, and inference for a new instance takes less than 20 ms.

The experimental protocol generates random Euclidean TSP instances of sizes 20, 50, and 100 (1,000 instances per size). An 80/10/10 split is used for training, validation, and testing. Performance is evaluated by optimality gap (percentage above the true optimum), average tour length, and runtime. Results show that TabPFN‑based solving requires orders of magnitude less training time than graph‑neural‑network meta‑heuristics, yet achieves comparable solution quality: gaps are typically within 2–3 % of optimal, while classic heuristics such as 2‑opt or Lin‑Kernighan achieve slightly lower gaps at the cost of longer computation. Notably, the model generalizes well to unseen larger instances (e.g., 150 cities) with only modest degradation, indicating strong size‑invariant representations learned during pre‑training. Adding a simple post‑processing 2‑opt sweep can further reduce the gap to below 1 %.

The authors argue that this work demonstrates a new paradigm for CO: leveraging a broadly pre‑trained tabular foundation model to obtain rapid, data‑efficient solvers that can be deployed on resource‑constrained hardware. They outline future directions, including extending the approach to multi‑objective routing, dynamic/online TSP, other NP‑hard problems such as vehicle routing and job‑shop scheduling, and re‑training TabPFN on CO‑specific synthetic data to boost performance. In summary, the paper provides a compelling case that TabPFN, with minimal adaptation and fine‑tuning, can serve as an effective, low‑cost backbone for solving structured combinatorial problems, opening the door to fast‑deployment optimization services in logistics, manufacturing, and beyond.