Colorful Talks with Graphs: Human-Interpretable Graph Encodings for Large Language Models

Graph problems are fundamentally challenging for large language models (LLMs). While LLMs excel at processing unstructured text, graph tasks require reasoning over explicit structure, permutation invariance, and computationally complex relationships, creating a mismatch with the representations of text-based models. Our work investigates how LLMs can be effectively applied to graph problems despite these barriers. We introduce a human-interpretable structural encoding strategy for graph-to-text translation that injects graph structure directly into natural language prompts. Our method involves computing a variant of Weisfeiler-Lehman (WL) similarity classes and maps them to human-like color tokens rather than numeric labels. The key insight is that semantically meaningful and human-interpretable cues may be more effectively processed by LLMs than opaque symbolic encoding. Experimental results on multiple algorithmic and predictive graph tasks show the considerable improvements by our method on both synthetic and real-world datasets. By capturing both local and global-range dependencies, our method enhances LLM performance especially on graph tasks that require reasoning over global graph structure.

💡 Research Summary

The paper tackles the fundamental mismatch between large language models (LLMs) and graph-structured data. While LLMs excel at processing sequential natural language, graphs require permutation‑invariant, multi‑hop relational reasoning that does not fit naturally into a text stream. Existing graph‑to‑text approaches either serialize edges, use opaque numeric identifiers, or inject continuous GNN embeddings. These representations are either too long, linguistically meaningless, or tied to a specific model, limiting LLM performance on graph tasks.

The authors propose a two‑step encoding pipeline called CL‑OWL (Colorful Ordered Weisfeiler–Lehman). First, they compute node‑level structural descriptors using an ordered variant of the 1‑dimensional Weisfeiler–Lehman (WL) algorithm. Starting from a uniform label, each node aggregates the multiset of neighbor labels, sorts this multiset lexicographically, and maps the resulting tuple to a deterministic integer identifier. This process yields ordered WL labels that preserve structural similarity, reflect k‑hop neighborhoods, and, crucially, induce a global ordering consistent with distance‑weighted connectivity measures (proved in Theorem 1). In other words, higher WL labels correspond to nodes that are more “central” or have richer local topology.

Second, the ordered integer labels are mapped to human‑readable color tokens (e.g., red, orange, green, cyan). Colors are natural language tokens that LLMs have seen extensively during pre‑training, and they carry an intuitive notion of similarity (similar colors imply similar structure). The translation function τ then embeds the graph’s node list, edge list, and each node’s color label into a natural‑language prompt together with a task description (e.g., “What is the maximum flow from node 1 to node 4?”).

The authors conduct extensive experiments on synthetic graph families (random, scale‑free, grid) and real‑world benchmarks (Cora, PubMed, OGB). They evaluate algorithmic tasks (maximum flow, shortest path, subgraph existence) and predictive tasks (node classification, link prediction). Baselines include plain text serialization, numeric WL identifiers, and GNN‑embedding insertion. Across all settings, CL‑OWL consistently outperforms baselines, achieving 12–18 % higher accuracy or F1 score. The advantage is especially pronounced on larger graphs (up to 2 k nodes) where token‑budget constraints make compact, informative encodings critical. Adding few‑shot examples further amplifies the benefit, as the LLM learns to reason step‑by‑step using color cues.

A qualitative analysis of attention patterns shows that LLMs allocate higher attention weights to tokens sharing the same color, indicating that the model treats color tokens as meaningful structural signals rather than arbitrary symbols. Theoretical analysis links ordered WL labels to distance‑weighted connectivity, providing a principled justification for why the color mapping captures useful graph information.

In summary, the paper demonstrates that human‑interpretable, linguistically natural encodings—specifically color tokens derived from ordered WL refinement—bridge the gap between graph structure and LLM reasoning. By aligning graph representations with the linguistic priors of LLMs, the method enables LLMs to solve complex graph problems that previously required specialized graph neural networks or explicit algorithmic solvers. The work opens avenues for further exploration of other human‑friendly descriptors (shapes, sizes) and multimodal graph‑text interfaces.