Proficient Graph Neural Network Design by Accumulating Knowledge on Large Language Models

High-level automation is increasingly critical in AI, driven by rapid advances in large language models (LLMs) and AI agents. However, LLMs, despite their general reasoning power, struggle significantly in specialized, data-sensitive tasks such as designing Graph Neural Networks (GNNs). This difficulty arises from (1) the inherent knowledge gaps in modeling the intricate, varying relationships between graph properties and suitable architectures and (2) the external noise from misleading descriptive inputs, often resulting in generic or even misleading model suggestions. Achieving proficiency in designing data-aware models – defined as the meta-level capability to systematically accumulate, interpret, and apply data-specific design knowledge – remains challenging for existing automated approaches, due to their inefficient construction and application of meta-knowledge. To achieve meta-level proficiency, we propose DesiGNN, a knowledge-centered framework that systematically converts past model design experience into structured, fine-grained knowledge priors well-suited for meta-learning with LLMs. To account for the inherent variability and external noise, DesiGNN aligns empirical property filtering from extensive benchmarks with adaptive elicitation of literature insights via LLMs. By constructing a solid meta-knowledge between unseen graph understanding and known effective architecture patterns, DesiGNN can deliver top-5.77% initial model proposals for unseen datasets within seconds and achieve consistently superior performance with minimal search cost compared to baselines.

💡 Research Summary

The paper tackles the challenge of automatically designing Graph Neural Networks (GNNs) for unseen graph datasets, a task where large language models (LLMs) have shown limited success. The authors identify two fundamental obstacles: (1) an inherent knowledge gap—LLMs lack fine‑grained, data‑specific understanding of how graph properties (e.g., homophily, density, clustering) map to effective GNN architectures; and (2) external noise—LLMs are easily misled by superficial user descriptions (e.g., “citation network”), leading to generic or even harmful design suggestions. To overcome these issues, the authors propose DesiGNN, a knowledge‑centered framework that systematically converts past GNN design experience into structured, fine‑grained knowledge priors and integrates them with LLM reasoning.

DesiGNN consists of three modules. The Graph Understanding component extracts 16 canonical graph properties from each dataset and leverages large GNN benchmark collections (e.g., NAS‑Bench‑Graph) to compute two rankings: a statistical distance ranking (SR) based on property similarity, and an empirical performance ranking (ER) obtained by transferring top‑performing GNNs from other graphs. By comparing SR and ER, the system quantifies how well each property predicts successful architecture transfer, thereby building a meta‑knowledge base that captures graph‑GNN performance relationships.

The Knowledge Retrieval module treats the LLM as a “knowledge aligner.” It prompts the LLM with the empirically derived property filters and asks it to retrieve relevant design insights from the graph learning literature (e.g., recommendations for heterophilic graphs, attention‑based aggregators, scaling tricks). This step mitigates external noise by grounding the LLM’s natural‑language generation in concrete, benchmark‑validated evidence.

Finally, the Model Suggestion & Refinement stage uses the constructed meta‑knowledge M : (G, Θ) → P to generate an initial set of GNN architectures that have high posterior probability P(θ | G) under a Bayesian formulation. Crucially, this posterior can be approximated without training any model on the unseen graph, because the prior P(θ) is derived from the meta‑knowledge pool and the likelihood P(G | θ) is inferred from the property‑performance mappings. The initial proposals are then refined through a lightweight, knowledge‑driven loop: a small number of quick evaluations (or proxy metrics) feed back into the LLM, which suggests targeted modifications (e.g., swapping a GCN layer for a GIN layer on heterophilic data).

Empirical evaluation spans three out‑of‑distribution (OOD) graph datasets and eight benchmark datasets. DesiGNN’s initial designs rank, on average, within the top 5.77 % of the entire architecture search space, and after only 2–3 refinement iterations they consistently outperform strong baselines, including reinforcement‑learning AutoGNNs, evolutionary search, differentiable NAS, and prior LLM‑based methods such as Auto2Graph, GPT4GNAS, and GHGNAS. In terms of efficiency, the full pipeline requires only seconds to produce the first suggestions and a few additional seconds for refinement, achieving more than a tenfold reduction in GPU time compared to conventional AutoGNN pipelines.

The paper’s contributions are threefold: (1) it formalizes “meta‑proficiency” as the ability to accumulate, interpret, and apply fine‑grained, data‑aware design knowledge; (2) it demonstrates how to fuse empirical benchmark statistics with LLM‑derived literature insights to construct trustworthy meta‑knowledge; and (3) it shows that Bayesian posterior estimation can replace exhaustive model testing, enabling rapid, high‑quality GNN design for unseen graphs. DesiGNN thus offers a novel, scalable pathway for combining LLMs with meta‑learning in graph‑centric AI, and its principles could be extended to other domains where data‑sensitive architecture search is required.