Innovation Discovery System for Networking Research

As networking systems become increasingly complex, achieving disruptive innovation grows more challenging. At the same time, recent progress in Large Language Models (LLMs) has shown strong potential for scientific hypothesis formation and idea generation. Nevertheless, applying LLMs effectively to networking research remains difficult for two main reasons: standalone LLMs tend to generate ideas by recombining existing solutions, and current open-source networking resources do not provide the structured, idea-level knowledge necessary for data-driven scientific discovery. To bridge this gap, we present SciNet, a research idea generation system specifically designed for networking. SciNet is built upon three key components: (1) constructing a networking-oriented scientific discovery dataset from top-tier networking conferences, (2) simulating the human idea discovery workflow through problem setting, inspiration retrieval, and idea generation, and (3) developing an idea evaluation method that jointly measures novelty and practicality. Experimental results show that \system consistently produces practical and novel networking research ideas across multiple LLM backbones, and outperforms standalone LLM-based generation in overall idea quality.

💡 Research Summary

The paper addresses the growing difficulty of producing disruptive innovations in networking systems by introducing SciNet, a research‑idea generation framework that tightly couples large language models (LLMs) with a domain‑specific knowledge base. The authors first construct a high‑quality dataset from SIGCOMM and NSDI papers published between 2021 and 2025. Using LLMs, each paper is distilled into three structured fields—Background, Problem, and Design—and domain labels are consolidated into 50 standardized categories, yielding a uniform JSON representation that preserves system‑level details often lost in abstracts or raw PDFs.

Two complementary knowledge graphs are then built. The “paper graph” captures explicit relationships among domains, problems, papers, and methods, enabling precise retrieval of existing solutions relevant to a user‑specified research problem. The “citation graph” augments this by linking papers through their reference lists and extracting the core methods of cited works, thereby exposing cross‑domain inspirations. Both graphs are queried via a GraphRAG‑style global search that provides explainable, holistic answers.

SciNet’s workflow mimics a human researcher: (1) the user defines a domain and concrete problem; (2) the system retrieves relevant existing methods from the paper graph and inspirational methods from the citation graph; (3) an LLM generates candidate ideas that combine the problem context with the retrieved methods. Each candidate is compared against the dataset; the one with the lowest similarity to existing work is selected as the initial idea. The idea is then iteratively refined: the LLM is prompted to list technical challenges, propose optimization suggestions, and incorporate them until a maturity criterion or a maximum number of iterations is reached.

To evaluate both novelty and practicality without costly implementation, the authors adopt a temporal split. Only papers and knowledge up to 2024 are used for graph construction and LLM prompting. Novelty is measured as low similarity to pre‑2024 methods, while practicality is inferred from high similarity to methods that actually appeared after 2024. Experiments across multiple LLM backbones (LLaMA‑2, GPT‑3.5, Claude) show that SciNet consistently outperforms standalone LLM generation, achieving lower pre‑2024 similarity (higher novelty) and higher post‑2024 similarity (greater practicality). An ablation study confirms that each component—dataset cleaning, the two knowledge graphs, and iterative refinement—contributes meaningfully; removing the citation graph, for instance, markedly reduces idea diversity and quality.

The paper acknowledges limitations such as LLM hallucinations, dependence on the quality of citation data, and the absence of full system‑level validation. Future work is proposed to integrate automated implementation checks, domain‑specific LLM pre‑training, and multimodal feedback (e.g., code generation, simulation) to further bridge the gap between generated concepts and deployable networking solutions. Overall, SciNet represents a pioneering step toward data‑driven, LLM‑augmented scientific discovery tailored to the networking research community.