Do Not Treat Code as Natural Language: Implications for Repository-Level Code Generation and Beyond

Large language models for code (CodeLLMs) have demonstrated remarkable success in standalone code completion and generation, sometimes even surpassing human performance, yet their effectiveness diminishes in repository-level settings where cross-file dependencies and structural context are essential. Existing Retrieval-Augmented Generation (RAG) approaches often borrow strategies from NLP, relying on chunking-based indexing and similarity-based retrieval. Chunking results in the loss of coherence between code units and overlooks structural relationships, while similarity-driven methods frequently miss functionally relevant dependencies such as helper functions, classes, or global variables. To address these limitations, we present Hydra, a repository-level code generation framework that treats code as structured code rather than natural language. Our approach introduces (i) a structure-aware indexing strategy that represents repositories as hierarchical trees of functions, classes, and variables, preserving code structure and dependencies, (ii) a lightweight dependency-aware retriever (DAR) that explicitly identifies and retrieves the true dependencies required by a target function, and (iii) a hybrid retrieval mechanism that combines DAR with similarity-based retrieval to provide both essential building blocks and practical usage examples. Extensive experiments on the challenging DevEval and RepoExec benchmarks, both requiring function implementation from real-world repositories with complex large repository context, show that Hydra achieves state-of-the-art performance across open- and closed-source CodeLLMs. Notably, our method establishes a new state of the art in repository-level code generation, surpassing strongest baseline by over 5% in Pass@1 and even enabling smaller models to match or exceed the performance of much larger ones that rely on existing retrievers.

💡 Research Summary

The paper “Do Not Treat Code as Natural Language: Implications for Repository‑Level Code Generation and Beyond” identifies a fundamental mismatch between current Retrieval‑Augmented Generation (RAG) approaches for code and the structural nature of software repositories. Existing methods borrow heavily from natural‑language processing: they split source files into fixed‑size text chunks and rely on similarity‑based retrieval (BM25, TF‑IDF, dense embedding cosine similarity). This leads to two major problems. First, chunking fragments logical units such as functions, classes, and global variables, discarding hierarchical relationships and cross‑file dependencies that are essential for real‑world development. Second, pure similarity ranking often misses functionally relevant context because the needed symbols may have little lexical overlap with the query, causing models to either re‑implement existing helpers or to receive noisy, unrelated examples.

To address these issues, the authors propose Hydra, a repository‑level code generation framework that treats code as structured data rather than plain text. Hydra consists of three core components:

1. Structure‑aware indexing – The entire repository is parsed with an AST parser into fine‑grained components (functions, classes, variables). These components are stored in a hierarchical tree that preserves parent‑child relationships, import links, and call‑graph edges. This representation enables retrieval at the level of logical code units instead of arbitrary text spans.

2. Dependency‑Aware Retriever (DAR) – Given a generation query (e.g., an incomplete function signature and its surrounding file), DAR predicts which symbols the target function will need. It traverses the indexed tree, guided by import statements and a lightweight graph neural network (or tree‑LSTM), to extract the exact definitions of required functions, classes, and globals. DAR is lightweight, requiring far fewer parameters than dense retrievers and achieving logarithmic‑time lookup even for large codebases.

3. Hybrid retrieval (Hydra Retriever) – DAR’s output (the “essential building blocks”) is combined with a traditional BM25 similarity search that supplies top‑k lexically similar snippets, offering concrete usage examples and style cues. The two sources are merged with a token‑budget‑aware weighting scheme and concatenated to the original prompt before being fed to a code generation model.

The authors evaluate Hydra on two recent Python‑focused benchmarks, DevEval and RepoExec, which require generating function implementations within real‑world repositories containing complex cross‑file dependencies. They test both open‑source models (Qwen2.5‑Coder 1.5 B and 7 B) and a closed‑source model (GPT‑4.1 mini). Across all settings, Hydra achieves state‑of‑the‑art results, improving Pass@1 by more than 5 % over the strongest baselines. Notably, the 1.5 B model equipped with Hydra matches or exceeds the performance of a 7 B model that uses conventional retrievers, effectively narrowing a four‑fold model‑size gap.

Ablation studies show that DAR alone recovers over 92 % of required dependencies with an average latency under 200 ms, while the hybrid combination adds an extra 3–4 % gain in Pass@1 by providing useful usage patterns. The paper also discusses token efficiency, retrieval speed, and the robustness of the approach across different model families.

In conclusion, Hydra demonstrates that treating code as structured, dependency‑rich data dramatically improves repository‑level code generation. By explicitly modeling hierarchical relationships and functional dependencies, it overcomes the limitations of chunk‑based indexing and similarity‑only retrieval. The authors suggest future work on multi‑language support, dynamic (runtime) dependency analysis, tighter integration with IDEs, and scaling to industrial‑size codebases. Hydra’s principles could also benefit related tasks such as code search, automated refactoring, and bug detection, marking a significant step toward more realistic, context‑aware AI‑assisted software development.