Improving Multi-step RAG with Hypergraph-based Memory for Long-Context Complex Relational Modeling

Preprint IMPROVING MULTI-STEP RAG WITH HYPERGRAPH- BASED MEMORY FOR LONG-CONTEXT COMPLEX RE- LATIONAL MODELING Chulun Zhou1∗, Chunkang Zhang∗, Guoxin Yu, Fandong Meng2, Jie Zhou2, Wai Lam1†, Mo Yu2† The Chinese University of Hong Kong1, WeChat AI2 {clzhou,wlam}@se.cuhk.edu.hk, zkang5051@gmail.com moyumyu@global.tencent.com ABSTRACT Multi-step retrieval-augmented generation (RAG) has become a widely adopted strategy for enhancing large language models (LLMs) on tasks that demand global comprehension and intensive reasoning. Many RAG systems incorporate a work- ing memory module to consolidate retrieved information. However, existing mem- ory designs function primarily as passive storage that accumulates isolated facts for the purpose of condensing the lengthy inputs and generating new sub-queries through deduction. This static nature overlooks the crucial high-order correlations among primitive facts, the compositions of which can often provide stronger guid- ance for subsequent steps. Therefore, their representational strength and impact on multi-step reasoning and knowledge evolution are limited, resulting in fragmented reasoning and weak global sense-making capacity in extended contexts. We introduce HGMEM, a hypergraph-based memory mechanism that extends the concept of memory beyond simple storage into a dynamic, expressive structure for complex reasoning and global understanding. In our approach, memory is repre- sented as a hypergraph whose hyperedges correspond to distinct memory units, en- abling the progressive formation of higher-order interactions within memory. This mechanism connects facts and thoughts around the focal problem, evolving into an integrated and situated knowledge structure that provides strong propositions for deeper reasoning in subsequent steps. We evaluate HGMEM on several chal- lenging datasets designed for global sense-making. Extensive experiments and in-depth analyses show that our method consistently improves multi-step RAG and substantially outperforms strong baseline systems across diverse tasks. 1 1 INTRODUCTION Single-step retrieval-augmented generation (RAG) often proves insufficient for resolving complex queries within long contexts (Trivedi et al., 2023; Shao et al., 2023; Cheng et al., 2025), motivat- ing the shift toward multi-step RAG methods that iteratively interleave retrieval with reasoning. To effectively capture dependencies across steps and condense the lengthy processing history, many ap- proaches incorporate working memory mechanisms inspired by human cognition (Lee et al., 2024; Zhong et al., 2024). However, current memory-enhanced multi-step RAG methods still face chal- lenges in complex relational modeling, especially for resolving global sense-making tasks over long contexts. During multi-step RAG execution, a straightforward implementation of working memory mech- anism is to let a large language model (LLM) summarize the interaction history into a plaintext description of current problem-solving state. This strategy has been widely adopted since early stud- ies (Li et al., 2023; Trivedi et al., 2023) as well as in commercial systems (Jones, 2025; Shen & Yang, 2025). Nonetheless, such unstructured memory mechanisms cannot be manipulated with sufficient *Equal contribution. †: Co-corresponding authors. 1We release our code at https://github.com/Encyclomen/HGMem 1 arXiv:2512.23959v2 [cs.CL] 2 Jan 2026 Preprint accuracy across steps and often lose the ability to back-trace references to retrieved texts. Conse- quently, recent research has shifted toward structured or semi-structured working memory, typically with predefined schemas such as relational tables (Lu et al., 2023), knowledge graphs (Oguz et al., 2022; Xu et al., 2025), or event-centric bullet points (Wang et al., 2025). However, existing memory mechanisms often treat memory as static storage that continually accu- mulates meaningful but primitive facts. This view overlooks the evolving nature of human working memory, which incrementally incorporates higher-order correlations from previously memorized content. This capacity is particularly crucial for resolving global sense-making tasks that involve complex relational modeling over long contexts. In such scenarios, the required knowledge for tack- ling a query is often composed of complex structures that extend beyond predefined schemas, and reasoning over long lists of primitive facts is both inefficient and prone to confusion with mixed or irrelevant information. Current memory mechanisms in multi-step RAG systems lack these abilities, preventing memory from effectively guiding LLMs’ interaction with external data sources. These limitations highlight the need for a working memory with stronger representational capacity. In this paper, we propose a hypergraph-based memory mechanism (HGMEM) for multi-step RAG systems, which enables memory to evolve into more expressive structures that support complex relational modeling to enhance LLMs’ unde

This content is AI-processed based on open access ArXiv data.