법률 분야 LLM 성능 향상을 위한 문서 구조 재배치와 역할 기반 프롬프트 연구

Structured Definitions and Segmentations for Legal Reasoning in LLMs: A Study on Indian Legal Data Mann Khatri1[0000−0002−5132−9223], Mirza Yusuf1[0000−0002−8293−5381], Rajiv Ratn Shah1[0000−0003−1028−9373], and Ponnurangam Kumaraguru2[0000−0001−5082−2078] 1 Indraprastha Institute of Information Technology, Delhi {mannk,rajivratn}@iiitd.ac.in, mirzayusuf1000@gmail.com 2 International Institute of Information Technology Hyderabad pk.guru@iiit.ac.in Abstract. Large Language Models (LLMs), trained on extensive datasets from the web, exhibit remarkable general reasoning skills. Despite this, they often struggle in specialized areas like law, mainly because they lack domain-specific pretraining. The legal field presents unique chal- lenges, as legal documents are generally long and intricate, making it hard for models to process the full text efficiently. Previous studies have examined in-context approaches to address the knowledge gap, boosting model performance in new domains without full domain alignment. In our paper, we analyze model behavior on legal tasks by conducting ex- periments in three areas: (i) reorganizing documents based on rhetorical roles to assess how structured information affects long context processing and model decisions, (ii) defining rhetorical roles to familiarize the model with legal terminology, and (iii) emulating the step-by-step reasoning of courts regarding rhetorical roles to enhance model reasoning. These ex- periments are conducted in a zero-shot setting across three Indian legal judgment prediction datasets. Our results reveal that organizing data or explaining key legal terms significantly boosts model performance, with a minimum increase of 1.5% and a maximum improvement of 4.36% in F1 score compared to the baseline. Keywords: Legal NLP · LEGAL AI · Legal Judgment Prediction · LJP 1 Introduction Large Language Models (LLMs) have exhibited impressive generalization ca- pabilities across a broad spectrum of natural language processing tasks [6, 36]. Their ability to follow instructions, reason over complex inputs, and generate co- herent text has made them powerful tools for downstream applications in diverse domains [9, 28, 10]. This success is largely attributed to pretraining on massive and heterogeneous datasets, which enables LLMs to capture a wide range of linguistic and semantic patterns. arXiv:2511.20669v1 [cs.CL] 14 Nov 2025 2 Mann et al. However, the computational and financial costs associated with pretraining such models are substantial. As a result, domain-specific LLMs are relatively rare, and most applications rely on general-purpose models that may not perform optimally in specialized contexts. This limitation is particularly evident in fields such as biomedicine, law, and finance, where domain knowledge and terminology diverge significantly from the data used during pretraining [38, 21]. To mitigate this domain mismatch without incurring the high cost of re- training, recent studies have explored in-context learning (ICL) as a strategy to guide LLMs at inference time [20]. By embedding relevant in-context information like examples into the prompt, ICL enables models to adapt to new tasks with minimal supervision. These approaches have shown encouraging results across a range of general tasks, demonstrating that prompt engineering can, to some extent, compensate for the lack of domain alignment. The legal domain, and in particular legal judgment prediction (LJP), poses unique challenges for LLMs. Legal documents are often lengthy, formally struc- tured, and replete with specialized vocabulary and rhetorical conventions [16], . Predicting outcomes based on such documents requires not only linguistic com- petence but also an understanding of domain-specific reasoning, procedural logic, and hierarchical structure [38, 35, 33, 31, 21]. These complexities make legal tasks especially demanding in zero-shot or few-shot settings [15]. While there has been growing interest in applying ICL to the legal domain, prior work has primarily focused on leveraging factual context or case retrieval [20]. Some studies have explored the use of exemplars or retrieved precedents to improve performance on legal reasoning tasks [32]. The extent to which restructuring or explicitly clarifying legal concepts can improve LLMs’ abilities in zero-shot LJP tasks has been inadequately explored. This research investigates how prompt design, especially by integrating legal rhetorical roles and structured reasoning, can enhance LLMs’ effectiveness in legal judgment predictions. We introduce three methodological approaches: (i) reorganizing documents based on rhetorical roles to help the model navigate through complex legal texts more easily; (ii) defining rhetorical roles to familiar- ize the model with specialized vocabulary; and (iii) mimicking court-like reason- ing to replicate the logical flow of legal arguments. These methods are assessed in a zero-shot context using three datasets rela

…(Full text truncated)…

This content is AI-processed based on ArXiv data.