Leverage Knowledge Graph and Large Language Model for Law Article Recommendation: A Case Study of Chinese Criminal Law

Judicial efficiency is critical to social stability. However, in many countries worldwide, grassroots courts face substantial case backlogs, and judicial decisions remain heavily dependent on judges’ cognitive efforts, with insufficient intelligent tools to enhance efficiency. To address this issue, we propose a highly efficient law article recommendation approach combining a Knowledge Graph (KG) and a Large Language Model (LLM). First, we construct a Case-Enhanced Law Article Knowledge Graph (CLAKG) to store current law articles, historical case information, and their interconnections, alongside an LLM-based automated construction method. Building on this, we propose a closed-loop law article recommendation framework integrating graph embedding-based retrieval and KG-grounded LLM reasoning. Experiments on judgment documents from China Judgments Online demonstrate that our method boosts law article recommendation accuracy from 0.549 to 0.694, outperforming strong baselines significantly. To support reproducibility and future research, all source code and processed datasets are publicly available on GitHub (see Data Availability Statement).

💡 Research Summary

The paper addresses the pressing problem of judicial inefficiency, especially in lower‑level courts that suffer from massive case backlogs and rely heavily on judges’ manual reasoning. To improve the accuracy and interpretability of law‑article recommendation, the authors propose a hybrid system that tightly integrates a Knowledge Graph (KG) with a Large Language Model (LLM).

First, they design a task‑oriented schema called the Case‑Enhanced Law Article Knowledge Graph (CLAKG). CLAKG consists of two sub‑graphs: the Law Article Knowledge Graph (LAKG), which stores the current criminal statutes, and the Adjudicated Cases Knowledge Graph (ACKG), which stores historical judgments. The schema defines entities such as “LawArticle”, “Case”, “LegalConcept” and relationships like “cites”, “belongsToChapter”, “hasFactPattern”, enabling a rich, structured representation of both statutory text and case law.

Second, the authors develop an LLM‑driven automated construction pipeline. Using carefully crafted prompts that assign the LLM a “legal‑expert” role and enforce strict output formats, the model extracts triples from raw judgment documents and statutes. A post‑processing step validates the triples against the predefined schema, removes duplicates, and inserts them into CLAKG. This automation dramatically reduces manual annotation effort and makes the graph scalable to large corpora.

Third, they introduce a closed‑loop recommendation framework. When a new case description is provided, a graph‑embedding retriever (e.g., TransE/RotatE) quickly fetches a set of candidate law articles and similar past cases from CLAKG. These candidates, together with the case facts, are fed to the LLM through a “CLAKG‑grounded” prompt. The LLM performs a multi‑step reasoning process inspired by Chain‑of‑Verification: it first drafts a list of applicable articles, then generates targeted verification questions, answers them independently, and finally revises the draft based on the verification results. This design explicitly forces the model to rely on verifiable KG evidence, thereby mitigating the well‑known hallucination problem of LLMs in legal contexts.

The system is evaluated on a large dataset of Chinese criminal judgments scraped from China Judgments Online. The primary metric is accuracy (with auxiliary F1 scores). Baselines include traditional text‑classification models (BERT, GRU, DPCNN) and several Retrieval‑Augmented Generation (RAG) approaches (TFIDF‑RAG, Graph‑RAG, Light‑RAG). The proposed CLAKG + LLM pipeline achieves an accuracy of 0.694, a substantial jump from the LLM‑only baseline (0.549) and from all strong baselines. Ablation studies show that removing any component—graph‑based retrieval, KG‑grounded prompting, or the verification loop—degrades performance, confirming the complementary contribution of each part.

A further innovation is the human‑machine collaboration loop. Legal experts review the LLM’s recommendations, correct any mistakes, and feed the corrections back into CLAKG. This continuous feedback mechanism keeps the knowledge graph up‑to‑date with new statutes, case law, and expert insights, enhancing long‑term robustness and adaptability.

In summary, the paper demonstrates that a tightly coupled KG‑LLM architecture can substantially improve law‑article recommendation accuracy while providing transparent, evidence‑based explanations. The use of KG‑grounded prompts and a verification‑oriented reasoning chain effectively curbs hallucinations, a critical requirement for high‑stakes legal AI. The authors suggest future work on multimodal evidence integration, automatic schema evolution, and cross‑jurisdictional generalization.