Large Language Models Hallucination: A Comprehensive Survey

Large language models (LLMs) have transformed natural language processing, achieving remarkable performance across diverse tasks. However, their impressive fluency often comes at the cost of producing false or fabricated information, a phenomenon known as hallucination. Hallucination refers to the generation of content by an LLM that is fluent and syntactically correct but factually inaccurate or unsupported by external evidence. Hallucinations undermine the reliability and trustworthiness of LLMs, especially in domains requiring factual accuracy. This survey provides a comprehensive review of research on hallucination in LLMs, with a focus on causes, detection, and mitigation. We first present a taxonomy of hallucination types and analyze their root causes across the entire LLM development lifecycle, from data collection and architecture design to inference. We further examine how hallucinations emerge in key natural language generation tasks. Building on this foundation, we introduce a structured taxonomy of detection approaches and another taxonomy of mitigation strategies. We also analyze the strengths and limitations of current detection and mitigation approaches and review existing evaluation benchmarks and metrics used to quantify LLMs hallucinations. Finally, we outline key open challenges and promising directions for future research, providing a foundation for the development of more truthful and trustworthy LLMs.

💡 Research Summary

This survey offers a comprehensive examination of hallucination phenomena in large language models (LLMs), covering their origins, detection methods, mitigation strategies, evaluation benchmarks, and future research directions. The authors begin by defining hallucination as the generation of fluent, syntactically correct text that lacks factual grounding or is outright inaccurate. They distinguish hallucination from creative generation, emphasizing that while creativity can be intentional and valuable, hallucination is typically unintended and detrimental, especially in high‑stakes domains such as medicine, law, finance, and education.

The paper structures the analysis around the full LLM development lifecycle, dividing it into six stages: data collection and preprocessing, model architecture design, pre‑training, fine‑tuning, evaluation, and inference. For each stage, the authors identify specific risk factors that can introduce hallucinations. In the data stage, noisy web crawls, biased sources, and insufficient curation lead to factual errors that propagate downstream. Architectural choices—particularly the use of massive self‑attention layers and autoregressive decoding—grant the model excessive freedom to generate plausible but unsupported statements. Pre‑training on massive, unfiltered corpora amplifies the model’s tendency to prioritize coherence over truth, while fine‑tuning can either alleviate or exacerbate hallucination depending on the quality of supervision signals and loss functions. Evaluation practices that rely on narrow benchmarks may mask hallucination problems, and inference‑time factors such as prompt phrasing, temperature, and sampling strategies heavily influence the model’s confidence and consistency.

Hallucination types are organized into a two‑dimensional taxonomy. Intrinsic hallucinations contradict information present in the source document, often due to entity confusion or mis‑interpretation of context. Extrinsic hallucinations introduce content not found in the source; this content may be factually correct but remains unverifiable against the given input. The authors further refine the classification into factuality (errors relative to real‑world facts) and faithfulness (errors relative to the provided context), noting that both dimensions can manifest as contradictions, fabrications, or omissions.

Detection approaches are grouped into five major categories:

1. Retrieval‑based detection – compares model output against external knowledge bases or search results. Effective for factual errors but highly dependent on the coverage, freshness, and reliability of the external source.
2. Uncertainty‑based detection – leverages model‑internal confidence scores, calibration metrics, or entropy measures. Useful for flagging low‑confidence outputs but prone to missing high‑confidence hallucinations.
3. Embedding‑based detection – measures semantic distance between generated text and source/reference embeddings. Robust to paraphrasing but vulnerable to domain shift and low‑resource languages.
4. Learning‑based detection – trains supervised classifiers on annotated hallucination datasets. Achieves high accuracy when sufficient labeled data exist, yet suffers from annotation cost and potential bias.
5. Self‑consistency‑based detection – generates multiple samples for the same prompt and assesses consistency across them. Captures logical inconsistencies without external evidence but struggles with subtle factual errors and requires careful sampling design.

The survey emphasizes that no single detector excels across all scenarios; hybrid systems that combine complementary signals (e.g., retrieval + learning, uncertainty + self‑consistency) show the most promise.

Mitigation strategies are organized into four overarching families:

• Prompt‑centric techniques – structured prompting, chain‑of‑thought (CoT), few‑shot exemplars, and instruction tuning that steer the model toward factual generation.
• Retrieval‑augmented generation (RAG) – integrates external knowledge at inference time, grounding responses in up‑to‑date factual sources.
• Reasoning‑centric methods – CoT, self‑consistency, iterative refinement, and verification loops that encourage the model to articulate and check its own reasoning steps.
• Model‑centric adaptations – architectural modifications (e.g., factuality‑aware attention), specialized loss functions that penalize factual errors, continual fine‑tuning on fact‑checked corpora, and parameter‑efficient adapters focused on truthfulness.

Empirical evidence presented in the survey suggests that hybrid approaches—combining prompt engineering, retrieval grounding, reasoning verification, and model‑level training—yield the greatest reductions in hallucination rates. The authors also discuss multilingual and low‑resource challenges: cross‑lingual transfer, multilingual fine‑tuning, and language‑specific prompt adaptation can mitigate hallucinations, but performance gaps persist due to uneven knowledge‑base quality and limited annotated data.

The evaluation section reviews existing benchmarks such as FactCC, QAGS, SummEval, and newer fact‑checking datasets, highlighting their focus on English and high‑resource domains. Metrics surveyed include precision/recall, F1, BLEU, ROUGE, BERTScore, FactScore, and newer composite metrics that aim to capture factuality, faithfulness, and consistency simultaneously. The authors argue for a multi‑dimensional evaluation framework that blends automatic scores with human judgments to better reflect real‑world utility.

Finally, the paper outlines open research directions: (1) building large‑scale, cost‑effective hallucination annotation pipelines; (2) developing dynamic knowledge‑update mechanisms that keep external grounding sources current; (3) integrating multimodal evidence (e.g., images, tables) for richer fact verification; (4) advancing meta‑learning techniques that enable models to self‑diagnose and correct hallucinations; and (5) extending robust detection and mitigation to under‑represented languages and domains. By mapping the entire LLM lifecycle, categorizing causes, and systematically reviewing detection and mitigation methods, this survey provides a foundational roadmap for creating more truthful, reliable, and trustworthy language generation systems.