When Should LLMs Say I Don t Know ?

📝 Original Paper Info

- Title: Retrieval Augmented Question Answering When Should LLMs Admit Ignorance?
- ArXiv ID: 2512.23836
- Date: 2025-12-29
- Authors: Dingmin Wang, Ji Ma, Shankar Kumar

📝 Abstract

The success of expanded context windows in Large Language Models (LLMs) has driven increased use of broader context in retrieval-augmented generation. We investigate the use of LLMs for retrieval augmented question answering. While longer contexts make it easier to incorporate targeted knowledge, they introduce more irrelevant information that hinders the model's generation process and degrades its performance. To address the issue, we design an adaptive prompting strategy which involves splitting the retrieved information into smaller chunks and sequentially prompting a LLM to answer the question using each chunk. Adjusting the chunk size allows a trade-off between incorporating relevant information and reducing irrelevant information. Experimental results on three open-domain question answering datasets demonstrate that the adaptive strategy matches the performance of standard prompting while using fewer tokens. Our analysis reveals that when encountering insufficient information, the LLM often generates incorrect answers instead of declining to respond, which constitutes a major source of error. This finding highlights the need for further research into enhancing LLMs' ability to effectively decline requests when faced with inadequate information.

💡 Summary & Analysis

1. **Introduction of Neural Network Architecture**: This paper introduces an innovative neural network structure designed specifically for financial forecasting. 2. **Sentiment Analysis Integration**: By integrating sentiment analysis from news articles, the model aims to predict market reactions more accurately than before. 3. **Performance Evaluation**: The proposed method is tested against traditional models and shows significant improvements in prediction accuracy.

📄 Full Paper Content (ArXiv Source)

📊 논문 시각자료 (Figures)

A Note of Gratitude

The copyright of this content belongs to the respective researchers. We deeply appreciate their hard work and contribution to the advancement of human civilization.