Deep Learning-based Method for Expressing Knowledge Boundary of Black-Box LLM

Large Language Models (LLMs) have achieved remarkable success, however, the emergence of content generation distortion (hallucination) limits their practical applications. The core cause of hallucination lies in LLMs’ lack of awareness regarding their stored internal knowledge, preventing them from expressing their knowledge state on questions beyond their internal knowledge boundaries, as humans do. However, existing research on knowledge boundary expression primarily focuses on white-box LLMs, leaving methods suitable for black-box LLMs which offer only API access without revealing internal parameters-largely unexplored. Against this backdrop, this paper proposes LSCL (LLM-Supervised Confidence Learning), a deep learning-based method for expressing the knowledge boundaries of black-box LLMs. Based on the knowledge distillation framework, this method designs a deep learning model. Taking the input question, output answer, and token probability from a black-box LLM as inputs, it constructs a mapping between the inputs and the model’ internal knowledge state, enabling the quantification and expression of the black-box LLM’ knowledge boundaries. Experiments conducted on diverse public datasets and with multiple prominent black-box LLMs demonstrate that LSCL effectively assists black-box LLMs in accurately expressing their knowledge boundaries. It significantly outperforms existing baseline models on metrics such as accuracy and recall rate. Furthermore, considering scenarios where some black-box LLMs do not support access to token probability, an adaptive alternative method is proposed. The performance of this alternative approach is close to that of LSCL and surpasses baseline models.

💡 Research Summary

The paper addresses the problem of expressing the knowledge boundaries of large language models (LLMs) that are only accessible through black‑box APIs. Existing work on knowledge‑boundary expression largely assumes white‑box access, relying on fine‑tuning or internal signals, and typically reduces the problem to a binary “Know/Unknow” classification. Both assumptions are unrealistic for modern commercial LLM services such as GPT‑4, DeepSeek‑V3, or Claude, which expose only input‑output interfaces and often hide token‑level probabilities.

To fill this gap, the authors propose LSCL (LLM‑Supervised Confidence Learning), a deep‑learning framework that infers an LLM’s internal knowledge state from three observable signals: the user question, the model’s generated answer, and (when available) the token‑level probability distribution of that answer. LSCL is built on a knowledge‑distillation paradigm: the black‑box LLM acts as a teacher that provides supervision, while a lightweight student network learns to map the observable signals to a confidence score that reflects the model’s true mastery of the required knowledge.

A central contribution is the definition of Correctness‑Adjusted Token Probability (CATP). CATP combines the binary correctness of the answer (determined from an external ground‑truth label) with the raw token probability, thereby correcting two well‑known deficiencies of raw token probabilities: (1) high token probability does not guarantee correctness, and (2) low token probability does not necessarily indicate ignorance. CATP yields a calibrated confidence value that aligns with answer correctness and can serve as a training target for the student network.

The LSCL architecture consists of three modules:

1. Question‑Answer Alignment Component – This sub‑network captures both local semantic similarity (word‑level) and global consistency (sentence‑level) between the input question and the LLM’s answer. By explicitly modeling alignment, the student network can better infer whether the answer is grounded in relevant knowledge.

2. Confidence Learning Module – Using the aligned representations, a shallow transformer or multilayer perceptron predicts the CATP score for each query‑answer pair. The model is deliberately lightweight; the authors demonstrate successful training and inference on a consumer‑grade GPU (NVIDIA RTX 4060 Ti, 16 GB).

3. Adaptive Thresholding Module – Rather than fixing a single confidence cutoff, LSCL automatically partitions the distribution of predicted CATP scores on a validation set into three intervals, corresponding to three knowledge states: “Know”, “Sciolism”, and “Unknow”. The newly introduced “Sciolism” class captures intermediate cases where the model produces a correct answer with low confidence or an incorrect answer with high confidence, reflecting partial knowledge or uncertainty.

The authors evaluate LSCL on multiple public datasets spanning medical, financial, and general‑knowledge domains, and on several prominent black‑box LLMs (GPT‑4, DeepSeek‑V3, Claude). Baselines include (a) raw token‑probability confidence, (b) prompt‑based uncertainty markers, and (c) white‑box fine‑tuning methods adapted to the black‑box setting. Metrics reported are accuracy, recall, F1‑score, and AUC of CATP‑based classification. LSCL consistently outperforms all baselines, achieving 12‑18 % higher scores across metrics. In particular, the three‑way classification yields a markedly higher recall for the “Sciolism” state, enabling more nuanced risk management in safety‑critical applications.

The paper also tackles the practical scenario where a black‑box LLM does not expose token probabilities. An adaptive alternative that relies solely on question and answer embeddings is proposed; its performance degrades by less than 3 % relative to the full LSCL, confirming robustness.

Limitations acknowledged by the authors include: (1) dependence on labeled correctness for CATP computation, which may be costly in fully unlabeled settings; (2) potential domain bias if training data do not cover the target application area; and (3) current focus on textual inputs, leaving multimodal extensions for future work.

Future directions suggested are (i) integrating semi‑supervised or self‑training techniques to reduce label dependence, (ii) applying meta‑learning for rapid domain adaptation, and (iii) extending the framework to multimodal LLMs that process images or tables.

In summary, LSCL offers a practical, scalable solution for quantifying and expressing the knowledge boundaries of black‑box LLMs. By introducing a calibrated confidence metric (CATP) and a three‑state taxonomy that includes an intermediate “Sciolism” class, the method advances the state of the art in LLM reliability and safety, providing a valuable tool for developers and researchers deploying large language models in real‑world, high‑stakes environments.