ContextualSHAP : Enhancing SHAP Explanations Through Contextual Language Generation

📝 Abstract

Explainable Artificial Intelligence (XAI) has become an increasingly important area of research, particularly as machine learning models are deployed in high-stakes domains. Among various XAI approaches, SHAP (SHapley Additive exPlanations) has gained prominence due to its ability to provide both global and local explanations across different machine learning models. While SHAP effectively visualizes feature importance, it often lacks contextual explanations that are meaningful for end-users, especially those without technical backgrounds. To address this gap, we propose a Python package that extends SHAP by integrating it with a large language model (LLM), specifically OpenAI’s GPT, to generate contextualized textual explanations. This integration is guided by user-defined parameters (such as feature aliases, descriptions, and additional background) to tailor the explanation to both the model context and the user perspective. We hypothesize that this enhancement can improve the perceived understandability of SHAP explanations. To evaluate the effectiveness of the proposed package, we applied it in a healthcare-related case study and conducted user evaluations involving real end-users. The results, based on Likert-scale surveys and follow-up interviews, indicate that the generated explanations were perceived as more understandable and contextually appropriate compared to visual-only outputs. While the findings are preliminary, they suggest that combining visualization with contextualized text may support more user-friendly and trustworthy model explanations.

💡 Analysis

Explainable Artificial Intelligence (XAI) has become an increasingly important area of research, particularly as machine learning models are deployed in high-stakes domains. Among various XAI approaches, SHAP (SHapley Additive exPlanations) has gained prominence due to its ability to provide both global and local explanations across different machine learning models. While SHAP effectively visualizes feature importance, it often lacks contextual explanations that are meaningful for end-users, especially those without technical backgrounds. To address this gap, we propose a Python package that extends SHAP by integrating it with a large language model (LLM), specifically OpenAI’s GPT, to generate contextualized textual explanations. This integration is guided by user-defined parameters (such as feature aliases, descriptions, and additional background) to tailor the explanation to both the model context and the user perspective. We hypothesize that this enhancement can improve the perceived understandability of SHAP explanations. To evaluate the effectiveness of the proposed package, we applied it in a healthcare-related case study and conducted user evaluations involving real end-users. The results, based on Likert-scale surveys and follow-up interviews, indicate that the generated explanations were perceived as more understandable and contextually appropriate compared to visual-only outputs. While the findings are preliminary, they suggest that combining visualization with contextualized text may support more user-friendly and trustworthy model explanations.

📄 Content

ContextualSHAP : Enhancing SHAP Explanations Through Contextual Language Generation LATIFA DWIYANTI, Kanazawa University, Japan and Institut Teknologi Bandung, Indonesia SERGIO RYAN WIBISONO, Institut Teknologi Bandung, Indonesia HIDETAKA NAMBO, Kanazawa University, Japan Explainable Artificial Intelligence (XAI) has become an increasingly important area of research, particularly as machine learning models are deployed in high-stakes domains. Among various XAI approaches, SHAP (SHapley Additive exPlanations) has gained prominence due to its ability to provide both global and local explanations across different machine learning models. While SHAP effectively visualizes feature importance, it often lacks contextual explanations that are meaningful for end-users, especially those without technical backgrounds. To address this gap, we propose a Python package that extends SHAP by integrating it with a large language model (LLM), specifically OpenAI’s GPT, to generate contextualized textual explanations. This integration is guided by user-defined parameters (such as feature aliases, descriptions, and additional background) to tailor the explanation to both the model’s context and the user’s perspective. We hypothesize that this enhancement can improve the perceived understandability of SHAP explanations. To evaluate the effectiveness of the proposed package, we applied it in a healthcare-related case study and conducted user evaluations involving real end-users. The results, based on Likert-scale surveys and follow-up interviews, indicate that the generated explanations were perceived as more understandable and contextually appropriate compared to visual-only outputs. While the findings are preliminary, they suggest that combining visualization with contextualized text may support more user-friendly and trustworthy model explanations. Additional Key Words and Phrases: XAI, SHAP, Context-Aware Explanation, LLM, OpenAI GPT, Human-Centered AI, Perceived Understandability 1 Introduction As noted by Molnar [1], interpretability plays a crucial role in encouraging the adoption of machine learning (ML) and artificial intelligence (AI), since transparency helps build trust and supports deployment in high-stakes domains. In line with the goals of AI interpretability, DARPA launched the Explainable AI (XAI) program in 2015 to help users understand, trust and manage AI systems more effectively [2]. In 2021, they published a comprehensive summary that summarizes the program’s goals, structure, and research progress, further amplifying interest in XAI and inspiring widespread research in the field. Using the open API of the Semantic Scholar platform, which leverages AI through the Semantic Scholar Academic Graph (S2AG) to improve relevance in academic searches [3], the author conducted a query to retrieve articles related to XAI. A total of 5,544 papers were retrieved using three filtering parameters: the keyword “XAI,” publication years between 2020 and 2024, and inclusion in the field of Computer Science. The yearly distribution of these publications is shown in Figure 1, highlighting the continued growth and increasing interest in this research area. Currently, there is no universally accepted definition of XAI. The term is often used interchangeably with others such as Interpretable Machine Learning, Reasonable AI, or Understandable AI [4]. The goals of XAI can vary widely, as outlined by Arrieta et al. in their comprehensive review [5], ranging from enhancing trustworthiness, causality, and transferability to promoting informativeness, fairness, interactivity, and awareness of privacy. Among these, the goal most consistently Authors’ Contact Information: Latifa Dwiyanti, latifa@stu.kanazawa-u.ac.id, Kanazawa University, Kanazawa, Ishikawa, Japan and Institut Teknologi Bandung, Bandung, Indonesia; Sergio Ryan Wibisono, Institut Teknologi Bandung, Bandung, Indonesia; Hidetaka Nambo, Kanazawa University, Kanazawa, Ishikawa, Japan. arXiv:2512.07178v1 [cs.AI] 8 Dec 2025 Fig. 1. XAI Growing Papers emphasized in all studies is informativeness, the idea that explanations should provide users with sufficient information to support decision making [5]. Arun Rai beautifully captures the essence of XAI by likening it to transforming a ’black box’ into a ’glass box’, allowing users to understand the rationale behind AI predictions [6]. XAI has been applied in a wide range of domains, including medicine, healthcare, law, civil engineering, marketing, education, cybersecurity, transportation, and agriculture [7, 8]. However, several challenges persist, such as the trade-off between explainability and model performance, varying human interpretations, the absence of universal standards, biased data leading to unfair outcomes, and the lack of consensus on how to evaluate XAI methods [7]. To address human-centric challenges—particularly users with varying abilities to understand explanations—recent efforts have focused on developing con

This content is AI-processed based on ArXiv data.