Logic Explanation of AI Classifiers by Categorical Explaining Functors

The most common methods in explainable artificial intelligence are post-hoc techniques which identify the most relevant features used by pretrained opaque models. Some of the most advanced post hoc methods can generate explanations that account for the mutual interactions of input features in the form of logic rules. However, these methods frequently fail to guarantee the consistency of the extracted explanations with the model’s underlying reasoning. To bridge this gap, we propose a theoretically grounded approach to ensure coherence and fidelity of the extracted explanations, moving beyond the limitations of current heuristic-based approaches. To this end, drawing from category theory, we introduce an explaining functor which structurally preserves logical entailment between the explanation and the opaque model’s reasoning. As a proof of concept, we validate the proposed theoretical constructions on a synthetic benchmark verifying how the proposed approach significantly mitigates the generation of contradictory or unfaithful explanations.

💡 Research Summary

The paper tackles a fundamental shortcoming of most post‑hoc explainable AI (XAI) techniques: while they can extract logical rules that approximate a pretrained opaque model, the extracted rules often contradict each other or fail to faithfully reflect the model’s internal reasoning, especially when explanations are generated for different layers and then composed. To address this, the authors turn to category theory as a formal framework for guaranteeing consistency and compositionality of explanations.

First, they define two basic categories. The fuzzy‑function category F has objects that are hypercubes