Scientific Theory of a Black-Box: A Life Cycle-Scale XAI Framework Based on Constructive Empiricism

Explainable AI (XAI) offers a growing number of algorithms that aim to answer specific questions about black-box models. What is missing is a principled way to consolidate explanatory information about a fixed black-box model into a persistent, auditable artefact, that accompanies the black-box throughout its life cycle. We address this gap by introducing the notion of a scientific theory of a black (SToBB). Grounded in Constructive Empiricism, a SToBB fulfils three obligations: (i) empirical adequacy with respect to all available observations of black-box behaviour, (ii) adaptability via explicit update commitments that restore adequacy when new observations arrive, and (iii) auditability through transparent documentation of assumptions, construction choices, and update behaviour. We operationalise these obligations as a general framework that specifies an extensible observation base, a traceable hypothesis class, algorithmic components for construction and revision, and documentation sufficient for third-party assessment. Explanations for concrete stakeholder needs are then obtained by querying the maintained record through interfaces, rather than by producing isolated method outputs. As a proof of concept, we instantiate a complete SToBB for a neural-network classifier on a tabular task and introduce the Constructive Box Theoriser (CoBoT) algorithm, an online procedure that constructs and maintains an empirically adequate rule-based surrogate as observations accumulate. Together, these contributions position SToBBs as a life cycle-scale, inspectable point of reference that supports consistent, reusable analyses and systematic external scrutiny.

💡 Research Summary

The paper addresses a fundamental gap in current Explainable AI (XAI) research: the lack of a persistent, auditable artifact that aggregates explanatory information about a fixed black‑box model throughout its entire life cycle. To fill this gap, the authors introduce the concept of a Scientific Theory of a Black‑Box (SToBB). Drawing on Van Fraassen’s Constructive Empiricism, they formulate three core obligations for any SToBB: (i) empirical adequacy – the surrogate must reproduce the black‑box’s behavior on all observed input‑output pairs; (ii) adaptability – explicit update commitments must restore adequacy whenever new observations appear; and (iii) auditability – all assumptions, construction choices, and update policies must be transparently documented for third‑party assessment.

The framework is operationalised through four components. First, an observation base defines the space of measurable inputs, predictions, and auxiliary metrics, and enforces data‑quality standards. Second, a hypothesis class specifies the form of the interpretable surrogate (e.g., rule‑based or decision‑tree models) together with structural constraints that guarantee human‑readability. Third, an algorithmic engine called Constructive Box Theoriser (CoBoT) incrementally builds and revises the surrogate in an online fashion: it detects mismatches between the current surrogate and the black‑box, generates candidate rules that resolve the mismatch, resolves conflicts using priority and comprehensibility criteria, and records every change as part of the update policy. Fourth, a documentation layer records meta‑information about assumptions, design decisions, and adaptation events, thereby enabling independent verification by auditors or regulators.

To demonstrate feasibility, the authors instantiate a complete SToBB for a neural‑network classifier trained on the Abalone dataset (a tabular task predicting whether an abalone has reached a target age). Starting from the training data, CoBoT constructs a rule‑based surrogate achieving 85 % accuracy on the observed data. As the model is deployed, streaming observations are continuously fed into the system; CoBoT updates the rule set, ultimately attaining 99.2 % agreement with the black‑box on the entire accumulated observation set. Explanations for diverse stakeholder queries—global feature importance for developers, local counterfactuals for operators, subgroup fairness checks for auditors, and price justification for buyers—are all obtained by querying the maintained SToBB record rather than by running separate XAI methods. This yields consistent, reusable explanations and eliminates the “disagreement problem” that plagues isolated XAI outputs.

The authors discuss several limitations. The current formulation assumes a static black‑box; handling model versioning or retraining would require extensions to preserve continuity of the theory. Restricting the surrogate to rule‑based forms may limit the ability to capture highly non‑linear relationships, and the adequacy guarantee hinges on a sufficiently rich observation base; sparse or biased observations could undermine the theory’s empirical adequacy. Future work is suggested on dynamic model management, hybrid surrogates (combining rules with linear or neural components), and optimal observation design.

In sum, the paper reframes XAI from a collection of ad‑hoc explanation methods into a scientifically grounded, life‑cycle‑scale artifact. By aligning XAI with the philosophical principles of Constructive Empiricism, it provides a systematic route to achieve consistency, adaptability, and regulatory auditability—features increasingly demanded by standards such as the EU AI Act, NIST AI Risk Management Framework, and ISO/IEC 42001. The SToBB concept thus offers a promising blueprint for turning explainability into a durable knowledge asset rather than a fleeting visualisation.