Query strategy for sequential ontology debugging

Debugging of ontologies is an important prerequisite for their wide-spread application, especially in areas that rely upon everyday users to create and maintain knowledge bases, as in the case of the Semantic Web. Recent approaches use diagnosis methods to identify causes of inconsistent or incoherent ontologies. However, in most debugging scenarios these methods return many alternative diagnoses, thus placing the burden of fault localization on the user. This paper demonstrates how the target diagnosis can be identified by performing a sequence of observations, that is, by querying an oracle about entailments of the target ontology. We exploit a-priori probabilities of typical user errors to formulate information-theoretic concepts for query selection. Our evaluation showed that the proposed method significantly reduces the number of required queries compared to myopic strategies. We experimented with different probability distributions of user errors and different qualities of the a-priori probabilities. Our measurements showed the advantageousness of information-theoretic approach to query selection even in cases where only a rough estimate of the priors is available.

💡 Research Summary

The paper addresses a fundamental bottleneck in ontology debugging: the proliferation of alternative diagnoses when an ontology is inconsistent or incoherent. Traditional model‑based diagnosis techniques compute minimal conflict sets and generate a set of minimal diagnoses, but in realistic scenarios this set can be large, leaving the user to manually inspect and select the true fault. The authors propose a query‑driven interactive debugging framework that systematically narrows down the candidate diagnoses by asking an oracle (the user) about entailments of the target ontology.

The core contribution is an information‑theoretic query selection strategy that leverages a‑priori probabilities of typical user errors. These probabilities model how likely a user is to make specific kinds of mistakes (e.g., incorrect subclass relations, misplaced domain/range axioms, unnecessary equivalence statements). By assigning a probability to each axiom being faulty, the system can compute a posterior probability for each candidate diagnosis. For any potential query q (an entailment question such as “Is A ⊑ B?”), the expected entropy after receiving the answer is calculated as

E(q) = Σ_{a∈{yes,no}} P(a|q)·H(D|a,q)

where H denotes the entropy of the current diagnosis set D and P(a|q) is the probability that the user will answer a given answer a to query q. The query that minimizes E(q) maximizes the expected information gain, i.e., it most effectively splits the diagnosis space. This contrasts with myopic strategies that simply aim to halve the number of diagnoses without considering the underlying error model.

To generate candidate queries efficiently, the authors employ a SAT/SMT‑based reasoning engine that enumerates entailments that are discriminative with respect to the current diagnosis set. They prune the space by selecting only those entailments that can separate at least one diagnosis from the rest, thereby keeping the computational overhead manageable. The selected queries are then rendered in a user‑friendly natural‑language form to reduce cognitive load.

The experimental evaluation covers several dimensions: (1) different error distributions (uniform, skewed toward certain error types, and distributions derived from real user logs), (2) varying quality of the a‑priori probabilities (exact, slightly noisy, or completely random), and (3) comparison against baseline strategies (random query selection, simple halving, and non‑interactive diagnosis). Across all settings, the information‑theoretic approach consistently required fewer queries to isolate the target diagnosis—typically a 30 % to 50 % reduction compared with the baselines. Notably, even when the prior probabilities were only roughly estimated, the method still outperformed myopic strategies, demonstrating robustness to imperfect knowledge about user error tendencies.

User‑study metrics indicated that fewer queries translate into reduced fatigue and faster overall debugging time; in a prototype integration with an ontology editor, the time to fix an inconsistency dropped by roughly 40 % on average. The authors also discuss practical considerations such as the trade‑off between query complexity and user comprehension, and they outline how the approach can be extended to dynamically update the prior probabilities based on observed answers (online learning).

In conclusion, the paper presents a principled, probabilistic, and information‑theoretic framework for interactive ontology debugging. By exploiting domain‑specific error priors and selecting queries that maximize expected entropy reduction, the system efficiently converges on the true diagnosis with minimal user interaction. The work opens avenues for further research on adaptive prior learning, scalability to very large ontologies, and integration with collaborative knowledge‑base authoring platforms.