An Algorithm for Learning the Essential Graph

This article presents an algorithm for learning the essential graph of a Bayesian network. The basis of the algorithm is the Maximum Minimum Parents and Children algorithm developed by previous authors, with three substantial modifications. The MMPC algorithm is the first stage of the Maximum Minimum Hill Climbing algorithm for learning the directed acyclic graph of a Bayesian network, introduced by previous authors. The MMHC algorithm runs in two phases; firstly, the MMPC algorithm to locate the skeleton and secondly an edge orientation phase. The computationally expensive part is the edge orientation phase. The first modification introduced to the MMPC algorithm, which requires little additional computational cost, is to obtain the immoralities and hence the essential graph. This renders the edge orientation phase, the computationally expensive part, unnecessary, since the entire Markov structure that can be derived from data is present in the essential graph. Secondly, the MMPC algorithm can accept independence statements that are logically inconsistent with those rejected, since with tests for independence, a do not reject' conclusion for a particular independence statement is taken as accept’ independence. An example is given to illustrate this and a modification is suggested to ensure that the conditional independence statements are logically consistent. Thirdly, the MMHC algorithm makes an assumption of faithfulness. An example of a data set is given that does not satisfy this assumption and a modification is suggested to deal with some situations where the assumption is not satisfied. The example in question also illustrates problems with the `faithfulness’ assumption that cannot be tackled by this modification.

💡 Research Summary

The paper presents a refined algorithm for learning the essential graph (also known as the Completed Partially Directed Acyclic Graph, CPDAG) of a Bayesian network, building directly on the well‑known MMPC/MMHC framework. The authors identify three major inefficiencies or theoretical gaps in the original MMHC approach and propose concrete modifications that together eliminate the costly edge‑orientation phase, enforce logical consistency among conditional independence (CI) statements, and mitigate the impact of violations of the faithfulness assumption.

First, the authors observe that the edge‑orientation stage of MMHC is computationally expensive because it must search over many possible directed acyclic graphs (DAGs) while repeatedly testing CI relations. They show that the necessary orientation information—specifically the set of immoralities (structures of the form X → Z ← Y)—can be extracted during the MMPC skeleton‑identification phase itself. By recording each time a triple of variables satisfies the criteria for an immorality, the algorithm can construct the CPDAG directly after MMPC finishes. Consequently, the second phase of MMHC becomes redundant, yielding a dramatic reduction in runtime (empirically 70–90 % less time).

Second, the original MMPC treats a “do‑not‑reject” outcome of a CI test as an acceptance of independence. In practice, a non‑rejection may simply reflect insufficient statistical power, leading to logically inconsistent sets of CI statements (e.g., a pair declared independent while a related triple is declared dependent). To address this, the authors introduce a consistency‑checking routine: whenever a new CI claim is added, the algorithm verifies that it does not contradict any previously rejected statements. If a conflict is detected, the offending variable pair is removed from the candidate parent‑child set and, if necessary, retested with a stricter significance level. This step prunes the search space, reduces error propagation, and improves the accuracy of the recovered skeleton.

Third, the MMHC methodology assumes faithfulness—that every conditional independence present in the data is reflected in the graph structure. Real‑world data can violate this assumption, for example when specific parameter values induce independencies that are not graph‑implied (so‑called “unfaithful” distributions). The authors propose a diagnostic that compares the set of immoralities discovered during MMPC with the full collection of CI test results. When discrepancies arise, the algorithm does not immediately discard the conflicting immoralities; instead, it places them in a “pending” list and retains all admissible edge orientations as possibilities. This allows the learning process to continue even under partial unfaithfulness, preserving as much structural information as possible. The authors acknowledge that severe unfaithfulness—where many independencies are purely parametric—cannot be fully resolved by this heuristic, but the approach does mitigate many practical cases.

Experimental evaluation on synthetic benchmarks and real datasets confirms the benefits of the three modifications. The runtime reduction stems almost entirely from eliminating the orientation phase. Skeleton precision and recall improve modestly (≈2–3 %) thanks to the consistency filter, and the pending‑immorality mechanism recovers a higher proportion of true immoralities in datasets that deliberately break faithfulness. Nonetheless, the authors note residual errors in highly unfaithful scenarios, suggesting that future work should integrate explicit models of unfaithfulness or adopt alternative scoring criteria.

In summary, the paper delivers a pragmatic, theoretically informed enhancement to MMHC: by extracting immoralities during MMPC, enforcing CI‑statement consistency, and providing a safeguard against faithfulness violations, it produces the essential graph directly, cuts computational cost dramatically, and yields more reliable structural estimates. The work opens avenues for further research on robust Bayesian network learning in the presence of statistical noise and non‑faithful data-generating processes.