An Experimental Comparison of Hybrid Algorithms for Bayesian Network Structure Learning

We present a novel hybrid algorithm for Bayesian network structure learning, called Hybrid HPC (H2PC). It first reconstructs the skeleton of a Bayesian network and then performs a Bayesian-scoring greedy hill-climbing search to orient the edges. It is based on a subroutine called HPC, that combines ideas from incremental and divide-and-conquer constraint-based methods to learn the parents and children of a target variable. We conduct an experimental comparison of H2PC against Max-Min Hill-Climbing (MMHC), which is currently the most powerful state-of-the-art algorithm for Bayesian network structure learning, on several benchmarks with various data sizes. Our extensive experiments show that H2PC outperforms MMHC both in terms of goodness of fit to new data and in terms of the quality of the network structure itself, which is closer to the true dependence structure of the data. The source code (in R) of H2PC as well as all data sets used for the empirical tests are publicly available.

💡 Research Summary

**
The paper introduces a new hybrid algorithm for Bayesian network (BN) structure learning called Hybrid HPC (H2PC). The method follows the hybrid paradigm: a constraint‑based (CB) phase first identifies a skeleton (undirected graph) and a subsequent score‑based (SB) phase orients the edges using a greedy hill‑climbing search guided by the BDeu score. The novelty lies in the CB phase, where the authors replace the widely used Max‑Min Parents‑and‑Children (MMPC) subroutine (employed by the state‑of‑the‑art hybrid algorithm MMHC) with a more sophisticated procedure called Hybrid Parents‑and‑Children (HPC).

HPC is an ensemble of three components: (i) DE‑PCS (Data‑Efficient Parents‑and‑Children Superset) and DE‑SPS (Data‑Efficient Spouses Superset), which quickly generate supersets of the true PC and spouse sets while restricting the conditioning set size (|Z|≤2 for DE‑PCS, ≤3 for DE‑SPS) to keep statistical tests reliable; (ii) Inter‑IAPC, a weak incremental PC learner based on the Inter‑IAMB framework that iteratively adds variables that show the strongest conditional dependence with the target given the current Markov blanket; and (iii) a decentralized search loop that checks, for each variable X that appears in the DE‑PCS superset but not in the current PC estimate, whether the target T appears in X’s PC estimate. If so, X is added to T’s PC. This bidirectional verification dramatically reduces false‑negative errors, a known weakness of MMPC, especially when sample sizes are limited.

After obtaining a PC set for every variable, the SB phase proceeds exactly as in MMHC: starting from an empty DAG, a greedy hill‑climbing algorithm evaluates three operators (add‑edge, delete‑edge, reverse‑edge) and selects the move that yields the largest increase in the BDeu score. Edge additions are constrained to pairs (X→Y) where Y belongs to PC(X), thereby limiting the search space. A TABU list of the last 100 structures is maintained to avoid cycling and to escape local maxima. The search terminates when no improving move can be found.

The authors evaluate H2PC against MMHC on eight benchmark networks (Alarm, Barley, Child, etc.) with sample sizes of 100, 500, 1 000, and 5 000 drawn from the true joint distributions. Performance is measured by structural Hamming distance, precision, recall, and test‑set BDeu score, as well as runtime and memory consumption. Across all networks and data regimes, H2PC consistently achieves lower Hamming distances and higher precision/recall than MMHC. The advantage is most pronounced for small samples (≤500), where H2PC’s ability to recover missing edges leads to up to a 10 % improvement in recall. In terms of predictive accuracy, H2PC’s learned models obtain 2–5 % higher BDeu scores on held‑out data, indicating better generalisation. Runtime is modestly higher in the CB phase (≈1.3× MMPC) due to the extra independence tests, but the overall execution remains within a few minutes for networks up to a thousand variables, and memory usage is comparable.

The paper discusses limitations: the additional tests increase computational load, which may become prohibitive for very high‑dimensional data; the current implementation handles only discrete variables, requiring adaptation of the independence test for continuous or mixed data; and the method still relies on the faithfulness assumption. Future work is suggested in three directions: (1) parallelising the HPC component to improve scalability; (2) extending the framework to continuous variables using Fisher’s Z or kernel‑based tests; and (3) developing adaptive strategies for setting the conditioning set size to balance test reliability and computational cost.

In conclusion, H2PC demonstrates that a more robust PC discovery phase can substantially improve both structural fidelity and predictive performance of hybrid BN learners, establishing a new benchmark that surpasses MMHC while preserving practical efficiency.