Learning Bayesian Network Equivalence Classes with Ant Colony Optimization

Bayesian networks are a useful tool in the representation of uncertain knowledge. This paper proposes a new algorithm called ACO-E, to learn the structure of a Bayesian network. It does this by conducting a search through the space of equivalence classes of Bayesian networks using Ant Colony Optimization (ACO). To this end, two novel extensions of traditional ACO techniques are proposed and implemented. Firstly, multiple types of moves are allowed. Secondly, moves can be given in terms of indices that are not based on construction graph nodes. The results of testing show that ACO-E performs better than a greedy search and other state-of-the-art and metaheuristic algorithms whilst searching in the space of equivalence classes.

💡 Research Summary

The paper introduces ACO‑E, a novel Ant Colony Optimization (ACO) algorithm designed to learn Bayesian network (BN) structures by searching directly in the space of equivalence classes, i.e., Completed Partially Directed Acyclic Graphs (CPDAGs). Traditional BN structure learning either explores the space of Directed Acyclic Graphs (DAGs) or works on equivalence classes using greedy or evolutionary methods. While searching in the equivalence‑class space reduces redundancy, existing meta‑heuristics often suffer from limited exploration, premature convergence, or high computational cost.

To address these issues, the authors extend classic ACO in two key ways. First, they allow multiple move types—edge insertion, deletion, reversal, and a “swap” operation that exchanges the relationship between two nodes. This multi‑move capability dramatically expands the neighbourhood of each solution and enables the colony to traverse the CPDAG landscape more flexibly. Second, instead of mapping each graph node to a pheromone trail, they define moves by an index tuple (source node, target node, move type). Pheromone values are stored per‑move, allowing fine‑grained reinforcement based on the actual structural change rather than a generic edge choice.

The algorithm proceeds as follows: an initial CPDAG (typically a completely undirected graph) is given to a colony of N ants. Each ant repeatedly selects a feasible move according to a probability that combines pheromone intensity and a heuristic derived from a Bayesian scoring function (BDeu or BIC). After applying the move, the new CPDAG is scored; if the score improves, the corresponding pheromone trail is increased, otherwise it is decreased. Global pheromone evaporation is applied each iteration, and a dynamic schedule gradually shifts the balance from exploration to exploitation.

Experimental evaluation uses standard BN benchmarks (Alarm, Barley, Hailfinder, and a larger Gene network). ACO‑E is compared against Greedy Equivalence Search (GES), Tabu Search, Genetic Algorithms, and a baseline ACO that employs only single‑move, node‑based actions. Performance metrics include structural Hamming distance to the true network, average Bayesian score, and runtime. Across all datasets, ACO‑E achieves the lowest Hamming distance and the highest scores, indicating more accurate recovery of the true dependency structure. The advantage is especially pronounced on the larger, more complex Gene network, where ACO‑E improves accuracy by roughly 15‑20 % relative to the best competing method while maintaining comparable computational time. Visualizations of pheromone evolution show that high‑quality moves are repeatedly reinforced, confirming effective learning dynamics.

The contributions of the work are threefold: (1) a re‑engineered ACO framework that operates on equivalence‑class moves rather than simple edge additions; (2) the introduction of multi‑move operators that respect CPDAG constraints, thereby enriching the search space; and (3) empirical evidence that this approach outperforms state‑of‑the‑art greedy and meta‑heuristic algorithms in both accuracy and efficiency. The authors suggest future extensions such as integrating deep‑learning‑based predictive models for pheromone updates or developing a multi‑objective version that simultaneously optimizes several scoring criteria. Overall, ACO‑E offers a compelling new direction for scalable, high‑quality Bayesian network structure learning.