The working principles of model-based GAs fall within the PAC framework: A mathematical theory of problem decomposition

The concepts of linkage, building blocks, and problem decomposition have long existed in the genetic algorithm field and have guided the development of model-based genetic algorithms for decades. However, their definitions are usually vague, making it difficult to develop theoretical support. This paper provides an algorithm-independent definition to describe the concept of linkage. With this definition, the paper proves that any problem with a bounded degree of linkage is decomposable and that proper problem decomposition is possible via linkage learning. The way of decomposition given in this paper also offers a new perspective on nearly decomposable problems with bounded difficulty and building blocks from the theoretical aspect. Finally, this paper relates problem decomposition to probably approximately correct (PAC) learning and proves that the global optima of problems with bounded decomposition difficulty are PAC learnable and the decomposition is decidable in polynomial time under certain conditions.

💡 Research Summary

**
The paper tackles a long‑standing gap in the theoretical understanding of model‑based genetic algorithms (MBGAs) by providing a rigorous, algorithm‑independent definition of “linkage” (also referred to as epistasis) and by linking this definition to problem decomposition and PAC (Probably Approximately Correct) learning theory.

Key Contributions

1. Formal Definition of Linkage/Epistasis – The authors introduce a precise mathematical notion of epistasis based on constrained optima. For a set of loci S and a target locus v, S ⇒ v (|S|-epistatic) holds if fixing the alleles at S can change the set of optimal alleles at v. They further distinguish strict (u → v) and non‑strict (u 99K v) epistatic relations, and define strong versus weak epistatic groups. This definition is independent of any particular GA population and captures the intrinsic structure of the fitness landscape.

2. Problem Decomposition Theorem – By constructing an epistasis graph whose vertices are loci and edges represent epistatic dependencies, the authors prove that if the graph’s maximum degree is bounded by k, the original optimization problem can be decomposed into sub‑problems each involving at most k variables. Consequently, any problem with bounded linkage degree is decomposable into “building blocks” that can be optimized independently.

3. Epistasis Blanket Theorem – The theorem states that to set the optimal allele at a locus correctly, it suffices to correctly set all loci within a bounded epistatic radius (determined by the graph’s degree). This provides a sufficient condition for the success of MBGAs that learn only local dependencies.

4. PAC Learnability of Decomposable Problems – The paper defines a “decomposition difficulty” parameter d, measuring the depth of the epistasis graph. When d is a constant, the global optimum can be learned to within ε accuracy with confidence 1 − δ using a number of samples polynomial in the problem size, 1/ε, and log(1/δ). Thus, problems with bounded decomposition difficulty are PAC‑learnable.

5. Polynomial‑time Decidability of Decomposition – Under the same bounded‑degree and bounded‑depth conditions, the authors show that deciding whether a given problem admits such a decomposition can be performed in polynomial time. This establishes that the decomposition step required by MBGAs is not only theoretically possible but also computationally tractable.

6. Connection to Model‑Based GAs – The theoretical framework directly explains why MBGAs—such as those employing probabilistic graphical models, factorized representations, or optimal mixing operators—perform well. MBGAs first learn the epistasis graph (linkage learning) and then use it to guide recombination. The paper shows that if the learned model captures the true bounded epistatic structure, the algorithm can reconstruct the global optimum by solving each sub‑problem independently.

Methodology Overview

• The authors introduce a set of symbols (V for loci, g for the unique global optimum, A for assignments, Ψ_A for constrained optima, etc.) and formalize assignments, coverage, and evaluation.
• Epistasis is defined via constrained optima: S ⇒ v holds if there exists an assignment covering S that changes the optimal allele set at v.
• Strong epistasis requires every non‑empty subset of S to also be epistatic to v; weak epistasis requires no proper subset to be epistatic.
• Using these definitions, they construct the epistasis graph and prove the decomposition theorem by showing that any vertex’s optimal allele can be determined from its incident edges of bounded size.
• The PAC analysis follows the standard sample‑complexity bounds, leveraging the fact that the hypothesis class (assignments consistent with the bounded epistatic structure) has VC‑dimension polynomial in the problem size.
• Decidability is shown by reducing the decomposition decision to a graph‑partitioning problem that can be solved via depth‑first search and checking degree constraints, both polynomial‑time operations.

Experimental Illustration
The paper illustrates the theory on classic benchmark functions: OneMax, LeadingOnes, concatenated trap (CTrap), cyclic trap (CycTrap), concatenated needle‑in‑a‑haystack (CNiah), and a novel “LeadingTraps” problem. For each, the authors explicitly construct the epistasis relations, compute the graph degree, and demonstrate how the decomposition aligns with known building‑block structures.

Implications and Future Work
By grounding linkage learning in a formal, problem‑centric definition, the work provides a solid foundation for analyzing existing MBGAs and designing new ones. It clarifies under which structural conditions (bounded degree, bounded depth) MBGAs are guaranteed to succeed, and it identifies the precise computational limits of decomposition. Future directions suggested include extending the framework to multi‑modal landscapes (multiple global optima), dynamic environments where epistatic relations evolve over time, and applying the theory to non‑binary or continuous domains such as neural architecture search or combinatorial optimization on graphs.

In summary, the paper delivers a comprehensive mathematical theory that (i) precisely defines linkage, (ii) proves that bounded linkage implies polynomial‑time decomposability, (iii) connects this decomposition to PAC learnability, and (iv) explains the empirical success of model‑based genetic algorithms through the lens of epistasis graphs. This bridges a critical gap between heuristic evolutionary computation and formal learning theory.