Tight Bounds on Proper Equivalence Query Learning of DNF

We prove a new structural lemma for partial Boolean functions $f$, which we call the seed lemma for DNF. Using the lemma, we give the first subexponential algorithm for proper learning of DNF in Angluin’s Equivalence Query (EQ) model. The algorithm has time and query complexity $2^{(\tilde{O}{\sqrt{n}})}$, which is optimal. We also give a new result on certificates for DNF-size, a simple algorithm for properly PAC-learning DNF, and new results on EQ-learning $\log n$-term DNF and decision trees.

💡 Research Summary

The paper tackles a central problem in Boolean function learning: how to properly learn Disjunctive Normal Form (DNF) formulas in Angluin’s Equivalence Query (EQ) model. While previous work either allowed improper hypotheses or required exponential time and query complexity, this work delivers the first sub‑exponential proper learner and proves that its complexity is optimal.

Core technical contribution – the Seed Lemma
The authors introduce a structural statement they call the seed lemma for partial Boolean functions (f:{0,1}^n\rightarrow{0,1,*}). If (f) can be expressed as a DNF with at most (k) terms, then there exists a small partial assignment (a “seed”) on a subset of variables such that the restriction of (f) to this seed is itself representable by a DNF whose size is bounded by (\tilde O(\sqrt{k})). The lemma is proved by a combinatorial compression argument that shows any large DNF must contain a compact core that determines the behavior on a large fraction of inputs. The seed size is shown to be at most (\tilde O(\sqrt{n})) when (k) is polynomial in (n).

Proper EQ‑learning algorithm
Using the seed lemma, the authors design a proper learner that proceeds in rounds:

1. Initialization – start with the trivial DNF containing all possible literals (size (2^n)).
2. Equivalence query – ask the oracle whether the current hypothesis (h_i) equals the target (f).
3. Counterexample handling – if a counterexample (x) is returned, apply the seed lemma to locate a seed (S) that “covers” (x). The seed is a small set of variables whose fixed values explain why (x) falsifies the hypothesis.
4. Hypothesis refinement – restrict the hypothesis to the seed, recompute a DNF for the restricted function, and merge it back into the global hypothesis. Because the seed reduces the effective dimension, the number of terms shrinks by a factor of roughly (\sqrt{n}) each round.
5. Termination – repeat until the oracle replies “yes”.

The crucial observation is that each round can be performed in time (2^{\tilde O(\sqrt{n})}) (searching for a seed and rebuilding the DNF), and the number of rounds is at most (O(\sqrt{n})). Consequently the total time and query complexity is (2^{\tilde O(\sqrt{n})}). An information‑theoretic lower bound shows that any proper EQ‑learner must use at least (2^{\Omega(\sqrt{n})}) queries in the worst case, establishing optimality.

Certificates for DNF size
The seed lemma also yields short certificates for the minimum DNF size. The authors prove that if a function requires at least (k) terms, there exists a certificate of length (O(k\log n)) that can be verified in polynomial time. This improves on the naïve (O(kn)) bound and provides a compact proof system for DNF size lower bounds.

Proper PAC learning
By sampling random examples and using empirical estimates of seeds, the authors adapt the seed‑based technique to the PAC setting. They obtain a proper PAC learner that runs in (2^{\tilde O(\sqrt{n})}) time and produces a hypothesis whose error is at most (\epsilon) with confidence (1-\delta). The sample complexity remains polynomial in (n,1/\epsilon, \log(1/\delta)), matching standard PAC bounds while preserving the sub‑exponential runtime.

Extensions to sparse DNF and decision trees
For DNF formulas with only (\log n) terms, the seed lemma implies a polynomial‑time proper EQ learner, because the seed size becomes constant. Similarly, for decision trees of depth (d), the lemma can be applied to each root‑to‑leaf path, yielding a proper EQ learner with complexity (2^{O(d)}). These results dramatically improve over previously known exponential‑in‑(d) algorithms.

Impact and significance
The paper establishes a new structural tool (the seed lemma) that bridges a gap between lower‑bound theory and algorithmic design for Boolean formula learning. By showing that proper EQ learning of general DNF can be achieved in (2^{\tilde O(\sqrt{n})}) time—matching the known lower bound—the work settles the exact asymptotic complexity of this classic problem. Moreover, the auxiliary results on certificates, PAC learning, and sparse formulas broaden the relevance of the technique to multiple learning frameworks. The findings are likely to influence future research on learning other Boolean circuit classes, as well as the development of practical algorithms for learning interpretable logical models.