We prove a new structural lemma for partial Boolean functions $f$, which we call the seed lemma for DNF. Using the lemma, we give the first subexponential algorithm for proper learning of DNF in Angluin's Equivalence Query (EQ) model. The algorithm has time and query complexity $2^{(\tilde{O}{\sqrt{n}})}$, which is optimal. We also give a new result on certificates for DNF-size, a simple algorithm for properly PAC-learning DNF, and new results on EQ-learning $\log n$-term DNF and decision trees.
Over twenty years ago, Angluin began study of the equivalence query (EQ) learning model [2,3]. Valiant [20] had asked whether DNF formulas were poly-time learnable in the PAC model; this question is still open. Angluin asked the same question in the EQ model. Using approximate fingerprints, she proved that any proper algorithm for EQ-learning DNF formulas requires superpolynomial query complexity, and hence super-polynomial time. In a proper DNF learning algorithm, all hypotheses are DNF formulas.
Angluin’s work left open the problem of determining the exact complexity of EQ-learning DNF, both properly and improperly. Tarui and Tsukiji noted that Angluin’s fingerprint proof can be modified to show that a proper EQ algorithm must have query complexity at least 2 ( Õ√ n) [19].
(They did not give details, but we prove this explicitly as a consequence of a more general result.) The most efficient improper algorithm for EQ-learning DNF is due to Klivans and Servedio (Corollary 12 of [17]), and runs in time 2 Õ(n 1/3 ) . In this paper, we give the first subexponential algorithm for proper learning of DNF in the EQ model. Our algorithm has time and query complexity that, like the lower bound, is 2 ( Õ√ n) .
Our EQ algorithm implies a new result on certificates for DNF size. Hellerstein et al. asked whether DNF has “poly-size certificates” [14], that is, whether there are polynomials q and r such that for all s, n > 0, functions requiring DNF formulas of size greater than q(s, n) have certificates of size r(s, n) certifiying that they do not have DNF formulas of size at most s. (This is equivalent to asking whether DNF can be properly MEQ-learned within polynomial query complexity [14].) Our result does not resolve this question, but it shows that there are analogous subexponential certificates. More specifically, it shows that there exists a function r(s, n) = 2 O( √ n log s log n) such that for all s, n > 0, functions requiring DNF formulas of size greater than r(s, n) have certificates of size r(s, n) certifying that they do not have DNF formulas of size at most s.
Our EQ algorithm is based on a new structural lemma for partial Boolean functions f , which we call the seed lemma for DNF. It states that if f has at least one positive example and is consistent with a DNF of size s, then f has a projection f p , induced by fixing the values of O( √ n log s)
variables, such that f p has at least one positive example, and is consistent with a monomial. We also use the seed lemma for DNF to obtain a new subexponential proper algorithm for PAC-learning DNFs which is simpler than the previous algorithm of Alekhnovich et al. [1], with the same bounds. That algorithm uses a procedure that runs multiple recursive calls in round robin fashion until one succeeds. In contrast, ours is an iterative procedure with a straightforward analysis.
Decision-trees can be PAC and EQ-learned in time n O(log s) , where s is the size of the tree [12,18]. We prove a seed lemma for decision trees as well, and use it to obtain an algorithm that learns decision trees using DNF hypotheses in time n O(log s 1 ) , where s 1 is the number of 1-leaves in the tree. (For any “minimal” tree, the number of 0-leaves is at most ns 1 ; this bound is tight for the optimal tree computing a monomial of n variables.)
We prove a lower bound result that quantifies the tradeoff between the number of queries needed to properly EQ-learn DNF formulas, and the size of such queries. One consequence is a lower bound of 2 Ω( that learns O(log n)-term DNF using DNF hypotheses, with equivalence queries alone. In contrast, Angluin and Kharitonov showed that, under cryptographic assumptions, membership queries do not help in PAC-learning unrestricted DNF formulas [5]. Blum and Singh gave an algorithm that PAC-learns log n-term DNF using DNF hypotheses of size n O(log n) in time n O(log n) [7]; our results imply that no significant improvement of this result is possible for PAC-learning log n-term DNF using DNF hypotheses.
A literal is a variable or its negation. A term, also called a monomial, is a possibly empty conjunction (∧) of literals. If the term is empty, all assignments satisfy it. The size of a term is the number of literals in it. We say that term t covers assignment
formula is either the constant 0, the constant 1, or a formula of the form t 1 ∨ • • • ∨ t k , where k ≥ 1 and each t i is a term. A k-term DNF is a DNF formula consisting of at most k terms. A k-DNF is a DNF formula where each term has size at most k. The size of a DNF formula is the number of its terms.
A partial Boolean function f maps {0, 1} n to {0, 1, * }, where * means undefined. A Boolean formula φ is consistent with a partial function f (and vice versa) if φ(x) = f (x) for all x ∈ {0, 1} n where f (x) = * . If f is a partial function, then dnf -size(f ) is the size of the smallest DNF formula consistent with f . Let X n = {x 1 , . . . , x n }. A projection of a (partial) function
This content is AI-processed based on open access ArXiv data.
