Distribution-Independent Evolvability of Linear Threshold Functions

Valiant’s (2007) model of evolvability models the evolutionary process of acquiring useful functionality as a restricted form of learning from random examples. Linear threshold functions and their various subclasses, such as conjunctions and decision lists, play a fundamental role in learning theory and hence their evolvability has been the primary focus of research on Valiant’s framework (2007). One of the main open problems regarding the model is whether conjunctions are evolvable distribution-independently (Feldman and Valiant, 2008). We show that the answer is negative. Our proof is based on a new combinatorial parameter of a concept class that lower-bounds the complexity of learning from correlations. We contrast the lower bound with a proof that linear threshold functions having a non-negligible margin on the data points are evolvable distribution-independently via a simple mutation algorithm. Our algorithm relies on a non-linear loss function being used to select the hypotheses instead of 0-1 loss in Valiant’s (2007) original definition. The proof of evolvability requires that the loss function satisfies several mild conditions that are, for example, satisfied by the quadratic loss function studied in several other works (Michael, 2007; Feldman, 2009; Valiant, 2010). An important property of our evolution algorithm is monotonicity, that is the algorithm guarantees evolvability without any decreases in performance. Previously, monotone evolvability was only shown for conjunctions with quadratic loss (Feldman, 2009) or when the distribution on the domain is severely restricted (Michael, 2007; Feldman, 2009; Kanade et al., 2010)

💡 Research Summary

The paper investigates distribution‑independent evolvability within Valiant’s (2007) evolutionary learning framework, focusing on linear threshold functions (LTFs) and their subclasses such as conjunctions and decision lists. The central open question—whether conjunctions can evolve without any assumption on the underlying data distribution—has remained unresolved since Feldman and Valiant (2008). The authors resolve this by proving a negative result for conjunctions and a positive result for LTFs that enjoy a non‑negligible margin, using a non‑linear loss function.

Negative result for conjunctions
To show that conjunctions are not distribution‑independent evolvable, the authors introduce a new combinatorial parameter, called the correlation‑based learning complexity lower bound. This parameter quantifies how much correlation between hypotheses can be amplified by a given mutation set. A high value indicates that any learning algorithm that relies on correlation (as Valiant’s model does) must incur a large sample complexity or a very small performance gain per mutation step. By analyzing the structure of conjunctions, the authors prove that this parameter is large for the class, which forces any mutation‑selection scheme to have an expected performance increase bounded by an arbitrarily small ε under some distribution. Consequently, no algorithm that respects Valiant’s mutation constraints can guarantee monotonic improvement for all distributions, establishing that conjunctions are not distribution‑independent evolvable. This result holds regardless of the loss function used, thereby closing the long‑standing conjecture in the negative.

Positive result for margin‑large LTFs
The second part of the paper turns to LTFs of the form f(x)=sign(w·x−θ). When the margin γ = min_i |w·x_i−θ| is at least Ω(1/poly(n)), the authors design a simple evolutionary algorithm that works with any loss function satisfying mild conditions: convexity, continuity, and a unique global minimum at zero loss (the quadratic loss is a canonical example). The algorithm proceeds as follows:

1. From the current weight vector w, generate a candidate set by perturbing each coordinate up or down by a small δ (producing 2n candidates).
2. Evaluate each candidate on a fresh random sample using the chosen non‑linear loss (e.g., squared loss).
3. Select the candidate with the smallest empirical loss; if its loss is strictly lower than that of w, replace w with this candidate.

Because the loss is smooth, a small perturbation in the direction that reduces the loss always exists when a positive margin is present. The authors prove that the expected loss decrease per iteration is bounded below by a constant ε>0, which yields a monotonic (non‑decreasing) improvement guarantee. Moreover, after a polynomial number of steps the algorithm converges to a hypothesis whose error is O(1/γ), matching the standard PAC‑learning guarantees for large‑margin LTFs. Importantly, the algorithm’s monotonicity holds for any data distribution, establishing distribution‑independent evolvability for this subclass of LTFs.

Technical contributions and context

• The combinatorial lower‑bound parameter provides a new tool for proving impossibility results in Valiant’s model, extending beyond the previously used information‑theoretic arguments.
• The positive result demonstrates that the choice of loss function is crucial: while Valiant’s original definition employs 0‑1 loss, switching to a smooth loss such as quadratic loss enables monotone evolution even under the strict mutation constraints.
• The algorithm’s monotonicity improves upon earlier works where monotone evolvability was shown only for conjunctions with quadratic loss (Feldman, 2009) or under highly restricted distributions (Michael, 2007; Kanade et al., 2010).
• The paper bridges a gap between evolutionary learning and classic margin‑based learning theory, showing that large‑margin LTFs behave similarly to perceptrons under gradient‑like updates, even when the updates are constrained to the limited mutation set of Valiant’s framework.

Implications and future directions
The negative result settles the conjecture that conjunctions could be universally evolvable, indicating that any future positive evolvability results must either relax the mutation model, restrict the distribution, or consider alternative concept classes. The positive result suggests a broader research agenda: exploring which other function classes become evolvable when equipped with a suitable margin condition and a smooth loss. Extending the combinatorial parameter to decision lists, neural networks, or deeper Boolean circuits could yield a systematic classification of evolvability across the landscape of concept classes. Moreover, investigating how biologically realistic mutation operators (e.g., crossover, insertion, deletion) interact with smooth loss functions may bring the theoretical model closer to actual evolutionary processes.

In summary, the paper delivers a decisive dichotomy: conjunctions are provably not distribution‑independent evolvable, while linear threshold functions with a non‑negligible margin are distribution‑independent evolvable via a simple, monotone mutation algorithm that relies on a non‑linear loss such as the quadratic loss. This advances our theoretical understanding of Valiant’s evolutionary framework and opens new avenues for studying evolvability under realistic loss functions and margin conditions.