Ranking-Based Black-Box Complexity

Randomized search heuristics such as evolutionary algorithms, simulated annealing, and ant colony optimization are a broadly used class of general-purpose algorithms. Analyzing them via classical methods of theoretical computer science is a growing field. While several strong runtime analysis results have appeared in the last 20 years, a powerful complexity theory for such algorithms is yet to be developed. We enrich the existing notions of black-box complexity by the additional restriction that not the actual objective values, but only the relative quality of the previously evaluated solutions may be taken into account by the black-box algorithm. Many randomized search heuristics belong to this class of algorithms. We show that the new ranking-based model gives more realistic complexity estimates for some problems. For example, the class of all binary-value functions has a black-box complexity of $O(\log n)$ in the previous black-box models, but has a ranking-based complexity of $\Theta(n)$. For the class of all OneMax functions, we present a ranking-based black-box algorithm that has a runtime of $\Theta(n / \log n)$, which shows that the OneMax problem does not become harder with the additional ranking-basedness restriction.

💡 Research Summary

The paper introduces a new theoretical framework for analyzing randomized search heuristics—such as evolutionary algorithms, simulated annealing, and ant‑colony optimization—by refining the notion of black‑box complexity. Traditional unrestricted black‑box models allow an algorithm to query the exact objective value of any search point, which does not reflect the way most heuristics actually operate: they only compare fitness values and never use the absolute numbers. To capture this reality, the authors define the ranking‑based black‑box model, in which an algorithm, after each query, only learns the relative order of all previously evaluated points (i.e., whether f(x) < f(y), f(x) = f(y), or f(x) > f(y)). No absolute fitness information is available.

The paper studies two well‑known function classes under this model.

1. BinaryValue*ₙ – the set of all functions that compute a weighted sum of bits with weights 2^{i‑1}, possibly permuted and with some bits complemented. In the unrestricted model this class can be optimized with O(log n) queries because the algorithm can directly read the binary representation of the optimum. In the ranking‑based setting, however, the algorithm cannot distinguish the contribution of individual bits from order information alone. The authors prove a lower bound of Ω(n) and present a simple hill‑climber that flips bits one by one, achieving an O(n) upper bound. Thus the ranking‑based complexity of BinaryValue*ₙ is Θ(n), a dramatic increase that aligns better with the difficulty experienced by real heuristics.

2. OneMaxₙ – the class of all functions that count the number of positions where a candidate string matches a hidden target string. Both unrestricted and unbiased black‑box complexities are known to be Θ(n/ log n). The authors show that even when only rankings are available, the same asymptotic bound can be achieved. Their algorithm samples O(n/ log n) random points, uses the ranking of their fitnesses to infer information about each bit, fixes bits whose value becomes certain, and repeats the process on the remaining positions. The analysis demonstrates that the loss of absolute fitness values does not increase the query complexity for OneMax. Consequently, the ranking‑based black‑box complexity of OneMaxₙ remains Θ(n/ log n).

The paper also connects the OneMax result to the black‑peg version of Mastermind, where the codemaker only reports the number of correctly placed colors (black pegs). This is exactly the ranking‑based feedback model, and the authors’ algorithm corresponds to an optimal Mastermind strategy that solves the game in Θ(n/ log n) guesses.

By comparing the ranking‑based model with the previously studied unbiased black‑box model (which restricts variation operators but still allows absolute fitness values), the authors argue that the ranking restriction is more faithful to many practical heuristics. The unbiased model can still exploit exact fitness values, leading to unrealistically low complexity for some function classes. The ranking‑based model eliminates this loophole, yielding more realistic lower bounds, as illustrated by the BinaryValue*ₙ case.

Overall, the contribution is threefold: (i) a formal definition of ranking‑based black‑box complexity, (ii) tight upper and lower bounds for two canonical function classes, demonstrating that the restriction can either increase complexity dramatically (BinaryValue) or leave it unchanged (OneMax), and (iii) a conceptual bridge to classic combinatorial games like Mastermind. The work opens a new line of inquiry for measuring the difficulty of optimization problems under information‑theoretic constraints that match the actual data used by randomized search heuristics. Future research may extend the model to other problem families, incorporate memory limits, or study multi‑objective settings.