Labeled compression schemes for extremal classes

It is a long-standing open problem whether there always exists a compression scheme whose size is of the order of the Vapnik-Chervonienkis (VC) dimension $d$. Recently compression schemes of size exponential in $d$ have been found for any concept class of VC dimension $d$. Previously, compression schemes of size $d$ have been given for maximum classes, which are special concept classes whose size equals an upper bound due to Sauer-Shelah. We consider a generalization of maximum classes called extremal classes. Their definition is based on a powerful generalization of the Sauer-Shelah bound called the Sandwich Theorem, which has been studied in several areas of combinatorics and computer science. The key result of the paper is a construction of a sample compression scheme for extremal classes of size equal to their VC dimension. We also give a number of open problems concerning the combinatorial structure of extremal classes and the existence of unlabeled compression schemes for them.

💡 Research Summary

The paper addresses a central open problem in learning theory: whether every concept class admits a sample compression scheme whose size depends only on its Vapnik‑Chervonenkis (VC) dimension d. While exponential‑size schemes are known for arbitrary classes, optimal schemes of size d have been constructed only for maximum classes—those whose cardinality meets the Sauer‑Shelah bound with equality. The authors introduce a broader family called extremal (or shattering‑extremal) classes. An extremal class C satisfies the “Sandwich Theorem” with equality, i.e., the number of shattered sets equals the number of strongly shattered sets, which implies that every shattered set is the dimension set of a full‑dimensional cube inside C. All maximum classes are extremal, but many extremal classes are not maximum, illustrating that extremality captures a richer combinatorial structure.

The main contribution is a constructive labeled compression scheme for any extremal class of VC dimension d that compresses any realizable sample to a subsample of exactly d points. The scheme proceeds by iteratively selecting dimensions (features) based on the sample and the cube structure of the class. At each step a “reduction” operation removes a chosen dimension while preserving extremality; the selected points form the compressed representation. The analysis relies on down‑shifting arguments: repeated down‑shifts transform any extremal class into a downward‑closed class where the cardinalities of the class, its shattered sets, and its strongly shattered sets coincide. In this canonical form the compression bound follows directly, and the construction can be traced back to the original class via the inverse of the reductions. When applied to a maximum class, the scheme coincides with the classical compression scheme of Floyd and Warmuth (1995), showing that the new method truly generalizes the known optimal result.

Beyond labeled compression, the authors discuss the existence of unlabeled compression schemes for extremal classes. For maximum classes, greedy “peeling” procedures yield unlabeled schemes (Kuzmin & Warmuth 2007; Rubinstein & Rubinstein 2012). Whether a similar unlabeled scheme exists for all extremal classes remains open. The paper relates this question to structural properties such as the existence of minimal cube decompositions and specific graph‑theoretic characteristics of the one‑inclusion graph. Several concrete open problems are listed, inviting further combinatorial investigation.

In summary, the work expands the frontier of optimal sample compression from the narrow class of maximum concepts to the substantially larger family of extremal classes, providing a simple yet technically sophisticated construction. It bridges combinatorial geometry (cubes, reductions, down‑shifts) with learning‑theoretic notions (VC dimension, compression), and opens new avenues for research on unlabeled compression and the deeper combinatorial anatomy of extremal concept classes.