A General Notion of Useful Information

In this paper we introduce a general framework for defining the depth of a sequence with respect to a class of observers. We show that our general framework captures all depth notions introduced in complexity theory so far. We review most such notions, show how they are particular cases of our general depth framework, and review some classical results about the different depth notions.

💡 Research Summary

The paper proposes a unified framework for measuring the “depth” of a binary or symbolic sequence with respect to a class of observers, thereby offering a general notion of useful information. The authors begin by observing that many depth concepts have been introduced in computational complexity theory—Bennett’s logical depth, computational depth, polynomial depth, randomness depth, and several variants—each defined with respect to a particular model of computation or resource bound. Because these definitions are scattered across different settings, it has been difficult to compare them, to transfer results from one setting to another, or to devise new depth notions for emerging computational paradigms.

To address this fragmentation, the paper introduces the abstract entity of an observer. An observer is any algorithm that, given a finite string x, produces an output (for example, a compressed description, a prediction, or a reconstruction) while respecting a predefined resource limitation (time, space, circuit size, quantum gates, etc.). A class of observers 𝒪 is then a set of such algorithms that share the same resource bound. For a fixed class 𝒪, the depth of a string x, denoted Depth𝒪(x), is defined by comparing the best performance achievable by two observers A, B ∈ 𝒪 on x. Roughly, if A can produce a shorter description of x or can reconstruct x more quickly than B, then x is considered deeper with respect to B than to A. Formally, Depth𝒪(x) captures the trade‑off between description length and the computational effort required to obtain that description within the constraints of 𝒪.

The authors then demonstrate that this abstract definition subsumes all previously studied depth notions. By choosing appropriate observer classes, the framework reproduces:

• Logical depth – 𝒪 consists of all Turing machines with a fixed exponential time bound; Depth𝒪(x) measures the gap between the shortest program and the time needed to run it.
• Computational depth – 𝒪 is the set of all polynomial‑time compressors; Depth𝒪(x) reflects how much extra time is needed to achieve a given compression ratio.
• Polynomial depth – 𝒪 adds a polynomial‑space restriction to the above, yielding a finer granularity.
• Randomness depth – 𝒪 includes statistical tests for randomness; a string that passes many tests but still admits a non‑trivial compression is deemed deep.

Because each concrete depth is a special case of the same definition, the framework yields immediate structural relationships. If observer class 𝒪₁ is a subset of 𝒪₂ (i.e., 𝒪₂ allows strictly more computational power), then for every string x we have Depth𝒪₂(x) ≥ Depth𝒪₁(x). This monotonicity explains, for instance, why polynomial depth is never larger than logical depth. Moreover, the framework makes it trivial to define new depth notions: one merely replaces 𝒪 with a class reflecting quantum circuits, streaming algorithms, or neural‑network predictors, obtaining “quantum depth,” “streaming depth,” etc., without reinventing the underlying theory.

Beyond unification, the paper explores the conceptual link between depth and information theory. Classical Shannon entropy quantifies average uncertainty but ignores the computational effort required to resolve that uncertainty. Depth, by contrast, measures the time‑space resources needed to compress or reconstruct a string, thus capturing a notion of “useful information” that is hidden from purely statistical measures. A string with low entropy but high depth contains structure that is hard to exploit computationally; such strings are precisely those that are valuable in cryptographic keys, scientific data sets, or model parameters in machine learning. The authors argue that depth therefore provides a quantitative handle on the practical usefulness of information.

The final sections discuss open problems and future research directions. Key questions include:

1. Observer selection for applications – How should one choose an observer class that reflects real‑world constraints (e.g., hardware limits, parallelism) when evaluating the depth of data used in databases or compression utilities?
2. Depth versus learnability – What is the precise relationship between a string’s depth and its amenability to PAC‑learning or other statistical learning frameworks?
3. Complexity class separations – Can depth be leveraged to construct new oracle separations or to provide evidence for major open problems such as P vs NP?
4. Algorithmic estimation – Designing practical algorithms that approximate Depth𝒪(x) for natural observer classes remains a challenging task; empirical studies on real data could illuminate the relevance of depth in practice.

In summary, the paper delivers a robust, observer‑based definition of depth that not only unifies all previously known depth notions but also clarifies their interrelationships, extends the concept to new computational models, and positions depth as a meaningful measure of “useful information.” By bridging the gap between abstract complexity theory and concrete informational usefulness, the framework opens a fertile ground for both theoretical exploration and practical applications.