Abstraction Super-structuring Normal Forms: Towards a Theory of Structural Induction

Induction is the process by which we obtain predictive laws or theories or models of the world. We consider the structural aspect of induction. We answer the question as to whether we can find a finite and minmalistic set of operations on structural elements in terms of which any theory can be expressed. We identify abstraction (grouping similar entities) and super-structuring (combining topologically e.g., spatio-temporally close entities) as the essential structural operations in the induction process. We show that only two more structural operations, namely, reverse abstraction and reverse super-structuring (the duals of abstraction and super-structuring respectively) suffice in order to exploit the full power of Turing-equivalent generative grammars in induction. We explore the implications of this theorem with respect to the nature of hidden variables, radical positivism and the 2-century old claim of David Hume about the principles of connexion among ideas.

💡 Research Summary

The paper tackles the structural side of inductive inference, asking whether a finite, minimal set of operations on structural elements can serve as a universal building block for any theory that can be expressed by a Turing‑equivalent generative system. By interpreting theories as generative grammars—terminals representing observable data and non‑terminals representing internal (possibly hidden) variables—the authors map the process of structural induction onto the derivation process of a grammar.

Two primitive operations are identified: Abstraction, which groups similar entities into a single non‑terminal (formalized as a renaming rule A → B), and Super‑structuring, which combines topologically or temporally adjacent entities into a composite structure (rule A → BC). These correspond to the intuitive steps of “chunking” and “generalizing” that a scientist performs when building a model. However, the authors prove that these two alone are insufficient to achieve the full expressive power of unrestricted grammars (or Turing machines). They introduce the dual operations Reverse Abstraction (B → A) and Reverse Super‑structuring (AB → C). The latter compresses two non‑terminals into a new one, effectively allowing the grammar to simulate context‑sensitive productions.

The central technical contribution is a series of Abstraction‑Super‑structuring Normal Forms (ASNF). For context‑free grammars (CFGs) they show a Weak‑CF‑ASNF in which any CFG can be transformed into a grammar containing only three rule types: renamings (A → B), binary compositions (A → BC), and terminal introductions (A → a). By imposing a strong‑uniqueness condition—each left‑hand side and each right‑hand side appears in at most one rule—they obtain a Strong‑CF‑ASNF that eliminates redundancy and makes the search space deterministic.

For unrestricted grammars (General Grammars, GG) they extend the weak normal form with a fourth rule type, Reverse Super‑structuring (AB → C), yielding a Weak‑GEN‑ASNF. Applying the same strong‑uniqueness transformation gives a Strong‑GEN‑ASNF that still retains full Turing‑equivalence. The paper provides constructive proofs (with details relegated to an appendix) showing how any original grammar can be systematically rewritten into one of these normal forms without changing the generated language.

Beyond the formal results, the authors discuss philosophical implications. They argue that the existence of reverse operations legitimizes the use of hidden variables: while positivist traditions demand direct observational grounding, reverse abstraction and reverse super‑structuring demonstrate that hidden constructs can be introduced for explanatory compression and later eliminated without loss of expressive power. This resonates with Carnap’s relaxation of strict positivism and with David Hume’s “principles of connexion,” which the authors reinterpret as the logical underpinnings of super‑structuring (contiguity) and abstraction (similarity).

Practically, the ASNF framework suggests a new paradigm for inductive learning algorithms. Instead of enumerating an infinite space of arbitrary rewrite rules, a learner can restrict its hypothesis space to sequences of the four primitive operations. This reduction transforms theory search into a combinatorial optimization problem amenable to dynamic programming, A* search, or other heuristic methods, promising greater computational efficiency and interpretability.

In conclusion, the paper establishes that a minimal set of four structural operations—abstraction, super‑structuring, and their reverses—suffices to capture any computable theory, provides normal‑form transformations for both CFGs and unrestricted grammars, and links these technical findings to longstanding philosophical debates about hidden variables and the nature of scientific explanation. Future work is suggested to integrate ASNF‑based search into concrete machine‑learning systems and to evaluate its performance on real‑world data.