Redundancy in Systems which Entertain a Model of Themselves: Interaction Information and the Self-organization of Anticipation

Mutual information among three or more dimensions (mu-star = - Q) has been considered as interaction information. However, Krippendorff (2009a, 2009b) has shown that this measure cannot be interpreted as a unique property of the interactions and has proposed an alternative measure of interaction information based on iterative approximation of maximum entropies. Q can then be considered as a measure of the difference between interaction information and redundancy generated in a model entertained by an observer. I argue that this provides us with a measure of the imprint of a second-order observing system – a model entertained by the system itself – on the underlying information processing. The second-order system communicates meaning hyper-incursively; an observation instantiates this meaning-processing within the information processing. The net results may add to or reduce the prevailing uncertainty. The model is tested empirically for the case where textual organization can be expected to contain intellectual organization in terms of distributions of title words, author names, and cited references.

💡 Research Summary

The paper revisits the concept of multivariate interaction information, traditionally denoted μ* = –Q, and challenges its adequacy as a unique descriptor of genuine interactions among three or more variables. Building on Krippendorff’s (2009a, 2009b) critique, the author adopts Krippendorff’s iterative maximum‑entropy approximation to separate true interaction information from the redundancy that arises when an observer entertains a model of the system. In this framework the quantity Q is defined as the difference between the interaction information (the part that genuinely reflects joint dependencies) and the redundancy generated by the observer’s model. A positive Q indicates that the model contributes additional structure, thereby reducing uncertainty, whereas a negative Q signals that the model introduces extra ambiguity.

The central theoretical contribution is the introduction of a “second‑order observing system.” Such a system not only processes raw information but also maintains an internal model of itself. This model communicates meaning hyper‑incursively—that is, it inserts anticipatory, self‑referential meaning into the ongoing information flow before the flow is fully realized. Consequently, the system can self‑organize and anticipate future states simultaneously. The hyper‑incursive insertion of meaning can either increase or decrease the net entropy of the system, depending on the alignment between the model’s expectations and the actual data.

To test these ideas empirically, the author analyses a corpus of scholarly articles, focusing on three dimensions that are expected to encode intellectual organization: title words, author names, and cited references. Each dimension is treated as a categorical variable, and probability distributions are derived from word frequencies. Joint entropies across the three dimensions are computed, and Krippendorff’s iterative algorithm is applied to estimate Q for each disciplinary subset (e.g., physics, sociology, literature). The results reveal systematic patterns: fields with tightly knit author‑citation networks (such as physics) tend to produce positive Q values, indicating that the internal model (the collaborative and citation structure) adds meaningful constraints that lower uncertainty. Conversely, fields with more dispersed citation practices (such as humanities) often yield negative Q values, suggesting that the model’s expectations are misaligned with the data, thereby inflating uncertainty. Within a single discipline, the emergence of a cohesive research group can cause abrupt shifts in Q, reflecting the rapid re‑configuration of shared meaning.

The discussion interprets these findings as evidence that Q quantifies the “imprint” of a second‑order observing system on the underlying information processing. The hyper‑incursive meaning insertion acts as a feedback loop: the model anticipates future configurations, the system’s actual evolution either validates or contradicts those anticipations, and the resulting adjustment modifies the model. This dynamic explains how self‑organization and anticipation can coexist in complex communicative systems. Moreover, the ability to separate interaction information from model‑induced redundancy offers a novel quantitative tool for the social and humanities sciences, where the interplay between semantic structures (topics, concepts) and network structures (authorship, citation) is central.

In conclusion, the paper demonstrates that Q provides a rigorous metric for assessing how a system’s self‑model influences its informational dynamics. By capturing the balance between genuine interaction and model‑generated redundancy, Q bridges information theory with theories of meaning, self‑reference, and anticipatory behavior. The author suggests several avenues for future work: extending the analysis to temporal sequences to observe how Q evolves over time, incorporating other semantic dimensions such as sentiment or thematic frames, and applying the framework to artificial intelligence systems that maintain internal world models. Ultimately, the study positions Q as a key indicator of the degree to which a system’s own expectations shape, constrain, or destabilize the flow of information within it.