Generalized Measures of Information Transfer

Transfer entropy provides a general tool for analyzing the magnitudes and directions—but not the \emph{kinds}—of information transfer in a system. We extend transfer entropy in two complementary ways. First, we distinguish state-dependent from state-independent transfer, based on whether a source’s influence depends on the state of the target. Second, for multiple sources, we distinguish between unique, redundant, and synergistic transfer. The new measures are demonstrated on several systems that extend examples from previous literature.

💡 Research Summary

The paper “Generalized Measures of Information Transfer” expands the classic Transfer Entropy (TE) framework in two complementary directions, providing a richer taxonomy of information flow in complex systems. First, the authors decompose TE between a single source Y and a target X into two distinct components: State‑Independent Transfer Entropy (SITE) and State‑Dependent Transfer Entropy (SDTE). Using the Partial Information (PI) decomposition, TE = I(X_{t+1};Y_t | X_t) is expressed as the sum of (i) the unique information that Y_t provides about X_{t+1} irrespective of X_t (SITE) and (ii) the synergistic information that emerges only when Y_t is combined with the current state of X_t (SDTE). This separation aligns with control‑theoretic concepts: SITE corresponds to open‑loop control where the controller acts independently of the system’s state, while SDTE corresponds to closed‑loop control where the controller’s effect depends on the current state. The authors prove that perfect controllability with open‑loop control is equivalent to maximal SITE, and that SDTE quantifies the additional contribution of closed‑loop mechanisms.

Second, the paper addresses multivariate information transfer from multiple sources (Y, Z) to a target X. Traditional multivariate TE simply conditions on other sources, which inadvertently removes shared information but also injects synergistic contributions, leading to ambiguous interpretations. By applying PI‑decomposition to the conditional mutual information I(X_{t+1};Y_t, Z_t | X_t), the authors define four quantities: (a) Redundant transfer T_{Y}{Z}→X = I_min, the minimum specific information that either source provides about each target state; (b) Unique transfer from Y (or Z), T_{Y→X\Z} = T_{Y→X} – T_{Y}{Z}→X, representing influence that can only be attributed to that source; (c) Synergistic transfer T_{Y,Z}→X = T_{Y,Z→X} – I_max, capturing information that emerges only when the sources act jointly; and (d) the total multivariate TE, T_{Y,Z→X} = I(X_{t+1};Y_t, Z_t | X_t). These measures disentangle situations where apparent influence from several sources is actually due to a single dominant driver (redundancy), where each source contributes distinct information (uniqueness), or where the sources cooperate to produce new information (synergy).

The theoretical developments are illustrated with several examples. A binary Markov pair (X, Y) with a coupling parameter d demonstrates a smooth transition from pure SITE (d=0) to pure SDTE (d=1), highlighting how the same system can shift between open‑ and closed‑loop regimes. A three‑process binary system (X, Y, Z) with parameters c and d showcases regimes of pure unique transfer (c=0, d=0), pure redundant transfer (c=1, d=0), and pure synergistic transfer (c=1, d=1). These synthetic cases validate that the proposed metrics correctly identify the underlying information‑flow structure.

Finally, the authors apply the framework to real physiological data recorded from a sleep‑apnea patient: breath volume, heart rate, and blood oxygen concentration. Conventional analyses had compared TE and time‑delayed mutual information (TDMI) between breath and heart signals, but these measures conflate SITE, SDTE, and shared history effects. By computing SITE and SDTE separately, the authors find that SITE is essentially zero in both directions, indicating that the observed TE is driven entirely by state‑dependent effects. Plotting TE as a function of the target breath state reveals a bimodal distribution, confirming strong SDTE when the chest volume is low or high and minimal influence near the mean. Extending the analysis to the joint influence of heart rate and blood oxygen on breathing, the unique transfer from heart rate dominates, while redundant and synergistic components are of comparable magnitude, and the unique contribution from oxygen is negligible. This suggests that oxygen’s apparent effect on breathing is mediated through heart rate.

In summary, the paper provides a principled extension of Transfer Entropy that (i) distinguishes state‑independent from state‑dependent information transfer, linking these concepts to open‑ and closed‑loop controllability, and (ii) decomposes multivariate information flow into unique, redundant, and synergistic components. The methodology clarifies the limitations of earlier measures such as TDMI, offers deeper insight into causal interactions in complex systems, and demonstrates practical utility in both synthetic models and real physiological recordings.