Information-Theoretic Inequalities on Unimodular Lie Groups

Classical inequalities used in information theory such as those of de Bruijn, Fisher, and Kullback carry over from the setting of probability theory on Euclidean space to that of unimodular Lie groups. These are groups that posses integration measures that are invariant under left and right shifts, which means that even in noncommutative cases they share many of the useful features of Euclidean space. In practical engineering terms the rotation group and Euclidean motion group are the unimodular Lie groups of most interest, and the development of information theory applicable to these Lie groups opens up the potential to study problems relating to image reconstruction from irregular or random projection directions, information gathering in mobile robotics, satellite attitude control, and bacterial chemotaxis and information processing. Several definitions are extended from the Euclidean case to that of Lie groups including the Fisher information matrix, and inequalities analogous to those in classical information theory are derived and stated in the form of fifteen small theorems. In all such inequalities, addition of random variables is replaced with the group product, and the appropriate generalization of convolution of probability densities is employed.

💡 Research Summary

The paper “Information‑Theoretic Inequalities on Unimodular Lie Groups” extends a core set of information‑theoretic results—de Bruijn’s identity, Fisher information inequalities, Kullback‑Leibler divergence properties, and the entropy power inequality—to the setting of unimodular Lie groups. Unimodular groups possess a Haar measure that is invariant under both left and right translations, which makes them the natural non‑commutative analogue of Euclidean space for probability theory. The authors begin by defining probability densities with respect to the normalized Haar measure and introduce a convolution operation based on the group product. Because the groups may be non‑abelian, left‑ and right‑convolutions are distinguished, and the order of multiplication matters.

A central contribution is the definition of a Fisher information matrix on a Lie group. Instead of ordinary partial derivatives, the authors use left‑invariant (or right‑invariant) vector fields to differentiate the log‑density, yielding a matrix that lives in the Lie algebra and respects the group’s geometry. This matrix reduces to the classical Fisher information when the group is ℝⁿ, but for groups such as SO(3) it captures information about angular velocity and orientation uncertainty.

Using the heat kernel on the group, the authors prove a de Bruijn‑type identity: the time derivative of the differential entropy of a density evolving under the group’s heat flow equals the trace of the Fisher information matrix. This links entropy growth to the geometry‑induced diffusion on the manifold.

The paper then presents fifteen compact theorems that parallel classical inequalities. Highlights include:

1. Monotonicity of Fisher information under convolution (the information never increases when independent random elements are combined via the group product).
2. Entropy increase for convolutions, showing that the differential entropy of the product distribution is at least as large as each factor’s entropy.
3. Kullback‑Leibler triangle inequality adapted to group convolution, providing a bound on the divergence between a convolved density and a third reference density.
4. Entropy Power Inequality (EPI) for unimodular groups, where the “entropy power” is defined using the Lie‑algebra norm (the natural quadratic form on the algebra). The inequality states that the entropy power of the product distribution dominates the sum of the individual powers.
5. Cramér‑Rao bound in the group context, establishing a lower bound on the covariance of any unbiased estimator of a group parameter in terms of the inverse Fisher information matrix defined on the algebra.

Each theorem is accompanied by precise regularity assumptions (smoothness, finite second moments, normalization of the Haar measure) and a sketch of the proof that relies on the representation theory of the group, the properties of the Laplace–Beltrami operator, and the invariance of the Haar measure.

To illustrate the abstract results, the authors specialize the theory to two groups of primary engineering interest: the rotation group SO(3) and the Euclidean motion group SE(3). For SO(3) they construct a Gaussian‑like kernel using the exponential map and compute explicit forms of entropy, Fisher information, and the associated inequalities. For SE(3) they treat the translational part with ordinary Euclidean tools while handling the rotational component with the SO(3) machinery, thereby obtaining a unified framework for pose uncertainty.

The paper discusses several concrete applications. In tomographic imaging, random projection directions can be modeled as random elements of SO(3); the derived entropy and Fisher bounds quantify the information loss due to irregular sampling. In mobile robotics, the pose of a robot evolves on SE(3) and sensor measurements are naturally convolved with motion noise; the group‑based Fisher information guides optimal sensor placement and motion planning. Satellite attitude control benefits from the entropy power inequality on SO(3) to design control laws that minimize orientation uncertainty. Finally, the authors mention bacterial chemotaxis, where the stochastic reorientation of cells can be viewed as a random walk on a rotation group, allowing the same information‑theoretic tools to assess signal processing capabilities of microorganisms.

In conclusion, the work provides a rigorous and comprehensive translation of fundamental information‑theoretic inequalities into the language of unimodular Lie groups. It demonstrates that, despite non‑commutativity, the essential relationships between entropy, Fisher information, and convolution survive when the appropriate geometric structures (Haar measure, invariant vector fields, Lie algebra norms) are employed. The authors suggest future directions such as extending the framework to non‑unimodular groups, exploring information‑geometric distances on manifolds, and validating the theoretical predictions with experimental data from robotics and aerospace systems.