Information Abstraction for Data Transmission Networks based on Large Language Models

Biological systems, particularly the human brain, achieve remarkable energy efficiency by abstracting information across multiple hierarchical levels. In contrast, modern artificial intelligence and communication systems often consume significant energy overheads in transmitting low-level data, with limited emphasis on abstraction. Despite its implicit importance, a formal and computational theory of information abstraction remains absent. In this work, we introduce the Degree of Information Abstraction (DIA), a general metric that quantifies how well a representation compresses input data while preserving task-relevant semantics. We derive a tractable information-theoretic formulation of DIA and propose a DIA-based information abstraction framework. As a case study, we apply DIA to a large language model (LLM)-guided video transmission task, where abstraction-aware encoding significantly reduces transmission volume by $99.75%$, while maintaining semantic fidelity. Our results suggest that DIA offers a principled tool for rebalancing energy and information in intelligent systems and opens new directions in neural network design, neuromorphic computing, semantic communication, and joint sensing-communication architectures.

💡 Research Summary

The paper tackles a fundamental inefficiency in modern AI‑enabled communication systems: the transmission of low‑level data that consumes orders of magnitude more energy than local computation. Inspired by the human brain, which achieves remarkable energy efficiency by abstracting sensory inputs into high‑level representations, the authors propose a formal, quantitative theory of information abstraction. Their central contribution is the Degree of Information Abstraction (DIA), a metric that simultaneously captures (i) how much the data has been compressed and (ii) how well the semantics of the original signal are preserved.

Formally, DIA is defined as Γ = C·Θ, where C = 1 − H(Y)/H(X) measures compression using Shannon entropy of the original data X and its abstracted representation Y, and Θ = 1/D_KL( \hat X_S ‖ \hat Y_S ) quantifies semantic preservation by projecting X and Y into a shared latent space S (via modality‑specific encoders) and computing the KL divergence between the resulting empirical distributions. A higher Γ indicates a more desirable balance between aggressive compression and faithful semantics.

The authors relate DIA to the classic Information Bottleneck (IB) framework. By assuming the latent space S is chosen such that KL divergence monotonically reflects the IB relevance term I(T;U), they show that maximizing log Γ is equivalent to minimizing the IB Lagrangian L_IB = I(X;T) − β I(T;U). Crucially, DIA avoids explicit mutual‑information estimation, which is notoriously intractable in high‑dimensional settings, and it naturally accommodates multimodal inputs because the semantic term is defined in the shared space S rather than a joint probability model.

To demonstrate practical impact, the paper applies DIA to a large‑language‑model‑guided video transmission scenario. Raw video frames X are encoded into a sequence of semantic tokens or latent vectors Y by a transmitter LLM. An optimization loop called OPRO (Optimization by Prompting) iteratively refines the encoding parameters via LLM prompts, directly maximizing the DIA objective. Additionally, a Video Semantic Differential Stream (VSDS) module transmits a residual semantic signal alongside the main token stream, improving reconstruction fidelity.

Experimental results on high‑resolution video streams show that DIA‑guided encoding reduces the transmitted data volume by 99.75 % while preserving downstream task performance (object detection, action recognition) at near‑original levels. This validates the hypothesis that transmitting only task‑relevant semantics can dramatically cut bandwidth and energy consumption without sacrificing functional utility.

Beyond the case study, the authors outline broader research avenues: (1) Neuromorphic computing, where DIA could guide the design of spiking networks that allocate spikes preferentially to semantically salient features; (2) Semantic communication, enabling dynamic bit‑allocation protocols that adapt to real‑time DIA scores; (3) Joint sensing‑communication, leveraging multimodal abstraction to co‑optimize sensor processing and wireless transmission.

The paper also acknowledges limitations. Designing an appropriate latent space S is non‑trivial; it must be modality‑invariant, semantically expressive, and amenable to stable KL estimation from finite samples. Moreover, the current evaluation is confined to video‑text domains, and generalization to other modalities (e.g., LiDAR, medical imaging) remains to be explored.

In summary, the Degree of Information Abstraction provides a theoretically grounded, computationally tractable metric for quantifying how well a representation abstracts raw data while retaining task‑relevant meaning. By integrating DIA with large language models, the authors demonstrate a concrete pathway to dramatically more energy‑efficient communication systems, opening a promising research frontier at the intersection of information theory, deep learning, and sustainable network design.