Towards Optimal Semantic Communications: Reconsidering the Role of Semantic Feature Channels

This paper investigates the optimization of transmitting the encoder outputs, termed semantic features (SFs), in semantic communication (SC). We begin by modeling the entire communication process from the encoder output to the decoder input, encompassing the physical channel and all transceiver operations, as the SF channel, thereby establishing an encoder-SF channel-decoder pipeline. In contrast to prior studies that assume a fixed SF channel, we note that the SF channel is configurable, as its characteristics are shaped by various transmission and reception strategies, such as power allocation. Based on this observation, we formulate the SF channel optimization problem under a mutual information constraint between the SFs and their reconstructions, and analytically derive the optimal SF channel under a linear encoder-decoder structure and Gaussian source assumption. Building upon this theoretical foundation, we propose a joint optimization framework for the encoder-decoder and SF channel, applicable to both analog and digital SCs. To realize the optimized SF channel, we also propose a physical-layer calibration strategy that enables real-time power control and adaptation to varying channel conditions. Simulation results demonstrate that the proposed SF channel optimization achieves superior task performance under various communication environments.

💡 Research Summary

The paper addresses a fundamental limitation of current semantic communication (SC) systems: they treat the transmission medium for semantic features (SFs) as a fixed, pre‑defined channel (e.g., AWGN or Rayleigh) while only optimizing the neural encoder‑decoder pair. The authors instead model the entire path from encoder output to decoder input—including power control, modulation, equalization, and channel fading—as a single “SF channel.” This abstraction reveals that the SF channel is a configurable entity whose characteristics can be shaped by physical‑layer (PHY) parameters.

The central research question is: given a constraint on the mutual information between transmitted and received SFs (I(z; ẑ) ≤ C for analog, I(b; ˆb) ≤ C for digital), what SF‑channel design maximizes the end‑task performance? To answer this, the authors first analyze a tractable case with a linear encoder‑decoder and a Gaussian source. By formulating a Lagrangian with the mutual‑information budget, they derive closed‑form expressions for the optimal per‑dimension noise variance (analog) or bit‑flip probability (digital). The solution shows that each SF dimension should receive a signal‑to‑noise ratio proportional to its allocated portion of the total information budget, which aligns with classical rate‑distortion theory; when the encoder output dimension is sufficiently large, the achievable distortion approaches the rate‑distortion bound.

Building on this insight, the paper proposes an end‑to‑end training framework that jointly optimizes (i) a deep neural network (DNN) encoder‑decoder and (ii) the SF‑channel parameters under a limited mutual‑information budget. In the analog setting, each SF is corrupted by Gaussian noise whose variance is a learnable parameter tied to a power‑allocation coefficient. In the digital setting, each transmitted bit is passed through a binary symmetric channel (BSC) with a learnable flip probability, effectively controlling the modulation order and coding rate. The mutual‑information constraint is treated as a rate‑allocation problem, ensuring that the total allocated rate does not exceed C.

To bridge the gap between the learned SF‑channel and real wireless hardware, the authors introduce a PHY‑calibration strategy. For single‑user analog SC, the strategy computes the required transmit power per feature so that the actual SNR matches the learned variance while minimizing overall power consumption. For multi‑user digital SC, it jointly selects transmit powers and modulation orders across users so that the observed bit‑error rates align with the learned flip probabilities. The calibration can also select among multiple pre‑trained SF‑channel candidates to adapt to changing channel conditions.

Simulation results focus on image reconstruction (CIFAR‑10) under various SNRs and mutual‑information limits. The jointly optimized system consistently outperforms conventional DeepJSCC that assumes a fixed channel, achieving 2–4 dB higher PSNR and significantly lower MSE across the board. Moreover, the PHY‑calibration step successfully reproduces the target SF‑channel characteristics in realistic fading environments, confirming that the theoretical optimum is attainable in practice. In multi‑user experiments, the joint power‑modulation optimization improves overall system throughput while maintaining fairness among users.

In summary, the paper makes four major contributions: (1) formulation of a joint encoder‑decoder and SF‑channel optimization problem with a mutual‑information constraint; (2) analytical derivation of the optimal SF‑channel for a linear Gaussian model and its connection to rate‑distortion theory; (3) a practical end‑to‑end training algorithm for both analog and digital SC that learns per‑feature noise variances or bit‑flip probabilities; and (4) a PHY‑calibration mechanism that implements the learned SF‑channel in real wireless systems. By treating the SF channel as a design variable rather than a static backdrop, the work opens a new pathway toward highly efficient, robust, and adaptable semantic communication systems for future AI‑driven wireless applications.