Neural Probabilistic Amplitude Shaping for Nonlinear Fiber Channels

We introduce neural probabilistic amplitude shaping, a joint-distribution learning framework for coherent fiber systems. The proposed scheme provides a 0.5 dB signal-to-noise ratio gain over sequence selection for dual-polarized 64-QAM transmission across a single-span 205 km link.

💡 Research Summary

The paper introduces Neural Probabilistic Amplitude Shaping (NP‑PAS), a joint‑distribution learning framework that extends conventional probabilistic amplitude shaping (PAS) to better cope with the nonlinear impairments of coherent optical fiber links while remaining fully compatible with systematic forward error correction (FEC). Traditional PAS only optimizes marginal symbol probabilities, which is insufficient for nonlinear fiber channels where both the individual symbol distribution and the temporal dependencies between symbols affect nonlinear interference (NLIN). Recent sequence‑selection approaches attempt to exploit joint distributions by generating many candidate sequences and selecting the best according to a nonlinear metric, but they suffer from high computational complexity, rate loss, and lack of control over candidate quality.

NP‑PAS addresses these shortcomings by employing an autoregressive recurrent neural network (RNN), specifically a single‑layer long short‑term memory (LSTM) network with hidden size 256, to model the joint distribution of unsigned amplitude symbols. During operation, an arithmetic distribution matcher (ADM) maps information bits to unsigned symbols according to the conditional probabilities produced by the RNN. The sign bits are supplied independently by the FEC parity stream, preserving the standard PAS architecture and enabling seamless integration with existing transceiver pipelines.

Training is performed end‑to‑end over a differentiable optical‑fiber channel model. The authors adopt the additive‑multiplicative (AM) perturbation model, which captures first‑order nonlinear phase rotation and additive Kerr‑induced distortions while remaining differentiable. Discrete symbol samples are drawn using the Gumbel‑Softmax trick with a straight‑through estimator, allowing gradients to flow through the sampling operation. After propagation through the channel model, a mismatched Gaussian demapper computes log‑likelihood ratios (LLRs), which are compared to the transmitted bits using an adjusted binary cross‑entropy (BCE) loss that serves as a surrogate for maximizing achievable information rate (AIR).

The simulation setup mirrors that of prior work: a 205 km single‑span, single‑mode fiber link carrying five wavelength‑division‑multiplexed (WDM) dual‑polarized 64‑QAM channels at 50 GBd with 17 ps/nm/km chromatic dispersion, nonlinear coefficient γ = 1.3 W⁻¹km⁻¹, 0.2 dB/km attenuation, and post‑span erbium‑doped fiber amplifiers (EDFAs) with 5 dB noise figure. Root‑raised‑cosine pulse shaping (roll‑off = 0.1), electronic dispersion compensation, and pilot‑aided linear carrier‑phase recovery (2.5 % pilot rate) are applied. The authors evaluate block lengths L ranging from 4 to 64, concatenating neighboring blocks to capture the effective channel memory introduced by nonlinear propagation.

Results show that for very short blocks (L ≤ 8) the previously proposed Neural PAS (NPS), which learns joint distributions over signed symbols, slightly outperforms NP‑PAS because it directly optimizes the full constellation. However, as L increases, NPS’s performance plateaus and eventually degrades, whereas NP‑PAS continues to improve, achieving comparable or better AIR and effective SNR. The advantage is attributed to NP‑PAS’s reduced optimization space (only the amplitude component), which yields more stable training in high‑dimensional settings. At the optimal launch power, NP‑PAS and NPS both surpass conventional PAS with enumerative sphere shaping (ESS) and ESS combined with sequence selection by more than 0.5 dB in effective SNR and 0.1 bits/2D in AIR, while also operating at a launch power roughly 0.5 dB higher—evidence of superior NLIN mitigation through learned temporal amplitude correlations.

The authors acknowledge that the presented gains are demonstrated on a single‑span link; extending the analysis to multi‑span transmission and real‑world hardware implementations remains future work. Nonetheless, NP‑PAS offers a practical pathway to embed joint‑distribution shaping within the existing PAS framework, preserving systematic FEC compatibility and avoiding the rate penalties and computational overhead associated with candidate‑based sequence selection. This makes NP‑PAS a promising candidate for next‑generation coherent optical transceivers seeking higher spectral efficiency and robustness against fiber nonlinearity.