Constellation Design for Robust Interference Mitigation

This paper investigates symbol detection for single-carrier communication systems operating in the presence of additive interference with Nakagami-m statistics. Such interference departs from the assumptions underlying conventional detection methods based on Gaussian noise models and leads to detection mismatch that fundamentally affects symbol-level performance. In particular, the presence of random interference amplitude and non-uniform phase alters the structure of the optimal decision regions and renders standard Euclidean distance-based detectors suboptimal. To address this challenge, we develop the maximum-likelihood Gaussian-phase approximate (ML-G) detector, a low-complexity detection rule that accurately approximates maximum-likelihood detection while remaining suitable for practical implementation. The proposed detector explicitly incorporates the statistical properties of the interference and induces decision regions that differ significantly from those arising under conventional metrics. Building on the ML-G framework, we further investigate constellation design under interference-aware detection and formulate an optimization problem that seeks symbol placements that minimize the symbol error probability subject to an average energy constraint. The resulting constellations are obtained numerically and adapt to the interference environment, exhibiting non-standard and asymmetric structures as interference strength increases. Simulation results demonstrate clear symbol error probability gains over established benchmark schemes across a range of interference conditions, particularly in scenarios with dominant additive interference.

💡 Research Summary

The paper tackles the problem of symbol detection and constellation design for single‑carrier wireless links that operate under strong, structured interference whose amplitude and phase follow a Nakagami‑m envelope‑phase distribution. Conventional receivers assume additive white Gaussian noise (AWGN) or treat interference as a deterministic amplitude with uniformly distributed phase; such simplifications ignore the joint randomness of amplitude and phase that arises from realistic propagation (multipath fading, angular dispersion). As a result, Euclidean‑distance‑based detectors become mismatched, leading to degraded symbol error probability (SEP).

The authors first model the received baseband sample as
 Y = √S·X + I + N,
where X is the transmitted symbol (unit average energy), N ~ CN(0,1) is thermal noise, and I = A e^{jΘ} is the interference. The envelope A follows a Nakagami‑m distribution with shape m and spread Ω, while the phase Θ follows a corresponding Nakagami‑m phase law. This joint model captures both amplitude fading and non‑uniform phase dispersion.

Using the maximum‑likelihood (ML) principle, the exact likelihood f(y|x) requires averaging over A and Θ. The amplitude integral can be solved in closed form, but the phase integral is analytically intractable. The authors therefore approximate the phase distribution by a Gaussian with mean π and variance σ²_Θ obtained via moment‑matching. This approximation is justified in Appendix A and shown to closely match the true distribution.

With the Gaussian phase approximation, the likelihood becomes a series expression:

 S(r,ϕ) = I₀,m(r) + Σ_{k=1}^{∞} w_k(ϕ) I_{k,m}(r),

where r = |y−√S x|, ϕ = arg(y−√S x), w_k(ϕ) = (−1)^k e^{−k²/(2σ²_Θ)} cos(kϕ), and I_{k,m}(r) involves Gamma functions and the confluent hypergeometric function 1F1. The series converges rapidly because w_k(ϕ) decays exponentially with k; in practice only 5–7 terms are needed to achieve an approximation error below 10⁻³ for the design region of interest.

The resulting detector, named the maximum‑likelihood Gaussian‑phase approximate (ML‑G) rule, is

 \hat{x}{ML‑G}(y) = arg min{x∈C}