Robust Design for Multi-Antenna LEO Satellite Communications with Fractional Delay and Doppler Shifts: An RSMA-OTFS Approach

Low-Earth-orbit (LEO) satellite communication systems face challenges due to high satellite mobility, which hinders the reliable acquisition of instantaneous channel state information at the transmitter (CSIT) and subsequently degrades multi-user transmission performance. This paper investigates a downlink multi-user multi-antenna system, and tackles the above challenges by introducing orthogonal time frequency space (OTFS) modulation and rate-splitting multiple access (RSMA) transmission. Specifically, OTFS enables stable characterization of time-varying channels by representing them in the delay-Doppler domain. However, realistic propagation introduces various inter-symbol and inter-user interference due to non-orthogonal yet practical rectangular pulse shaping, fractional delays, Doppler shifts, and imperfect (statistical) CSIT. In this context, RSMA offers promising robustness for interference mitigation and CSIT imperfections, and hence is integrated with OTFS to provide a comprehensive solution. A compact cross-domain input-output relationship for RSMA-OTFS is established, and an ergodic sum-rate maximization problem is formulated and solved using a weighted minimum mean-square-error based alternating optimization algorithm that does not depend on channel sparsity. Simulation results reveal that the considered practical propagation effects significantly degrade performance if unaddressed. Furthermore, the RSMA-OTFS scheme demonstrates improved ergodic sum-rate and robustness against CSIT uncertainty across various user deployments and CSIT qualities.

💡 Research Summary

This paper addresses two fundamental challenges inherent to low‑Earth‑orbit (LEO) satellite communications: the rapid time‑frequency variability caused by high orbital velocities and the resulting difficulty of obtaining accurate instantaneous channel state information at the transmitter (CSIT). To mitigate these issues, the authors propose a novel downlink transmission framework that jointly employs orthogonal time‑frequency space (OTFS) modulation and rate‑splitting multiple access (RSMA).

OTFS maps information symbols onto the delay‑Doppler (DD) domain, where a doubly‑selective channel appears almost time‑invariant, thereby exploiting the full time‑frequency diversity of the link. However, practical LEO links employ rectangular pulse shaping, fractional (non‑integer) delays and Doppler shifts, and only statistical CSIT is available due to long feedback latency. These realistic impairments break the ideal block‑circulant and sparsity structures that most existing OTFS precoding and detection schemes rely on.

RSMA, on the other hand, splits each user’s message into a common part (decoded by all users) and a private part (decoded only by the intended user). By transmitting a common stream first and then applying successive interference cancellation (SIC), RSMA provides robustness against CSIT errors and can achieve the optimal degrees‑of‑freedom region under partial CSIT. The combination of OTFS and RSMA therefore promises a transmission strategy that is both resilient to fast channel dynamics and tolerant of CSIT uncertainty.

The system model consists of an Nₜ‑antenna uniform linear array on a LEO satellite serving I single‑antenna users. In the DD grid (M delay bins, N Doppler bins) the common symbols s_c and private symbols s_{p,i} are first arranged by binary diagonal “arrangement” matrices Ψ_c and Ψ_{p,i}. Linear precoders p_c and p_{p,i} are then applied, yielding the vectorized DD‑domain transmit signal

x_DD = (p_c ⊗ Ψ_c) s_c + Σ_i (p_{p,i} ⊗ Ψ_{p,i}) s_{p,i}.

Through ISFFT, Heisenberg transform, and rectangular pulse shaping matrices G_tx/G_rx, the DD signal is converted to the time‑domain vector x_TD, which passes through the physical channel H_TD,i. After receive‑side filtering and SFFT, the received DD‑domain vector for user i can be expressed as

y_DD,i = ˜H_DD,i ˜P_c s_c + Σ_i ˜H_DD,i ˜P_{p,i} s_{p,i} + n_i,

where ˜H_DD,i = (F_N ⊗ I_M) H_TD,i captures the combined effect of fractional delays/Dopplers and rectangular pulses, and ˜P_j = p_j ⊗ ˜Ψ_j denotes the effective precoding matrix in the time domain.

The design objective is to maximize the ergodic sum‑rate (ESR) under a total transmit‑power constraint, while accounting for statistical CSIT. The optimization variables include the common and private precoders, power‑allocation coefficients, the common‑vs‑private split ratios, and the arrangement matrices. Because the problem is non‑convex and involves expectations over the CSIT distribution, the authors employ sample‑average approximation (SAA) to replace expectations with empirical averages, and then transform the ESR maximization into an equivalent weighted minimum‑mean‑square‑error (WMMSE) minimization problem.

A block‑coordinate alternating optimization (AO) algorithm is proposed: (1) with fixed precoders and arrangement matrices, optimal MMSE receivers and weighting matrices are updated; (2) with receivers and weights fixed, the precoders and power splits are optimized via convex sub‑problems; (3) the arrangement matrices are refined using a projected gradient step. This cycle repeats until convergence. Notably, the algorithm does not rely on DD‑domain sparsity, making it applicable to the realistic channel model where fractional effects destroy the usual block‑circulant structure.

Simulation results consider a range of practical scenarios: (i) integer versus fractional delay/Doppler, (ii) rectangular versus ideal bi‑orthogonal pulse shaping, and (iii) CSIT error variances from perfect to severe (e.g., σ_e² = –10 dB). The benchmark schemes include OFDM‑RSMA, OTFS‑SDMA with perfect CSIT, OTFS‑NOMA, and an idealized OTFS‑RSMA assuming integer shifts. Key findings are:

• Fractional delays/Dopplers and rectangular pulses cause a noticeable ESR loss for all baseline schemes; the proposed RSMA‑OTFS recovers most of this loss, showing only a 5–8 % degradation compared to the ideal case.
• As CSIT quality deteriorates, conventional SDMA and NOMA experience steep ESR drops (up to 40 % at σ_e² = –10 dB), whereas RSMA‑OTFS maintains a graceful performance decline (≈10 % loss), thanks to the common stream that absorbs residual interference.
• Joint optimization of precoders and arrangement matrices yields an additional 15–25 % ESR improvement over schemes that only optimize precoders, especially when the satellite is equipped with 4–8 antennas.

The paper’s contributions are threefold: (1) a comprehensive multi‑antenna RSMA‑OTFS transmission framework that incorporates realistic propagation effects (fractional delay/Doppler, non‑orthogonal pulse shaping) and statistical CSIT; (2) derivation of a compact cross‑domain input‑output relationship and formulation of a robust ESR maximization problem that jointly optimizes precoding, power allocation, message splitting, and DD‑domain symbol placement; (3) development of a WMMSE‑based AO algorithm that solves the non‑convex problem without relying on DD‑domain sparsity.

Overall, the study demonstrates that integrating RSMA with OTFS provides a powerful solution for LEO satellite downlinks, delivering higher average sum‑rates and superior robustness to CSIT uncertainty than existing OFDM‑based or OTFS‑only designs. The work paves the way for practical implementation of high‑throughput, interference‑aware satellite communications in future mega‑constellations, and suggests further research directions such as low‑complexity detection, multi‑satellite cooperation, and dynamic user scheduling under the same robust design philosophy.