Learning to Separate RF Signals Under Uncertainty: Detect-Then-Separate vs. Unified Joint Models

The increasingly crowded radio frequency (RF) spectrum forces communication signals to coexist, creating heterogeneous interferers whose structure often departs from Gaussian models. Recovering the interference-contaminated signal of interest in such settings is a central challenge, especially in single-channel RF processing. Existing data-driven methods often assume that the interference type is known, yielding ensembles of specialized models that scale poorly with the number of interferers. We show that detect-then-separate (DTS) strategies admit an analytical justification: within a Gaussian mixture framework, a plug-in maximum a posteriori detector followed by type-conditioned optimal estimation achieves asymptotic minimum mean-square error optimality under a mild temporal-diversity condition. This makes DTS a principled benchmark, but its reliance on multiple type-specific models limits scalability. Motivated by this, we propose a unified joint model (UJM), in which a single deep neural architecture learns to jointly detect and separate when applied directly to the received signal. Using tailored UNet architectures for baseband (complex-valued) RF signals, we compare DTS and UJM on synthetic and recorded interference types, showing that a capacity-matched UJM can match oracle-aided DTS performance across diverse signal-to-interference-and-noise ratios, interference types, and constellation orders, including mismatched training and testing type-uncertainty proportions. These findings highlight UJM as a scalable and practical alternative to DTS, while opening new directions for unified separation under broader regimes.

💡 Research Summary

The paper tackles the problem of recovering a desired single‑channel RF signal of interest (SOI) when it is corrupted by one of several possible interference types whose identity is unknown at runtime. Traditional data‑driven approaches train a dedicated separation model for each interference class, which does not scale with the number of interferers and requires class‑specific labeled data. The authors formalize this “interference‑type uncertainty” within a Bayesian framework and compare two strategies: (1) Detect‑Then‑Separate (DTS) and (2) a Unified Joint Model (UJM) that jointly detects and separates using a single deep neural network.

Theoretical contribution.
Assuming a Gaussian mixture model for the SOI and each interference class, the authors derive the MMSE estimator as a weighted sum of class‑conditioned linear MMSE (LMMSE) estimators, where the weights are the posterior probabilities P(k|y). They prove that under a “Temporal‑Diversity Condition” (TDC) – essentially requiring enough samples so that the MAP detector can reliably identify the interference class – a plug‑in DTS scheme (MAP detector followed by the corresponding LMMSE estimator) asymptotically achieves the same mean‑square error as the optimal MMSE estimator. This establishes DTS as a principled benchmark, rather than a heuristic.

Model designs.
For DTS, K separate UNet‑style complex‑valued networks are trained, each on mixtures generated with a single interference type. At inference, a classifier (or oracle in the experiments) predicts the interference class and the corresponding UNet is applied. For UJM, a single UNet with the same overall capacity as the entire DTS ensemble is trained on mixtures drawn from all interference classes, thereby learning to implicitly infer the class and perform separation in one forward pass.

Experiments.
The authors evaluate both approaches on synthetic data covering a wide range of SIR (‑10 dB to 0 dB), SNR (10–30 dB), and modulation orders (QPSK up to 256‑QAM), as well as on recorded real‑world RF recordings (Wi‑Fi, Bluetooth, ZigBee). Performance metrics include waveform MSE and bit‑error‑rate (BER). Key findings:

• When the DTS detector is given oracle knowledge of the interference class, its separation performance matches that of the UJM when the latter’s capacity is matched to the total DTS parameter count.
• Even with a realistic detector, DTS suffers only a modest degradation, confirming the robustness of the theoretical analysis.
• UJM requires a single set of parameters regardless of the number of interference types, offering a clear scalability advantage.
• In mismatched training/testing class‑distribution scenarios, UJM remains stable, whereas DTS would need retraining of the detector.
• On real recordings, the UJM’s BER is within 0.2 dB of the oracle‑DTS baseline, demonstrating practical viability.

Contributions and implications.

1. A rigorous proof that DTS is asymptotically MMSE‑optimal for Gaussian mixtures, providing a solid theoretical baseline for future work.
2. Introduction of a capacity‑matched UNet‑based UJM that can learn joint detection and separation, eliminating the need for multiple models and class‑specific data.
3. Extensive empirical validation across synthetic and real datasets, showing that a single unified model can achieve parity with an ensemble of specialized models even under high‑order modulation and low‑SIR conditions.

Future directions suggested include extending the analysis to non‑Gaussian, nonlinear interference, designing lightweight online UJM variants for real‑time deployment, and expanding the framework to multi‑antenna or multi‑channel scenarios. Overall, the study demonstrates that unified deep learning architectures can replace the traditional detect‑then‑separate pipeline, offering a scalable and practically effective solution for RF source separation under uncertainty.