Novel Blind Signal Classification Method Based on Data Compression

This paper proposes a novel algorithm for signal classification problems. We consider a non-stationary random signal, where samples can be classified into several different classes, and samples in each class are identically independently distributed with an unknown probability distribution. The problem to be solved is to estimate the probability distributions of the classes and the correct membership of the samples to the classes. We propose a signal classification method based on the data compression principle that the accurate estimation in the classification problems induces the optimal signal models for data compression. The method formulates the classification problem as an optimization problem, where a so called {“classification gain”} is maximized. In order to circumvent the difficulties in integer optimization, we propose a continuous relaxation based algorithm. It is proven in this paper that asymptotically vanishing optimality loss is incurred by the continuous relaxation. We show by simulation results that the proposed algorithm is effective, robust and has low computational complexity. The proposed algorithm can be applied to solve various multimedia signal segmentation, analysis, and pattern recognition problems.

💡 Research Summary

The paper introduces a novel blind signal classification algorithm that leverages the principle of data compression. The authors consider a non‑stationary random signal whose samples belong to an unknown number of classes; within each class the samples are i.i.d. but the class‑specific probability distributions are completely unknown. The central idea is that an accurate classification yields the most efficient compression model for the data, and conversely, a model that maximizes compression efficiency implicitly discovers the correct class structure.

To formalize this intuition the authors define a “classification gain” (G) as the reduction in entropy achieved by modeling the data with class‑specific distributions instead of a single global model:
 G = H(X) – Σ_c p_c H(X|c).
Here H(X) is the entropy of the whole dataset, p_c is the prior probability of class c, and H(X|c) is the entropy under the class‑specific model. Maximizing G therefore simultaneously improves compression and encourages a partition of the data that reflects true statistical differences among classes.

Directly maximizing G is a mixed integer‑continuous optimization problem because the class assignments are integer variables while the distribution parameters are continuous. Solving such a problem exactly is NP‑hard and impractical for realistic data sizes. The authors therefore propose a continuous relaxation: each sample i is assigned a soft membership vector γ_i = (γ_{i1},…,γ_{iK}) with γ_{ic}∈