Gene selection for cancer classification using a hybrid of univariate and multivariate feature selection methods

Various approaches to gene selection for cancer classification based on microarray data can be found in the literature and they may be grouped into two categories: univariate methods and multivariate methods. Univariate methods look at each gene in the data in isolation from others. They measure the contribution of a particular gene to the classification without considering the presence of the other genes. In contrast, multivariate methods measure the relative contribution of a gene to the classification by taking the other genes in the data into consideration. Multivariate methods select fewer genes in general. However, the selection process of multivariate methods may be sensitive to the presence of irrelevant genes, noises in the expression and outliers in the training data. At the same time, the computational cost of multivariate methods is high. To overcome the disadvantages of the two types of approaches, we propose a hybrid method to obtain gene sets that are small and highly discriminative. We devise our hybrid method from the univariate Maximum Likelihood method (LIK) and the multivariate Recursive Feature Elimination method (RFE). We analyze the properties of these methods and systematically test the effectiveness of our proposed method on two cancer microarray datasets. Our experiments on a leukemia dataset and a small, round blue cell tumors dataset demonstrate the effectiveness of our hybrid method. It is able to discover sets consisting of fewer genes than those reported in the literature and at the same time achieve the same or better prediction accuracy.

💡 Research Summary

The paper addresses the critical problem of gene selection for cancer classification using microarray data, where the number of measured genes (thousands) far exceeds the number of samples (tens). Traditional approaches fall into two categories: univariate methods, which evaluate each gene independently, and multivariate methods, which consider the joint contribution of genes. Univariate techniques such as t‑tests or information gain are computationally cheap and robust to noise, but they ignore gene‑gene interactions. Multivariate techniques like Recursive Feature Elimination (RFE) based on Support Vector Machines (SVM) capture interactions and usually produce smaller gene sets, yet they are computationally intensive and can be destabilized by irrelevant or noisy features present in the initial high‑dimensional space.
To combine the strengths of both families, the authors propose a hybrid pipeline that first applies the univariate Maximum Likelihood (LIK) method to rank all genes by their class‑specific likelihood ratios. The top N genes (empirically set to a few hundred) are retained as a reduced candidate pool. This step dramatically shrinks the dimensionality, removes many noisy variables, and keeps the computational burden of the subsequent stage low.
The reduced pool is then fed into a multivariate RFE procedure. RFE iteratively trains a linear SVM, computes the absolute weight of each gene, and removes the least important gene(s) at each iteration. Cross‑validation monitors classification accuracy, allowing the algorithm to stop when further removal would degrade performance. Because RFE operates on a pre‑filtered set, it is less prone to being misled by irrelevant features and converges faster.
The authors evaluate the hybrid method on two well‑known cancer microarray benchmarks. The first dataset comprises 72 leukemia samples (38 acute myeloid leukemia, 34 acute lymphoblastic leukemia) measured on 7,129 genes. The second dataset contains 83 samples of small round blue cell tumors (SRBCT) with roughly 2,000 genes. For each dataset, LIK selects the top 250–300 genes, after which RFE identifies the final minimal subset. In the leukemia case, the method achieves 100 % classification accuracy using only three to five genes, a dramatic reduction compared with previously reported gene panels of ten to thirty genes. In the SRBCT case, six to eight genes yield a classification accuracy above 96 %, again surpassing earlier results.
Beyond performance metrics, the selected genes correspond to known biological markers for the respective cancer types, suggesting that the hybrid approach does not merely overfit but captures biologically meaningful signals. The authors also discuss limitations: the choice of N in the LIK stage was not exhaustively explored, alternative univariate scoring functions (t‑test, information gain) were not compared, and the RFE component relied on a linear kernel, potentially missing nonlinear interactions. Future work could involve systematic sensitivity analysis of N, testing other univariate filters, and extending RFE to nonlinear kernels or deep‑learning‑based feature selectors.
In summary, the study demonstrates that a simple two‑stage pipeline—univariate LIK followed by multivariate RFE—can produce ultra‑compact gene signatures without sacrificing, and often improving, classification accuracy. This hybrid strategy offers a practical solution for researchers seeking cost‑effective biomarker panels and for clinicians requiring robust, interpretable diagnostic tools.