A critical evaluation of network and pathway based classifiers for outcome prediction in breast cancer

Recently, several classifiers that combine primary tumor data, like gene expression data, and secondary data sources, such as protein-protein interaction networks, have been proposed for predicting outcome in breast cancer. In these approaches, new composite features are typically constructed by aggregating the expression levels of several genes. The secondary data sources are employed to guide this aggregation. Although many studies claim that these approaches improve classification performance over single gene classifiers, the gain in performance is difficult to assess. This stems mainly from the fact that different breast cancer data sets and validation procedures are employed to assess the performance. Here we address these issues by employing a large cohort of six breast cancer data sets as benchmark set and by performing an unbiased evaluation of the classification accuracies of the different approaches. Contrary to previous claims, we find that composite feature classifiers do not outperform simple single gene classifiers. We investigate the effect of (1) the number of selected features; (2) the specific gene set from which features are selected; (3) the size of the training set and (4) the heterogeneity of the data set on the performance of composite feature and single gene classifiers. Strikingly, we find that randomization of secondary data sources, which destroys all biological information in these sources, does not result in a deterioration in performance of composite feature classifiers. Finally, we show that when a proper correction for gene set size is performed, the stability of single gene sets is similar to the stability of composite feature sets. Based on these results there is currently no reason to prefer prognostic classifiers based on composite features over single gene classifiers for predicting outcome in breast cancer.

💡 Research Summary

This study provides a rigorous, head‑to‑head comparison of breast‑cancer outcome predictors that rely on single‑gene expression versus those that incorporate secondary biological information such as protein‑protein interaction (PPI) networks and curated pathway databases. The authors selected three representative network‑or pathway‑based methods that have been widely cited: the Chuang et al. approach (greedy search of subnetworks in PPI graphs), the Lee et al. approach (selection of high‑t‑statistic gene subsets within predefined pathways from MsigDB/KEGG), and the Taylor et al. approach (using hub‑centric correlation differences as edge‑weights).

To eliminate methodological bias, the authors assembled a benchmark consisting of six publicly available breast‑cancer microarray cohorts, totaling several thousand patients. For each possible ordered pair of training and test cohorts (30 pairs), they applied a uniform pipeline: (i) construction of composite features exclusively on the training set, (ii) selection of the optimal number of features by internal cross‑validation, (iii) training of the final classifier, and (iv) evaluation on the untouched test set. Two classifiers were examined – a nearest‑mean classifier (NMC), previously shown to be robust for breast‑cancer tasks, and a logistic‑regression model (LOG).

Performance was measured by the area under the receiver‑operating‑characteristic curve (AUC). Across all 30 training‑test combinations, the single‑gene NMC consistently achieved the highest median AUC, while none of the composite‑feature models surpassed it. In fact, the Taylor‑based models performed significantly worse than the single‑gene baseline for certain network sources (e.g., I2D). Pairwise win‑loss matrices confirmed that the single‑gene classifier won more often than any composite method. The LOG classifier showed even greater sensitivity to the number of selected features, often failing to converge for the composite‑feature models, especially those derived from the Taylor approach.

A striking finding is that randomizing the secondary data (shuffling nodes and edges in the PPI or pathway graphs) did not degrade the performance of the composite‑feature classifiers. This suggests that the biological information encoded in the networks does not contribute meaningfully to outcome prediction under the evaluated conditions.

The authors further dissected four potential explanations for the lack of advantage: (1) feature‑selection strategy, (2) the initial gene pool (secondary data sources restrict the pool to genes present in the network or pathway), (3) training‑set size, and (4) heterogeneity among cohorts. Systematic analyses showed that none of these factors altered the overall conclusion. Notably, because many genes are absent from the secondary databases, composite‑feature methods are forced to ignore a substantial portion of the transcriptome, whereas single‑gene classifiers can freely select from the entire microarray.

Stability (reproducibility) of the selected signatures was also examined using Jaccard indices. After correcting for gene‑set size, the stability of composite‑feature signatures was comparable to that of single‑gene signatures, contradicting earlier claims that pathway‑based markers are inherently more reproducible.

In summary, the study demonstrates that, given a robust single‑gene classifier, integrating current PPI or pathway information does not improve prognostic accuracy or signature stability for breast‑cancer outcome prediction. The authors therefore advise caution in adopting composite‑feature models as a default and call for the development of more sophisticated integration strategies, better regularization techniques, and thorough benchmarking. All code, data, and results are made publicly available to facilitate future comparative studies.