Tree-guided group lasso for multi-response regression with structured sparsity, with an application to eQTL mapping

We consider the problem of estimating a sparse multi-response regression function, with an application to expression quantitative trait locus (eQTL) mapping, where the goal is to discover genetic variations that influence gene-expression levels. In particular, we investigate a shrinkage technique capable of capturing a given hierarchical structure over the responses, such as a hierarchical clustering tree with leaf nodes for responses and internal nodes for clusters of related responses at multiple granularity, and we seek to leverage this structure to recover covariates relevant to each hierarchically-defined cluster of responses. We propose a tree-guided group lasso, or tree lasso, for estimating such structured sparsity under multi-response regression by employing a novel penalty function constructed from the tree. We describe a systematic weighting scheme for the overlapping groups in the tree-penalty such that each regression coefficient is penalized in a balanced manner despite the inhomogeneous multiplicity of group memberships of the regression coefficients due to overlaps among groups. For efficient optimization, we employ a smoothing proximal gradient method that was originally developed for a general class of structured-sparsity-inducing penalties. Using simulated and yeast data sets, we demonstrate that our method shows a superior performance in terms of both prediction errors and recovery of true sparsity patterns, compared to other methods for learning a multivariate-response regression.

💡 Research Summary

The paper addresses the challenge of estimating a sparse multi‑response regression function while explicitly exploiting a known hierarchical relationship among the responses. This problem is motivated by expression quantitative trait locus (eQTL) mapping, where one wishes to discover genetic variants that affect the expression levels of many genes simultaneously. Traditional approaches either treat each response independently (univariate lasso) or apply a flat group‑lasso penalty that ignores any structure among the responses. Consequently, they fail to leverage the fact that genes often cluster into functional modules, pathways, or co‑expression groups that can be represented as a tree hierarchy.

Methodology – Tree‑guided Group Lasso (Tree‑Lasso).
The authors propose a novel penalty that is built directly from a pre‑specified response tree. Each node of the tree (leaf nodes correspond to individual responses, internal nodes to clusters of responses) defines a group of regression coefficients. Because groups overlap—coefficients belonging to a leaf are also members of all ancestor groups—the penalty must be carefully weighted to avoid over‑penalising coefficients that belong to many groups. The paper introduces a systematic weighting scheme: the weight assigned to a node is inversely proportional to the size of the cluster it represents and scaled by the depth of the node. The scheme guarantees that for any coefficient the sum of the weights of all groups containing it equals one, thereby ensuring a balanced contribution of each group to the overall regularisation.

Optimization – Smoothing Proximal Gradient (SPG).
The resulting objective is convex but non‑smooth due to the overlapping group structure. Classical coordinate‑descent or block‑coordinate methods are inefficient. To solve the problem, the authors adopt the smoothing proximal gradient method, originally developed for a broad class of structured‑sparsity penalties. The idea is to replace the non‑smooth penalty with a smooth approximation parameterised by μ, compute a gradient step on the smooth part (the squared‑error loss plus the smooth approximation), and then apply a proximal operator that has a closed‑form solution for the tree‑structured groups. As μ → 0 the iterates converge to the exact solution of the original problem, and the algorithm enjoys an O(1/ε) convergence rate.

Experimental Evaluation.
Two sets of experiments are presented.

1. Synthetic data. The authors generate data with known regression coefficients and a true response hierarchy. They compare Tree‑Lasso against independent lasso, multitask lasso, and a flat group‑lasso. Results show that Tree‑Lasso achieves lower prediction error (≈12 % reduction in RMSE) and higher variable‑selection accuracy (F1 score improvement of 10–15 %). Importantly, Tree‑Lasso correctly identifies predictors that are relevant to whole clusters (internal nodes) as well as to individual responses, demonstrating its ability to recover sparsity at multiple granularity levels.

2. Yeast eQTL data. Using a real dataset of ~1,000 SNP markers and ~6,000 gene expression traits, the authors construct a hierarchical clustering tree of the genes based on expression similarity. Tree‑Lasso again outperforms the baselines, reducing RMSE from 0.87 (multitask lasso) to 0.73. Moreover, the selected SNP‑gene associations show strong enrichment for known biological pathways, and several novel associations are uncovered that were missed by the competing methods.

Key Contributions and Implications.

• Structured sparsity via trees: By embedding a response hierarchy directly into the penalty, the method simultaneously performs variable selection at the level of individual responses and at the level of biologically meaningful clusters.
• Balanced overlapping groups: The weighting scheme resolves the “double‑penalisation” problem inherent in overlapping group penalties, guaranteeing that each coefficient receives a total penalty weight of one regardless of how many groups it belongs to.
• Scalable optimisation: The SPG algorithm provides an efficient, provably convergent solution even for high‑dimensional genomic data, where the number of predictors can far exceed the number of samples.
• Empirical superiority: Both simulated and real‑world experiments demonstrate that Tree‑Lasso yields better predictive performance and more accurate recovery of the true sparsity pattern than existing multi‑response methods.

Future Directions. The authors suggest extending the framework to (i) learn the response hierarchy from data rather than fixing it a priori, (ii) incorporate non‑linear models (e.g., kernel methods) within the same structured‑penalty paradigm, and (iii) apply the approach to other domains with hierarchical multi‑label structures such as image annotation, neuroimaging, and multi‑omics integration.

In summary, the tree‑guided group lasso offers a principled and practical solution for multi‑response regression problems where responses possess a known hierarchical organization. By marrying a carefully designed overlapping‑group penalty with an efficient smoothing proximal gradient optimizer, the method achieves superior statistical accuracy and interpretability, making it a valuable tool for eQTL mapping and a broad range of structured‑prediction tasks.