Empirical Bayes for Data Integration

We discuss the use of empirical Bayes for data integration, in the sense of transfer learning. Our main interest is in settings where one wishes to learn structure (e.g. feature selection) and one only has access to incomplete data from previous studies, such as summaries, estimates or lists of relevant features. We discuss differences between full Bayes and empirical Bayes, and develop a computational framework for the latter. We discuss how empirical Bayes attains consistent variable selection under weaker conditions (sparsity and betamin assumptions) than full Bayes and other standard criteria do, and how it attains faster convergence rates. Our high-dimensional regression examples show that fully Bayesian inference enjoys excellent properties, and that data integration with empirical Bayes can offer moderate yet meaningful improvements in practice.

💡 Research Summary

The paper addresses the problem of integrating external information into a primary analysis when only summary‐level data (meta‑covariates) are available from previous studies. This situation is common in modern transfer‑learning contexts: the analyst has full access to a target dataset y (e.g., human colon‑cancer gene expression) but only a list of important variables, effect estimates, or other coarse summaries Z from related studies (e.g., mouse experiments). The authors propose to use an empirical Bayes (EB) framework to incorporate such meta‑covariates into structural learning tasks, focusing on high‑dimensional variable selection.

The manuscript first distinguishes two data‑integration scenarios. The first, “full data,” assumes all datasets are observed and hierarchical models can share information. The second, “meta‑covariates,” assumes only the target data are observed while the auxiliary information is reduced to Z. The latter is the focus of the paper. The authors argue that a fully Bayesian approach would place a prior π(γ|ω) on model inclusion indicators γ that depends on Z through hyper‑parameters ω, and then assign a hyper‑prior π(ω). However, because ω is not learned from the data in the prior, the resulting posterior π(γ|y) cannot adapt to the target data beyond the likelihood term, limiting the benefit of the auxiliary information.

In the EB approach, ω is estimated from the data by maximizing the marginal likelihood p(y|ω). The prior becomes π(θ|Z, ω̂), a data‑driven prior that is expected to be closer to the true distribution of the parameters θ. This yields a posterior π(θ|y, ω̂) that enjoys better frequentist calibration than a fully Bayesian posterior with a possibly misspecified hyper‑prior. The authors discuss the philosophical tension between Bayesian coherence (sequential updating) and the data‑dependent nature of EB, and note potential pitfalls such as degenerate ω̂ that assign inclusion probabilities of 0 or 1, leading to over‑confident inference.

The methodological core is a concrete model for linear regression with p covariates and q meta‑covariates per predictor. For each predictor j, a meta‑covariate vector z_j∈ℝ^q is observed. The inclusion probability is modeled via a logistic link: m_j(ω)=1/(1+exp(−z_j^T ω)). Conditional on γ, the regression coefficients have a standard spike‑and‑slab prior, and the likelihood is Gaussian. The hyper‑parameter ω is estimated by an EM algorithm: the E‑step computes the posterior inclusion probabilities E