Bayesian Quantile Regression for Single-Index Models

Using an asymmetric Laplace distribution, which provides a mechanism for Bayesian inference of quantile regression models, we develop a fully Bayesian approach to fitting single-index models in conditional quantile regression. In this work, we use a Gaussian process prior for the unknown nonparametric link function and a Laplace distribution on the index vector, with the latter motivated by the recent popularity of the Bayesian lasso idea. We design a Markov chain Monte Carlo algorithm for posterior inference. Careful consideration of the singularity of the kernel matrix, and tractability of some of the full conditional distributions leads to a partially collapsed approach where the nonparametric link function is integrated out in some of the sampling steps. Our simulations demonstrate the superior performance of the Bayesian method versus the frequentist approach. The method is further illustrated by an application to the hurricane data.

💡 Research Summary

This paper introduces a fully Bayesian framework for single‑index models (SIM) in conditional quantile regression. The authors model the τ‑th conditional quantile of a response Y given a p‑dimensional predictor X as Q_{Y|X}(τ)=η(Xᵀβ), where η(·) is an unknown univariate link function and β∈ℝ^p is the index vector. To enable Bayesian inference, the residuals are assumed to follow an asymmetric Laplace distribution (ALD), which yields a likelihood that coincides with the check‑loss function used in frequentist quantile regression.

Following Kozumi and Kobayashi (2011), the ALD is expressed as a location‑scale mixture of a normal distribution with latent exponential variables e_i and standard normal variables z_i. Conditional on these latent variables, the model becomes Gaussian, facilitating Gibbs sampling.

For the nonparametric link η, a Gaussian process (GP) prior with zero mean and a squared‑exponential kernel C(x,x′)=γ exp{−(x−x′)²} is placed. This choice provides a flexible smooth function space while keeping the kernel hyper‑parameter γ as the only scale parameter. Importantly, the authors adopt the Gramacy‑Lian (2011) approach of not constraining β to have unit norm; instead, the kernel is defined directly on the projected values Xᵀβ, which eliminates the need for an additional range parameter d and simplifies prior specification for β.

The index vector β receives a Bayesian lasso (Laplace) prior, π(β|λ,σ)=∏_{j=1}^p (λ/(2σ)) exp(−λ|β_j|/σ), encouraging sparsity and enabling variable selection in high‑dimensional settings. The hyper‑parameter λ follows a Gamma prior, while σ (the ALD scale) and γ each have inverse‑Gamma priors. All hyper‑parameters are given weakly informative hyper‑priors (shape and scale 0.5).

A key methodological contribution is the design of a partially collapsed Markov chain Monte Carlo (MCMC) algorithm. By analytically integrating out the GP latent vector η in the conditional updates for β, σ, λ, and γ, the authors avoid the numerical instability associated with inverting the near‑singular kernel matrix C_n. They add a small nugget (ε=10⁻⁵) to C_n when necessary. This partial collapsing dramatically reduces autocorrelation in the β chain compared with a naïve Gibbs sampler that samples η and β jointly, leading to faster convergence and better mixing.

The authors evaluate the proposed Bayesian quantile single‑index model (BQSIM) through three simulation studies: (1) homoscedastic normal errors, (2) heteroscedastic errors where the variance depends on sin(Xᵀβ), and (3) exponentially distributed errors. For each scenario they consider sample sizes n=100 and n=200, five quantile levels (τ=0.1,0.25,0.5,0.75,0.9) and two additional extreme quantiles (0.95,0.99). In each setting 100 independent data sets are generated. Performance is measured by mean squared error (MSE) of the estimated index vector and by visual comparison of the fitted η against the true link. Across all settings BQSIM consistently yields lower MSE, smaller bias, and tighter credible intervals than the frequentist kernel‑based quantile SIM (QSIM) of Wu et al. (2010). The advantage is especially pronounced for extreme quantiles where data are sparse.

A real‑world application to hurricane wind‑speed data demonstrates the practical utility of the method. The estimated β reflects a meaningful linear combination of meteorological covariates (e.g., wind speed and pressure), while the posterior mean of η captures the conditional distribution of wind speed at various quantiles, revealing tail behavior that would be missed by mean regression.

In summary, the paper makes several notable contributions: (i) it extends Bayesian quantile regression to the single‑index setting, preserving the dimensionality reduction benefits of SIM; (ii) it leverages a GP prior for η without imposing a unit‑norm constraint on β, simplifying prior elicitation; (iii) it introduces a Bayesian lasso prior for β, enabling sparsity and variable selection; (iv) it develops a partially collapsed MCMC scheme that integrates out η to improve computational stability and mixing; and (v) it provides extensive simulation evidence and a real data case showing superior performance over existing frequentist approaches. Potential extensions include multi‑index models, non‑linear index functions (e.g., neural networks), and alternative asymmetric loss functions within the same Bayesian framework.