Upper bounds on the minimum coverage probability of confidence intervals in regression after variable selection

We consider a linear regression model, with the parameter of interest a specified linear combination of the regression parameter vector. We suppose that, as a first step, a data-based model selection (e.g. by preliminary hypothesis tests or minimizing AIC) is used to select a model. It is common statistical practice to then construct a confidence interval for the parameter of interest based on the assumption that the selected model had been given to us a priori. This assumption is false and it can lead to a confidence interval with poor coverage properties. We provide an easily-computed finite sample upper bound (calculated by repeated numerical evaluation of a double integral) to the minimum coverage probability of this confidence interval. This bound applies for model selection by any of the following methods: minimum AIC, minimum BIC, maximum adjusted R-squared, minimum Mallows’ Cp and t-tests. The importance of this upper bound is that it delineates general categories of design matrices and model selection procedures for which this confidence interval has poor coverage properties. This upper bound is shown to be a finite sample analogue of an earlier large sample upper bound due to Kabaila and Leeb.

💡 Research Summary

The paper addresses a fundamental problem in post‑selection inference for linear regression: the coverage of a nominal (1-\alpha) confidence interval for a linear combination (\theta = a^{\top}\beta) when the model has been chosen by a data‑driven procedure. In practice, analysts often fit a model, run a variable‑selection step (e.g., based on AIC, BIC, adjusted (R^{2}), Mallows’ (C_{p}), or sequential t‑tests), and then construct a confidence interval for (\theta) as if the selected model had been fixed a priori. This “naïve” approach ignores the dependence between the selected model (\hat M) and the estimator (\hat\beta_{\hat M}), leading to potentially severe under‑coverage.

The authors formalize the worst‑case performance by defining the minimum coverage probability \