The pigeonhole bootstrap

Recently there has been much interest in data that, in statistical language, may be described as having a large crossed and severely unbalanced random effects structure. Such data sets arise for recommender engines and information retrieval problems. Many large bipartite weighted graphs have this structure too. We would like to assess the stability of algorithms fit to such data. Even for linear statistics, a naive form of bootstrap sampling can be seriously misleading and McCullagh [Bernoulli 6 (2000) 285–301] has shown that no bootstrap method is exact. We show that an alternative bootstrap separately resampling rows and columns of the data matrix satisfies a mean consistency property even in heteroscedastic crossed unbalanced random effects models. This alternative does not require the user to fit a crossed random effects model to the data.

💡 Research Summary

The paper addresses a pervasive problem in modern data‑intensive applications such as recommender systems, information‑retrieval engines, and large bipartite weighted graphs: the data often exhibit a crossed random‑effects structure that is both highly unbalanced and heteroscedastic. In such settings each observation can be written as
(Y_{ij}= \mu + \alpha_i + \beta_j + \varepsilon_{ij}),
where (\alpha_i) and (\beta_j) are row‑ and column‑specific random effects and (\varepsilon_{ij}) is an error term whose variance may differ across cells. Traditional bootstrap procedures—whether naïve case resampling or block‑wise resampling—treat the data as if the rows and columns were independent. This ignores the intrinsic dependence induced by the crossed effects and leads to severely biased variance estimates and confidence intervals. McCullagh (2000) proved that no bootstrap can be exact for such models, a result often referred to as the “no‑bootstrap‑exactness” theorem.

To overcome these limitations the authors propose the “pigeonhole bootstrap.” The method resamples rows and columns separately: one draws a bootstrap sample of row indices ({I_1,\dots,I_B}) with replacement from ({1,\dots,m}) and independently draws a bootstrap sample of column indices ({J_1,\dots,J_B}) from ({1,\dots,n}). The bootstrap data matrix is then formed by pairing the selected rows and columns, i.e., (Y^{*}{ij}=Y{I_i J_j}). This construction preserves the crossed structure because each bootstrap observation still carries a row effect and a column effect, albeit from randomly chosen rows and columns. Crucially, the procedure does not require fitting a crossed random‑effects model beforehand; it is completely model‑free and only needs the index sets.

The authors provide a rigorous theoretical analysis. Under fairly general assumptions—zero‑mean independent row and column effects, possibly heteroscedastic errors, and independence of (\varepsilon_{ij}) conditional on the effects—they prove that the pigeonhole bootstrap satisfies a mean‑consistency property: the expectation of any linear statistic computed on the bootstrap sample converges to the expectation of the same statistic on the original data as the sample size grows. In symbols, (E^{*}_B