Bootstrap-based estimation and inference for measurement precision under ISO 5725

The ISO 5725 series frames interlaboratory precision through repeatability, between-laboratory, and reproducibility variances, yet practical guidance on deploying bootstrap methods within this one-way random-effects setting remains limited. We study resampling strategies tailored to ISO 5725 data and extend a bias-correction idea to obtain simple adjusted point estimators and confidence intervals for the variance components. Using extensive simulations that mirror realistic study sizes and variance ratios, we evaluate accuracy, stability, and coverage, and we contrast the resampling-based procedures with ANOVA-based estimators and common approximate intervals. The results yield a clear division of labor: adjusted within-laboratory resampling provides accurate and stable point estimation in small-to-moderate designs, whereas a two-stage strategy-resampling laboratories and then resampling within each-paired with bias-corrected and accelerated intervals offers the most reliable (near-nominal or conservative) confidence intervals. Performance degrades under extreme designs, such as very small samples or dominant between-laboratory variation, clarifying when additional caution is warranted. A case study from an ISO 5725-4 dataset illustrates how the recommended procedures behave in practice and how they compare with ANOVA and approximate methods. We conclude with concrete guidance for implementing resampling-based precision analysis in interlaboratory studies: use adjusted within-laboratory resampling for point estimation, and adopt the two-stage strategy with bias-corrected and accelerated intervals for interval estimation.

💡 Research Summary

The paper addresses a practical gap in the ISO 5725 series, which defines three key variance components for inter‑laboratory studies—repeatability (σ²_r), between‑laboratory (σ²_L), and reproducibility (σ²_R = σ²_r + σ²_L). While classical ANOVA provides unbiased point estimators for these components, confidence intervals are difficult to derive analytically, especially for σ²_L, and the ANOVA approach can suffer from bias or even negative estimates in small samples. The authors therefore explore bootstrap‑based methods that require no distributional assumptions and can be tailored to the one‑way random‑effects structure inherent in ISO 5725 data.

Five bootstrap resampling schemes are defined:

1. boot‑i – resample whole laboratories (the index i) while keeping all measurements within each laboratory fixed.
2. boot‑js – within each laboratory, resample the measurement replicates once (single resampling of j).
3. boot‑jr – within each laboratory, perform repeated resampling of the replicates (multiple draws of j for each bootstrap replicate).
4. boot‑ijr – a two‑stage hierarchical scheme: first resample laboratories, then apply the repeated within‑lab resampling of (3).
5. boot‑ijs – simultaneously resample both laboratory and replicate indices once.

Because bootstrap estimates are often biased in random‑effects models, the authors adapt the bias‑correction formulas originally proposed by Wiley (2010) for two‑way random‑effects without replication. For each scheme they derive adjustment factors that scale the bootstrap means of σ²_r and σ²_L (see equations 3.2‑3.4). The adjusted estimators are denoted σ̂²_r:ad and σ̂²_L:ad.

Three bootstrap confidence‑interval (CI) constructions are examined:

• Standard normal (Wald) CI – uses the bootstrap standard error and the normal quantile.
• Percentile CI – directly uses the empirical quantiles of the bootstrap replicates.
• Bias‑Corrected and Accelerated (BCa) CI – incorporates a bias‑correction term (z₀) and an acceleration term (a) to adjust the percentile locations, providing better coverage for skewed bootstrap distributions.

A comprehensive Monte‑Carlo simulation study evaluates the performance of the five resampling schemes, the three CI methods, and the traditional ANOVA approach. The simulation varies four key factors:

• Between‑lab variance ratio σ²_L/σ²_r ∈ {0.25, 0.5, 1, 2}.
• Number of laboratories k ∈ {3, 5, 10, 50}.
• Number of replicates per lab n ∈ {3, 5, 10, 50}.
• 1,000 Monte‑Carlo repetitions with 1,000 bootstrap replicates each.

Key findings:

1. Point estimation – For small to moderate designs (e.g., k = 5, n = 5), the adjusted boot‑jr and boot‑js estimators achieve near‑unbiased estimates of both σ²_r and σ²_L (within ±0.01 of the true values). In contrast, boot‑i, boot‑ijr, and boot‑ijs tend to overestimate σ²_r by 20‑25 % and underestimate σ²_L, especially when n is small. The bias diminishes as n grows, but the superiority of boot‑jr/boot‑js remains consistent across all k and variance‑ratio scenarios.

2. Standard errors – Bootstrap standard errors closely match the analytical ANOVA SEs for most configurations, but they become unstable when both k and n are very low (e.g., k = 3, n = 3), reflecting the limited information in such extreme designs.

3. Confidence intervals – The BCa intervals paired with the two‑stage boot‑ijr scheme provide coverage probabilities that are either at the nominal 95 % level or slightly conservative, outperforming both the percentile and Wald intervals. The percentile method suffers undercoverage (as low as 85 %) when the bootstrap distribution is skewed, while the Wald intervals are overly narrow when normality assumptions are violated.

4. Extreme designs – When the number of laboratories is very small or the between‑lab variance dominates (σ²_L/σ²_r ≥ 2), all bootstrap methods show degraded performance (higher bias, lower coverage). The authors recommend caution, possibly increasing the number of bootstrap replicates or augmenting the experimental design.

A real‑world illustration uses an ISO 5725‑4 dataset (k = 12 labs, n = 4 replicates). Adjusted boot‑jr point estimates align closely with the classical ANOVA estimates, but the BCa intervals derived from boot‑ijr are noticeably wider, reflecting their more reliable coverage.

Practical guidance distilled from the study:

• For point estimation of repeatability and between‑lab variances, employ the adjusted within‑lab resampling schemes (boot‑jr or boot‑js). These give the smallest bias and most stable estimates in typical inter‑laboratory study sizes.
• For interval estimation, adopt the two‑stage hierarchical resampling (boot‑ijr) together with the BCa confidence‑interval method. This combination yields near‑nominal or conservative coverage across a broad range of designs.
• In designs with very few laboratories or highly imbalanced variance components, supplement bootstrap analysis with additional Monte‑Carlo validation or consider increasing sample sizes before drawing definitive conclusions.
• Use at least 1,000 bootstrap replicates (the authors used M = 1,000) and verify convergence of the bias‑correction (z₀) and acceleration (a) estimates.

In summary, the paper demonstrates that bootstrap techniques, when paired with appropriate bias‑correction formulas and a two‑stage resampling strategy, provide a robust and practical toolkit for estimating ISO 5725 precision measures. The proposed methods outperform traditional ANOVA‑based confidence intervals, especially in small‑to‑moderate sample settings, and they offer clear, actionable recommendations for analysts conducting inter‑laboratory measurement studies.