Balancing Evidentiary Value and Sample Size of Adaptive Designs with Application to Animal Experiments

Reducing the number of experimental units is one of the three pillars of the 3R principles (Replace, Reduce, Refine) in animal research. At the same time, statistical error rates need to be controlled to enable reliable inferences and decisions. This paper proposes a novel measure to quantify the evidentiary value of one experimental unit for a given study design. The experimental unit information index (EUII) is based on power, Type-I error and sample size, and has attractive interpretations both in terms of frequentist error rates and Bayesian posterior odds. We introduce the EUII in simple statistical test settings and show that its asymptotic value depends only on the assumed relative effect size under the alternative. We then extend the definition to adaptive designs where early stopping for efficacy or futility may cause reductions in sample size. Applications to group-sequential designs and a recently proposed adaptive statistical test procedure show the usefulness of the approach when the goal is to maximize the evidentiary value of one experimental unit. A reanalysis of 2738 animal experiments with simulated results from (post-hoc) interim analyses illustrates the possible savings in sample size.

💡 Research Summary

The paper addresses a central challenge in pre‑clinical animal research: how to reduce the number of experimental units while still maintaining rigorous statistical inference. To this end the authors introduce the Experimental Unit Information Index (EUII), a novel metric that quantifies the evidentiary value contributed by a single experimental unit (e.g., one animal) under a given study design.

The development starts from the well‑known diagnostic odds ratio (DOR) used in clinical test evaluation. By treating a statistically significant result as a “positive test” and a non‑significant result as a “negative test,” the authors define the positive likelihood ratio (LR⁺) as power divided by the Type‑I error rate (α) and the negative likelihood ratio (LR⁻) as (1‑power)/(1‑α). The DOR, equal to LR⁺/LR⁻, can be expressed as

 DOR = (power · (1‑α)) / ((1‑power) · α).

This quantity has two complementary interpretations: (i) a frequentist view as the ratio of the odds of obtaining a significant result under the alternative hypothesis to the odds under the null, and (ii) a Bayesian view as the ratio of posterior odds of the alternative given a significant versus a non‑significant outcome.

To obtain a per‑unit measure, the DOR is normalized by the sample size n via the n‑th root:

 EUII = DOR^{1/n}.

Because likelihood ratios multiply across independent observations, taking the n‑th root yields the geometric average contribution of a single unit to the overall evidential strength. An EUII greater than one indicates that each additional unit improves the test’s ability to discriminate H₀ from H₁. For standard one‑sample z‑ or t‑tests the authors show that, as n grows while keeping α fixed, the EUII converges to exp(δ²/2), where δ is the standardized effect size. Thus the asymptotic per‑unit information depends only on the assumed effect size.

The framework is then extended to adaptive designs that allow early stopping for efficacy or futility. In such designs the expected sample size under the alternative (E₁) and under the null (E₀) differ. The authors propose an extended EUII that weights the DOR by the reciprocal of these expected sample sizes, effectively averaging the per‑unit information across the two possible stopping pathways.

Using this metric, the paper evaluates several group‑sequential designs (O’Brien‑Fleming, Pocock, and predictive‑power‑based futility boundaries) and a recently proposed “constrained sample augmentation” (CSA) procedure. Simulations reveal that designs maximizing EUII tend to achieve the same nominal power and α while requiring fewer expected animals. In particular, CSA— which caps the maximum sample size but adds subjects only when interim predictive power is low— outperforms traditional group‑sequential methods in scenarios with modest effect sizes and frequent futility stops.

To illustrate practical impact, the authors re‑analyze a database of 2,738 animal experiments from neuroscience and metabolism studies. By simulating interim analyses and applying optimal group‑sequential or CSA designs, they estimate that 15–22 % of animals could have been saved without compromising statistical conclusions. This quantitative demonstration aligns directly with the “Reduce” pillar of the 3R principles.

Overall, the paper makes four key contributions: (1) framing diagnostic odds ratio as a measure of evidential strength for hypothesis tests, (2) introducing the per‑unit Experimental Unit Information Index, (3) extending the concept to adaptive designs with differing expected sample sizes, and (4) providing empirical evidence that EUII‑guided designs can materially lower animal use. The EUII offers a transparent, interpretable tool for researchers and ethics committees to balance sample size against evidential quality, thereby promoting more humane and efficient pre‑clinical research.