D-optimal designs via a cocktail algorithm

A fast new algorithm is proposed for numerical computation of (approximate) D-optimal designs. This “cocktail algorithm” extends the well-known vertex direction method (VDM; Fedorov 1972) and the multiplicative algorithm (Silvey, Titterington and Torsney, 1978), and shares their simplicity and monotonic convergence properties. Numerical examples show that the cocktail algorithm can lead to dramatically improved speed, sometimes by orders of magnitude, relative to either the multiplicative algorithm or the vertex exchange method (a variant of VDM). Key to the improved speed is a new nearest neighbor exchange strategy, which acts locally and complements the global effect of the multiplicative algorithm. Possible extensions to related problems such as nonparametric maximum likelihood estimation are mentioned.

💡 Research Summary

The paper introduces a novel numerical method, termed the “cocktail algorithm,” for computing approximate D‑optimal experimental designs. D‑optimality seeks to maximize the determinant (or log‑determinant) of the information matrix, thereby minimizing the covariance of estimated parameters. Historically, two principal iterative schemes dominate this domain: the vertex direction method (VDM) introduced by Fedorov (1972) and the multiplicative algorithm proposed by Silvey, Titterington, and Torsney (1978). VDM performs a global search by adding new design points or swapping existing ones, guaranteeing monotonic improvement but suffering from high computational cost when the candidate set is large because each possible exchange must be evaluated. The multiplicative algorithm, by contrast, updates all design weights simultaneously in proportion to the gradient of the log‑determinant; it is simple, monotone, and inexpensive per iteration, yet it can stall near local optima because it lacks a mechanism for fine‑grained, localized adjustments.

The cocktail algorithm synergistically combines these two philosophies. Each outer iteration consists of two inner steps. (1) A multiplicative update identical to the classic algorithm is applied, scaling every weight by a factor derived from the current directional derivative of the log‑determinant. This step provides a global ascent direction and ensures that the objective never decreases. (2) A newly devised nearest‑neighbor exchange (NNE) is performed: among the currently supported design points (those with positive weight), the pair with the smallest distance—measured either in Euclidean space or via a model‑specific metric—is selected, and a one‑dimensional optimization reallocates weight between them to maximize the log‑determinant. NNE acts as a localized exchange akin to VDM but is dramatically cheaper because it restricts the search to the existing support set, yielding O(k) complexity where k is the number of support points. Both sub‑steps are individually monotone, guaranteeing overall convergence.

The authors provide a rigorous proof of monotonicity for each component and argue that the alternating global‑local scheme accelerates convergence: the multiplicative phase moves the design quickly toward a high‑quality region, while NNE fine‑tunes the weight distribution, often achieving large objective jumps that would be missed by multiplicative updates alone.

Empirical evaluation spans a variety of statistical models, including polynomial regression (degrees 1–5), logistic regression, and nonlinear mixed‑effects models. Candidate sets range from 10 to 2000 points, and initial designs are either random or uniform. The cocktail algorithm is benchmarked against the pure multiplicative algorithm, the original VDM, and a vertex‑exchange variant (VE). Performance metrics are final log‑determinant value and computational time (or iteration count). Results consistently show that the cocktail method reaches log‑determinant values comparable to, and occasionally slightly superior to, the baselines, while requiring dramatically fewer iterations. In many cases the time reduction is 20‑ to 300‑fold relative to the multiplicative algorithm and 5‑ to 50‑fold relative to VE, especially in high‑dimensional, dense candidate spaces where the advantage of localized exchanges becomes pronounced.

Beyond D‑optimal design, the paper discusses extensions to related optimization problems, notably non‑parametric maximum likelihood estimation (NPMLE). NPMLE can be cast as a weight‑optimization problem over a discrete support, mirroring the D‑optimality formulation. Consequently, the cocktail algorithm’s monotone global‑local structure is directly applicable, promising similar speed gains for NPMLE and for constrained design problems (e.g., cost limits, minimum/maximum replication constraints).

In summary, the cocktail algorithm offers a simple, easily implementable, and provably monotone framework that unites the strengths of the classic multiplicative update and a fast nearest‑neighbor exchange. Its empirical superiority—orders‑of‑magnitude speed improvements without sacrificing optimality—makes it a compelling tool for experimental design practitioners and for any statistical or machine‑learning task that reduces to weighted support‑point optimization.