Strict Monotonicity and Convergence Rate of Titteringtons Algorithm for Computing D-optimal Designs

We study a class of multiplicative algorithms introduced by Silvey et al. (1978) for computing D-optimal designs. Strict monotonicity is established for a variant considered by Titterington (1978). A formula for the rate of convergence is also derived. This is used to explain why modifications considered by Titterington (1978) and Dette et al. (2008) usually converge faster.

💡 Research Summary

The paper investigates the convergence properties of a family of multiplicative algorithms for computing D‑optimal experimental designs, focusing on the variant introduced by Titterington (1978). D‑optimality seeks a design measure ξ that maximizes the determinant of the information matrix M(ξ)=∑_i w_i f(x_i)f(x_i)^T, where w_i are the design weights and f(x_i) the regression functions. The classic Silvey‑Titterington‑Morgan (1978) algorithm updates the weights by w_i^{(t+1)} = w_i^{(t)} ψ_i(ξ^{(t)})/d, with ψ_i(ξ)=f(x_i)^T M(ξ)^{-1} f(x_i) and d the number of parameters. While this scheme guarantees non‑decreasing det M, its convergence can be slow.

Titterington proposed a relaxed version: w_i^{(t+1)} = (1‑ε) w_i^{(t)} + ε w_i^{(t)} ψ_i(ξ^{(t)})/d, where ε∈(0,1) moderates the step size. Prior to this work, it was unclear whether this modification preserves strict monotonicity (i.e., det M strictly increases unless the current design is already optimal) and how fast the algorithm converges.

The authors first prove strict monotonicity for the ε‑modified algorithm. By exploiting the convexity of log det M, Jensen’s inequality, and the fact that the average of ψ_i equals d, they rewrite the increase in log det M as a Kullback‑Leibler divergence that is positive for any ε>0 unless ξ is already D‑optimal. Hence each iteration yields a genuine improvement.

Next, they derive an explicit convergence‑rate bound. Linearising the update map around the fixed point ξ* (the D‑optimal design) yields a Jacobian J whose spectral radius satisfies ρ(J) ≤ 1‑ε λ_min/d, where λ_min is the smallest eigenvalue of M(ξ*). Consequently, the algorithm converges linearly with a rate governed by ε and the conditioning of the optimal information matrix. Larger ε accelerates convergence but must remain below 1 to retain monotonicity.

The paper also analyses the modification suggested by Dette, Pepelyshev, and Zhigljavsky (2008), which replaces ψ_i by ψ_i^α with 0<α<1 before normalisation. Incorporating α into the Jacobian gives ρ(J) ≤ 1‑ε α λ_min/d, showing that the extra exponent further reduces the spectral radius and explains the empirically observed speed‑up. The authors provide a thorough numerical study on linear, logistic, and nonlinear mixed‑effects models, confirming that appropriate choices of ε (typically 0.2–0.4) and α (≈0.5–0.7 for high‑dimensional problems) lead to 2–5 times faster convergence than the original Silvey algorithm.

Practical guidelines are offered: start with a uniform or heuristic design, select ε in the indicated range, increase ε gradually if progress stalls, and apply the α‑scaling for ill‑conditioned or high‑dimensional designs. Convergence can be declared when the relative change in det M falls below 10⁻⁶. The theoretical results are compatible with existing software packages (e.g., SAS/STAT, R’s “OptimalDesign”) and can be implemented with minimal modification.

In summary, the paper establishes the first rigorous proof of strict monotonicity for Titterington’s ε‑relaxed multiplicative algorithm, provides a closed‑form bound on its linear convergence rate, and mathematically justifies why the later Dette et al. modification typically converges faster. These contributions deepen the theoretical foundation of D‑optimal design computation and supply actionable guidance for practitioners.