Optimal designs for rational function regression

We consider optimal non-sequential designs for a large class of (linear and nonlinear) regression models involving polynomials and rational functions with heteroscedastic noise also given by a polynomial or rational weight function. The proposed method treats D-, E-, A-, and $\Phi_p$-optimal designs in a unified manner, and generates a polynomial whose zeros are the support points of the optimal approximate design, generalizing a number of previously known results of the same flavor. The method is based on a mathematical optimization model that can incorporate various criteria of optimality and can be solved efficiently by well established numerical optimization methods. In contrast to previous optimization-based methods proposed for similar design problems, it also has theoretical guarantee of its algorithmic efficiency; in fact, the running times of all numerical examples considered in the paper are negligible. The stability of the method is demonstrated in an example involving high degree polynomials. After discussing linear models, applications for finding locally optimal designs for nonlinear regression models involving rational functions are presented, then extensions to robust regression designs, and trigonometric regression are shown. As a corollary, an upper bound on the size of the support set of the minimally-supported optimal designs is also found. The method is of considerable practical importance, with the potential for instance to impact design software development. Further study of the optimality conditions of the main optimization model might also yield new theoretical insights.

💡 Research Summary

The paper addresses the long‑standing challenge of constructing optimal experimental designs for regression models that involve both polynomial and rational functions, while also allowing for heteroscedastic error structures expressed through polynomial or rational weight functions. Traditional optimal‑design theory has largely been confined to linear models with homoscedastic errors, and extensions to nonlinear or weighted settings have required ad‑hoc approximations or separate algorithms for each optimality criterion (D‑, E‑, A‑, Φp‑optimality). In contrast, the authors propose a unified mathematical‑optimization framework that simultaneously handles all these criteria and yields the support points of the optimal approximate design as the real zeros of a single polynomial.

The core of the methodology is the representation of the information matrix of the regression model as a rational function of the design points. By expressing D‑optimality (maximizing the determinant), E‑optimality (maximizing the smallest eigenvalue), A‑optimality (minimizing the trace), and the more general Φp‑optimality (maximizing a p‑norm of the eigenvalues) within a single objective function, the authors formulate a non‑convex optimization problem that nevertheless possesses a special structure: the Karush‑Kuhn‑Tucker (KKT) conditions lead to a polynomial whose coefficients depend linearly on the Lagrange multipliers. Consequently, the optimal design problem reduces to (i) solving a convex‑concave saddle‑point problem for the multipliers and (ii) finding the real roots of the resulting polynomial, which constitute the design’s support points.

Algorithmically, the saddle‑point problem is tackled with interior‑point methods or semidefinite‑programming relaxations that are well‑established in numerical optimization. The authors prove that the computational complexity grows polynomially with the degree of the underlying polynomial, and they establish a theoretical upper bound on the number of support points: for a polynomial of degree n, at most n + 1 points are needed. This bound not only guarantees a finite, small support set but also provides a stopping criterion for iterative algorithms.

The paper validates the approach on a series of numerical experiments ranging from low‑degree (degree 5) to high‑degree (degree 30) polynomials. In all cases, the running times are negligible (seconds on a standard laptop), and the resulting designs match or improve upon those obtained by specialized, criterion‑specific algorithms. The method’s numerical stability is demonstrated by an example involving very high‑degree polynomials, where traditional eigenvalue‑based methods suffer from ill‑conditioning.

Beyond linear models, the authors extend the framework to locally optimal designs for nonlinear regression models that contain rational terms. By linearizing the nonlinear model around a nominal parameter vector and treating the linearization point as an additional design variable, the same optimization machinery can be applied iteratively to converge to a locally optimal design. Robust design is also addressed: parameter uncertainty is modeled either as a bounded set or a probability distribution, and the worst‑case (min‑max) or average risk is incorporated into the Φp‑objective, yielding designs that retain efficiency under misspecification. Finally, the authors show how the technique can be adapted to trigonometric regression by interpreting sine and cosine terms as rational functions on the complex unit circle.

A noteworthy by‑product of the analysis is an explicit upper bound on the size of minimally‑supported optimal designs, which has practical implications for software that automatically determines the number of design points needed. The authors argue that their unified approach can be directly embedded into design‑of‑experiments packages, offering a single, efficient engine that replaces a suite of disparate algorithms.

In summary, the paper makes four major contributions: (1) a unified optimization model that simultaneously handles D, E, A, and Φp optimality for a broad class of polynomial‑rational regression models with heteroscedastic weights; (2) a constructive proof that optimal support points are the real zeros of a polynomial derived from the KKT conditions, providing an analytical handle on the design problem; (3) an algorithmic scheme with provable polynomial‑time complexity and demonstrated numerical stability even for high‑degree problems; and (4) extensions to locally optimal nonlinear designs, robust designs, and trigonometric regression, together with a theoretical bound on the minimal support size. These contributions collectively advance both the theory and practice of optimal experimental design, and they open a clear path toward more versatile and efficient design software.