Computer experiments involving both qualitative and quantitative (QQ) factors have attracted increasing attention. Gaussian process (GP) models have proven effective in this context by choosing specialized covariance functions for QQ factors. In this work, we extend the latent variable-based GP approach, which maps qualitative factors into a continuous latent space, by establishing a general framework to apply standard kernel functions to continuous latent variables. This approach provides a novel perspective for interpreting some existing GP models for QQ factors and introduces new covariance structures in some situations. The ordinal structure can be incorporated naturally and seamlessly in this framework. Furthermore, the Bayesian information criterion and leave-one-out cross-validation are employed for model selection and model averaging. The performance of the proposed method is comprehensively studied on several examples.
Computer experiments have attracted increasing attention in science, engineering, and business due to their ability to model complex systems. However, the high computational cost of running these simulations often necessitates the use of surrogate models or emulators. Among these, Gaussian process (GP) modeling has emerged as a powerful approach because it can approximate the behavior of simulations accurately and efficiently (Santner et al., 2003).
Recent developments have extended GP modeling to support a variety of input types, such as probability distributions (Bachoc et al., 2017) and functions (Li and Tan, 2022).
In many applications, the inputs of a computer experiment involve both quantitative and qualitative factors, commonly referred to as QQ inputs. For instance, in embankment system design, the inputs include one quantitative variable (shoulder distance from the centerline) and three qualitative variables (construction rate, Young’s modulus of columns, and reinforcement stiffness) (Liu and Rowe, 2015;Deng et al., 2017). Similarly, modeling the thermal dynamics of a data center requires consideration of qualitative factors such as diffuser location, return air vent location, and rack heat load nonuniformity, along with quantitative factors like rack temperature rise, rack heat load, and total diffuser flow rate (Schmidt et al., 2005;Qian et al., 2008). These examples show the importance of developing GP models that work well with QQ inputs.
An essential step of GP modeling is the construction of the covariance function. In recent years, a number of covariance structures have been proposed and investigated to improve prediction accuracy for handling computer experiments with QQ inputs (Qian et al., 2008;Zhou et al., 2011;Deng et al., 2017;Zhang et al., 2020;Roustant et al., 2020;Garrido-Merchán and Hernández-Lobato, 2020;Tao et al., 2021;Xiao et al., 2021;Lin et al., 2024), as reviewed in Section 2.2. Nevertheless, there still lacks a general framework that ties these approaches together. as
where µ represents the constant mean term and G(U, V) is a zero-mean GP. The primary goal is to model the covariance between responses Y (U, V) and Y (U ′ , V ′ ) corresponding to two distinct inputs (U, V) and (U ′ , V ′ ). By utilizing the covariance kernel, we can make predictions for new inputs (Santner et al., 2003).
Existing approaches focus on choosing or proposing covariance structures that have some of the attributes: intuitive, interpretable, or computationally efficient (McMillan et al., 1999;Qian et al., 2008;Zhou et al., 2011;Deng et al., 2017). Motivated by the idea that qualitative variables can be represented by some underlying numerical values, Zhang et al. (2020) proposed that the j-th qualitative factor v j corresponds to a latent vector z (j)
where 1 ≤ l j ≤ a j . Following this formulation, we define the concatenated latent vector
v 1 ) ⊤ , (z
Using this framework, we can state that the response Y (U, V) for input (U, V) follows the same distribution as the response Y for input (U, Z V ), i.e.,
Given the GP assumption, this distributional equivalence holds if and only if their covariance functions are identical. Therefore, we propose to model the covariance function of Y (U, V), the original process of interest, using that of the continuous input (U, Z V ) as follows
where σ 2 denotes the variance, and K U (•, •) and K Z (•, •) are respectively kernel functions for the quantitative factors and latent vectors associated with the qualitative factors. The last identity in (1) is based on the assumption that the effects of U and V on Y can be factorized.
Both kernels satisfy the normalization condition K U (U, U) = 1 and K Z (Z, Z) = 1 for all U ∈ R I and Z ∈ R J j=1 l j . This framework is flexible because it can accommodate various kernel functions to capture diverse patterns.
While the assumption of latent vectors may initially appear restrictive, we show that this modeling framework integrates numerous established approaches (Qian et al., 2008;Deng et al., 2017;Zhang et al., 2020;Tao et al., 2021) as special cases. Furthermore, its inherent generality offers potential for further methodological advancements and applications.
In the following, we provide a detailed discussion to establish connections between the framework and some existing methods in the literature.
Multiplicative Linear Kernel. By imposing a multiplicative structure among qualitative variables and adopting the linear kernel for continuous variables (Rojo-Álvarez et al., 2018), the correlation defined by the latent vectors is
• Case I (l j = a j ). The covariance structure proposed by Qian et al. (2008) is defined as
where τ (j) v,v ′ a j ×a j are J semi-positive definite matrices with unit diagonal elements (SPDUDE). Here, τ
v,v ′ represents the correlation between levels v and v ′ for the jth qualitative factor. By using the Cholesky decomposition (Pinheiro and Bates, 1996), we can represent τ (j) v j ,v
This content is AI-processed based on open access ArXiv data.