Orthogonal-by-construction augmentation of physics-based input-output models

This paper proposes a novel orthogonal-by-construction parametrization for augmenting physics-based input-output models with a learning component in an additive sense. The parametrization allows to jointly optimize the parameters of the physics-based model and the learning component. Unlike the commonly applied additive (parallel) augmentation structure, the proposed formulation eliminates overlap in representation of the system dynamics, thereby preserving the uniqueness of the estimated physical parameters, ultimately leading to enhanced model interpretability. By theoretical analysis, we show that, under mild conditions, the method is statistically consistent and guarantees recovery of the true physical parameters. With further analysis regarding the asymptotic covariance matrix of the identified parameters, we also prove that the proposed structure provides a clear separation between the physics-based and learning components of the augmentation structure. The effectiveness of the proposed approach is demonstrated through simulation studies, showing accurate reproduction of the data-generating dynamics without sacrificing consistent estimation of the physical parameters.

💡 Research Summary

The paper addresses a fundamental issue in hybrid modeling where a physics‑based input‑output (IO) model is augmented with a data‑driven learning component, typically an artificial neural network (ANN). In the conventional additive (parallel) augmentation, the baseline model ϕ(x)θ_b and the learning component fₐ(x;θ_a) are summed, and both parameter sets are jointly optimized. Because modern learning modules are heavily over‑parameterized, they can capture dynamics that the baseline model is already capable of representing. Consequently, the physical parameters θ_b may drift to unrealistic values, compromising interpretability and extrapolation performance. Existing remedies, such as regularization penalties or orthogonal‑projection regularizers, require careful tuning of hyper‑parameters and introduce a trade‑off between model accuracy and component separation.

The authors propose a “orthogonal‑by‑construction” parametrization that eliminates any overlap between the baseline and learning components at the data level. They define an auxiliary parameter θ_aux = (ΦᵀΦ)⁻¹ΦᵀFₐ, where Φ is the regressor matrix of the baseline model evaluated on the training data and Fₐ is the vector of raw ANN outputs. By projecting the ANN output onto the orthogonal complement of the column space of Φ, the modified learning term becomes ˜Fₐ = (I – Φ(ΦᵀΦ)⁻¹Φᵀ)Fₐ. Lemma 3 proves that Φᵀ˜Fₐ = 0, guaranteeing strict orthogonality on the training set.

This construction yields two key theoretical benefits. First, under a full‑rank condition on Φ (persistent excitation) and an identifiability condition on the baseline regressor, the least‑squares estimate of θ_b is statistically consistent and converges to the true physical parameters θ*_b. Second, the covariance between the estimated baseline parameters and the learning parameters is zero, meaning that uncertainty in the learning component does not contaminate the physical parameter estimates. These results are formalized in Theorem 4 and rely on an orthogonality condition between the baseline regressor and the unmodeled dynamics (Condition 4) that can be satisfied by appropriate experiment design.

The methodology is validated through two simulation studies. In the first, a linear baseline model is combined with a single‑layer ANN. The standard additive approach admits infinitely many (θ_b, W) pairs that achieve the same loss, whereas the orthogonal‑by‑construction method uniquely recovers the true θ_b by removing the ANN contribution from the Φ subspace. In the second study, a nonlinear baseline model is augmented with a multilayer ANN; again, the proposed scheme maintains prediction accuracy while dramatically reducing the estimation error of the physical parameters compared with the conventional additive scheme.

In summary, the paper introduces a principled framework for hybrid system identification that guarantees component orthogonality without the need for additional regularization hyper‑parameters. By embedding the orthogonal projection directly into the parametrization, it preserves the interpretability of physics‑based parameters, ensures statistical consistency, and improves overall model performance. This contribution is highly relevant for practitioners seeking reliable, interpretable models in control, robotics, and other engineering domains where physics‑based insight must be combined with modern machine‑learning flexibility.