Strong Linear Baselines Strike Back: Closed-Form Linear Models as Gaussian Process Conditional Density Estimators for TSAD

Research in time series anomaly detection (TSAD) has largely focused on developing increasingly sophisticated, hard-to-train, and expensive-to-infer neural architectures. We revisit this paradigm and show that a simple linear autoregressive anomaly score with the closed-form solution provided by ordinary least squares (OLS) regression consistently matches or outperforms state-of-the-art deep detectors. From a theoretical perspective, we show that linear models capture a broad class of anomaly types, estimating a finite-history Gaussian process conditional density. From a practical side, across extensive univariate and multivariate benchmarks, the proposed approach achieves superior accuracy while requiring orders of magnitude fewer computational resources. Thus, future research should consistently include strong linear baselines and, more importantly, develop new benchmarks with richer temporal structures pinpointing the advantages of deep learning models.

💡 Research Summary

The paper challenges the prevailing belief in the time‑series anomaly detection (TSAD) community that increasingly sophisticated deep neural networks are necessary for state‑of‑the‑art performance. Instead, the authors demonstrate that a simple linear autoregressive model trained with ordinary least squares (OLS) – optionally regularized with a tiny ridge term – and its low‑rank extension (Reduced‑Rank Regression, RRR) consistently match or surpass the accuracy of recent deep detectors while requiring a fraction of the computational resources.

Problem formulation and model
Given a univariate or multivariate series {yₜ}ₜ₌₁ᵀ (yₜ ∈ ℝᵈ), the authors construct lagged feature vectors xₜ =