Revisiting algorithms for generating surrogate time series

The method of surrogates is one of the key concepts of nonlinear data analysis. Here, we demonstrate that commonly used algorithms for generating surrogates often fail to generate truly linear time series. Rather, they create surrogate realizations with Fourier phase correlations leading to non-detections of nonlinearities. We argue that reliable surrogates can only be generated, if one tests separately for static and dynamic nonlinearities.

💡 Research Summary

The paper revisits the three most widely used surrogate‑generation algorithms—Fourier‑transform (FT) surrogates, amplitude‑adjusted FT (AAFT) surrogates, and iterative AAFT (IAAFT) surrogates—and demonstrates that the latter two do not fulfill the fundamental assumption of phase randomness. In surrogate‑based tests for nonlinearity, the surrogate data must share the linear properties of the original series (autocorrelation function or, equivalently, power spectrum) while possessing completely uncorrelated, uniformly distributed Fourier phases. FT surrogates satisfy this by construction; AAFT and IAAFT, however, introduce a rank‑ordering step (AAFT) and an iterative amplitude‑matching step (IAAFT) that alter the phase distribution.

To expose this flaw, the authors analyse two very different real‑world time series: (i) an X‑ray light curve of the narrow‑line Seyfert 1 galaxy Mrk 766 obtained with XMM‑Newton (≈1,540 data points) and (ii) daily returns of the Dow Jones Industrial Average spanning 1896–2012 (≈29,070 points). For each series they generate 200 realizations of FT, AAFT, and IAAFT surrogates. They then examine Fourier phases using phase maps (pairs (φ_k, φ_{k+Δ})) and compute the phase‑lag correlation coefficient c(Δ) for Δ = 1 … 10.

The FT surrogates produce phase maps that are uniformly scattered, and c(Δ) fluctuates around zero within the expected statistical limits. In contrast, both AAFT and IAAFT surrogates show clear structures in the phase maps, and c(Δ) deviates significantly from zero for many lags. For the AGN light curve, up to 138 IAAFT realizations and 110 AAFT realizations exceed the 3σ confidence interval; similar numbers are found for the financial series. This demonstrates that the rank‑ordering (AAFT) and the iterative amplitude‑adjustment (IAAFT) steps generate spurious phase correlations.

The authors then assess how these phase correlations affect a nonlinear statistic: the nonlinear prediction error (NLPE). Using delay‑coordinate embedding (dimension d = 3, delay τ chosen from autocorrelation zero‑crossing or first minimum), they compute NLPE for each surrogate and for the original data. For the AGN series, the correlation between c(Δ=1) and NLPE is strongly negative (AAFT: –0.68, IAAFT: –0.94), indicating that the NLPE is largely driven by the induced phase correlations rather than genuine dynamical nonlinearity. The financial series shows no clear trend, but the overall NLPE values are still inflated relative to FT surrogates.

Finally, the authors construct a significance measure S(ψ) = (ψ_original – ⟨ψ_surrogate⟩)/σ_surrogate, where ψ is the NLPE. FT surrogates yield highly significant detections of nonlinearity (S ≈ 8 for Mrk 766, S ≈ 6 for Dow Jones), well above the 3σ threshold. Both AAFT and IAAFT produce S values well below the threshold, leading to a false conclusion that the original series are compatible with linear models. Consequently, models that would be rejected by FT‑surrogate tests (e.g., global disk‑oscillation models for AGN variability) might be mistakenly accepted when using AAFT/IAAFT.

The paper concludes that surrogate‑based tests must separate static nonlinearity (non‑Gaussian amplitude distribution) from dynamic nonlinearity (temporal correlations). Dynamic nonlinearity should be tested with FT surrogates applied to a rank‑ordered (Gaussianized) version of the data, while static nonlinearity can be assessed by testing the Gaussianity of the amplitude distribution itself. Since no existing algorithm simultaneously preserves the power spectrum, amplitude distribution, and phase randomness, previous studies that relied on AAFT or IAAFT may need to be re‑examined, especially when searching for weak nonlinear signatures.

In summary, the work provides a rigorous critique of AAFT and IAAFT surrogate generation, shows that phase correlations are a generic by‑product of these methods, and offers practical guidance for reliable nonlinear time‑series analysis across fields such as astrophysics, finance, and physiology.