Nonlinear Dimensionality Reduction Methods in Climate Data Analysis

Linear dimensionality reduction techniques, notably principal component analysis, are widely used in climate data analysis as a means to aid in the interpretation of datasets of high dimensionality. These linear methods may not be appropriate for the analysis of data arising from nonlinear processes occurring in the climate system. Numerous techniques for nonlinear dimensionality reduction have been developed recently that may provide a potentially useful tool for the identification of low-dimensional manifolds in climate data sets arising from nonlinear dynamics. In this thesis I apply three such techniques to the study of El Nino/Southern Oscillation variability in tropical Pacific sea surface temperatures and thermocline depth, comparing observational data with simulations from coupled atmosphere-ocean general circulation models from the CMIP3 multi-model ensemble. The three methods used here are a nonlinear principal component analysis (NLPCA) approach based on neural networks, the Isomap isometric mapping algorithm, and Hessian locally linear embedding. I use these three methods to examine El Nino variability in the different data sets and assess the suitability of these nonlinear dimensionality reduction approaches for climate data analysis. I conclude that although, for the application presented here, analysis using NLPCA, Isomap and Hessian locally linear embedding does not provide additional information beyond that already provided by principal component analysis, these methods are effective tools for exploratory data analysis.

💡 Research Summary

**
The paper investigates whether modern nonlinear dimensionality‑reduction (NLDR) techniques can reveal additional structure in climate data beyond what is captured by the widely used linear method, principal component analysis (PCA). The focus is on El Niño Southern Oscillation (ENSO) variability, examined through tropical Pacific sea‑surface temperature (SST) and thermocline depth fields. Three NLDR algorithms are applied: (1) nonlinear principal component analysis (NLPCA) implemented as an auto‑encoder neural network, (2) Isomap, which preserves geodesic distances on a nearest‑neighbour graph and then performs multidimensional scaling, and (3) Hessian locally linear embedding (Hessian LLE), which retains second‑order curvature information in a locally linear approximation.

Data and preprocessing
Observational data cover the 1979‑2005 period, using ERSST v3b for SST and TAO/PMEL measurements for thermocline depth. Model data are drawn from the CMIP3 multi‑model ensemble, comprising more than fifteen coupled atmosphere‑ocean general circulation models (GCMs). All series are detrended, seasonally averaged, and standardized before analysis.

Methodological details
For NLPCA the network architecture consists of an input layer, a two‑dimensional bottleneck (the “nonlinear PCs”), and a reconstruction layer. Training minimizes the mean‑squared reconstruction error using back‑propagation with weight decay. In Isomap a k‑nearest‑neighbour graph (k varied between 5 and 15) is built, geodesic distances are computed via Dijkstra’s algorithm, and classical multidimensional scaling projects the distance matrix onto a two‑dimensional Euclidean space. Hessian LLE first computes local tangent spaces for each point’s k‑neighbourhood, then forms a Hessian estimator that captures curvature; the eigenvectors of the resulting generalized eigenproblem provide the low‑dimensional embedding.

Evaluation metrics
The authors assess (i) reconstruction error relative to the original high‑dimensional fields, (ii) the fraction of total variance explained by each extracted mode, and (iii) the physical realism of the modes by comparing their spatial patterns and temporal evolution with the canonical ENSO signature (east‑west SST dipole, thermocline shoaling in the east, deepening in the west). Both observational and model outputs are examined to test the robustness of the NLDR results across data sources.

Results
NLPCA’s first nonlinear mode reproduces the spatial pattern of the leading EOF (PC1) from linear PCA almost exactly; the second mode mirrors PC2. The reconstruction error is modestly lower (≈5 % reduction) than PCA, but no new dynamical mode emerges. Isomap preserves pairwise distances very well, yet the resulting two‑dimensional manifold is essentially a rotated version of the linear PCA subspace; varying k does not change this outcome, indicating that the underlying data manifold is close to linear. Hessian LLE, despite its theoretical ability to capture curvature, suffers from instability due to the limited sample size (27 annual means) and observational noise; its embeddings are noisy and do not provide clearer ENSO modes than PCA. Across all three NLDR methods, the dominant ENSO variability is captured, but the information content is not substantially richer than that obtained with standard PCA.

Discussion
The findings suggest that, for the ENSO phenomenon, the dominant variability resides on a nearly linear subspace of the high‑dimensional climate state space. Consequently, sophisticated NLDR tools do not automatically yield additional insight when applied to relatively short, noisy climate records. The study also highlights practical challenges: NLDR algorithms are sensitive to the choice of hyper‑parameters (k, bottleneck dimension, regularisation), require ample data to estimate manifolds reliably, and can be destabilised by measurement error.

Conclusions and future work
While NLPCA, Isomap, and Hessian LLE proved effective for exploratory analysis—offering visualisations and confirming the low‑dimensional nature of ENSO—they did not outperform PCA in extracting new modes of variability. The authors recommend applying NLDR techniques to longer, higher‑resolution datasets (e.g., paleoclimate reconstructions, high‑frequency satellite products) where the underlying manifolds may be more curved. They also suggest integrating automated hyper‑parameter optimisation, noise‑reduction schemes, and hybrid approaches that combine linear and nonlinear components to fully exploit the potential of NLDR in climate science.