Local EGOP for Continuous Index Learning

We introduce the setting of continuous index learning, in which a function of many variables varies only along a small number of directions at each point. For efficient estimation, it is beneficial for a learning algorithm to adapt, near each point $x$, to the subspace that captures the local variability of the function $f$. We pose this task as kernel adaptation along a manifold with noise, and introduce Local EGOP learning, a recursive algorithm that utilizes the Expected Gradient Outer Product (EGOP) quadratic form as both a metric and inverse-covariance of our target distribution. We prove that Local EGOP learning adapts to the regularity of the function of interest, showing that under a supervised noisy manifold hypothesis, intrinsic dimensional learning rates are achieved for arbitrarily high-dimensional noise. Empirically, we compare our algorithm to the feature learning capabilities of deep learning. Additionally, we demonstrate improved regression quality compared to two-layer neural networks in the continuous single-index setting.

💡 Research Summary

The paper introduces a novel learning setting called Continuous Index Learning (CIL), which generalizes multi‑index learning to the case where the underlying data lie near a smooth, low‑dimensional manifold M embedded in a high‑dimensional ambient space. Under the Supervised Noisy Manifold Hypothesis (SNMH), the target function f varies only along the tangent directions of M and is constant in the normal directions. Consequently, at any query point x* the function can be written as f(x)=g(π(x)), where π(x) denotes the nearest‑point projection onto M.

To exploit this structure, the authors propose a kernel‑based regression method whose metric adapts locally to the subspace where f actually changes. The key tool is the Expected Gradient Outer Product (EGOP) matrix L(μ)=E_μ