Algorithms and Hardness for Linear Algebra on Geometric Graphs

For a function $\mathsf{K} : \mathbb{R}^{d} \times \mathbb{R}^{d} \to \mathbb{R}_{\geq 0}$, and a set $P = { x_1, \ldots, x_n} \subset \mathbb{R}^d$ of $n$ points, the $\mathsf{K}$ graph $G_P$ of $P$ is the complete graph on $n$ nodes where the weight between nodes $i$ and $j$ is given by $\mathsf{K}(x_i, x_j)$. In this paper, we initiate the study of when efficient spectral graph theory is possible on these graphs. We investigate whether or not it is possible to solve the following problems in $n^{1+o(1)}$ time for a $\mathsf{K}$-graph $G_P$ when $d < n^{o(1)}$: $\bullet$ Multiply a given vector by the adjacency matrix or Laplacian matrix of $G_P$ $\bullet$ Find a spectral sparsifier of $G_P$ $\bullet$ Solve a Laplacian system in $G_P$’s Laplacian matrix For each of these problems, we consider all functions of the form $\mathsf{K}(u,v) = f(|u-v|_2^2)$ for a function $f:\mathbb{R} \rightarrow \mathbb{R}$. We provide algorithms and comparable hardness results for many such $\mathsf{K}$, including the Gaussian kernel, Neural tangent kernels, and more. For example, in dimension $d = Ω(\log n)$, we show that there is a parameter associated with the function $f$ for which low parameter values imply $n^{1+o(1)}$ time algorithms for all three of these problems and high parameter values imply the nonexistence of subquadratic time algorithms assuming Strong Exponential Time Hypothesis ($\mathsf{SETH}$), given natural assumptions on $f$. As part of our results, we also show that the exponential dependence on the dimension $d$ in the celebrated fast multipole method of Greengard and Rokhlin cannot be improved, assuming $\mathsf{SETH}$, for a broad class of functions $f$. To the best of our knowledge, this is the first formal limitation proven about fast multipole methods.

💡 Research Summary

The paper initiates a systematic study of linear‑algebraic operations on dense geometric graphs, where the edge weight between two points x_i, x_j in ℝ^d is given by a kernel function K(x_i,x_j). The authors focus on kernels of the form K(u,v)=f(‖u−v‖_2^2) – a broad class that includes Gaussian, exponential, polynomial, logarithmic kernels, and even the Neural Tangent Kernel after suitable reformulation. Three fundamental problems are examined: (1) multiplying a vector by the adjacency or Laplacian matrix of the kernel graph, (2) constructing a spectral sparsifier, and (3) solving a Laplacian linear system. The goal is to achieve n^{1+o(1)} time (almost linear) algorithms when the ambient dimension d is sub‑polynomial in n (e.g., d=O(log n) or d=o(log n)).

Key technical contributions

1. Parameterization of kernel difficulty – Two quantitative measures of f are introduced: (i) the ε‑approximate polynomial degree deg_ε(f), the smallest degree of a polynomial that ε‑approximates f over the relevant distance range, and (ii) the multiplicative Lipschitz constant L_m(f), which bounds how rapidly f changes under scaling of its argument. Combining these yields a single difficulty parameter p_f that governs algorithmic feasibility.

2. High‑dimensional algorithms (d≈Θ(log n)) – When p_f is small (i.e., f has a low‑degree polynomial approximation and modest Lipschitz growth), the authors develop a “kernel‑method” that expands f(‖u−v‖²) into a sum of inner‑product monomials. Each monomial corresponds to a low‑rank tensor product that can be evaluated in poly(d,log n) time per vector using fast Fourier techniques and a dimension‑aware variant of the Fast Multipole Method (FMM). Consequently, matrix‑vector multiplication and Laplacian solving run in poly(d,log n)·n^{1+o(1)} time, while spectral sparsification runs in poly(d,log n,1/ε)·n^{1+o(1)} time.

3. Low‑dimensional algorithms (d=o(log n)) – By applying Johnson‑Lindenstrauss dimensionality reduction and ε‑net constructions, the authors obtain algorithms whose running time is ( log n )^{O(d)}·n^{1+o(1)} for multiplication, solving, and sparsification. This matches the high‑dimensional results when d is constant but degrades gracefully as d grows.

4. Hardness under SETH – For kernels with large p_f, the paper shows reductions from classic SETH‑hard problems such as Orthogonal Vectors and All‑Pairs‑Shortest‑Path. The reductions demonstrate that any sub‑quadratic algorithm for the three target problems would violate SETH. Intuitively, when f cannot be approximated by a low‑degree polynomial or grows too sharply, computing K(u,v) essentially requires explicit distance comparisons, which are known to be SETH‑hard.

5. Limits of Fast Multipole Methods – The authors prove that the exponential dependence on dimension d in the original Greengard‑Rokhlin FMM is optimal for a wide class of kernels. Assuming SETH, no algorithm can improve the exp(O(d)) factor for kernels such as the Gaussian (exp(−‖x−y‖²)) or more general radial basis functions. This is the first formal evidence that the “curse of dimensionality” for FMM is a genuine complexity barrier.

Applications – The results immediately impact three representative tasks: (i) n‑body simulations (gravity, electrostatics) where each coordinate requires O(d) adjacency‑matrix multiplications; (ii) spectral clustering using Gaussian kernels, where a sparsifier enables fast eigenvector computation; and (iii) semi‑supervised learning via Laplacian regularization, where solving a Laplacian system yields label propagation. For each, the paper provides concrete parameter regimes (values of p_f, d, ε) under which near‑linear time algorithms are provably possible, and regimes where such speedups are impossible unless SETH fails.

Overall significance – By introducing the unified difficulty parameter p_f and providing a complete dichotomy—efficient almost‑linear algorithms versus SETH‑based lower bounds—the paper offers a clear theoretical map for algorithm designers working with kernel graphs. It clarifies when fast kernel methods (e.g., low‑rank approximations, multipole expansions) can be expected to succeed and when the intrinsic hardness of the kernel precludes sub‑quadratic performance. Moreover, the hardness result for FMM settles a long‑standing open question about the optimality of its dimensional dependence. The work thus bridges spectral graph theory, fine‑grained complexity, and practical kernel‑based machine‑learning algorithms, setting a foundation for future research on scalable linear algebra in high‑dimensional geometric settings.