Hitting and commute times in large graphs are often misleading

Given an undirected, weighted graph G = (V, E) with n vertices, the commute distance between two vertices u and v is defined as the expected time it takes the natural random walk starting in vertex u to travel to vertex v and back to u. It is equivalent (up to a constant) to the resistance distance, which interprets the graph as an electrical network and defines the distance between vertices u and v as the effective resistance between these vertices. See below for exact definitions and Doyle and Snell (1984), Klein and Randic (1993), Xiao and Gutman (2003), Fouss et al. (2006) for background reading. The commute distance is very popular in many different fields of computer science and beyond. As examples consider the tasks of graph embedding (Guattery, 1998, Saerens et al., 2004, Qiu and Hancock, 2006, Wittmann et al., 2009), graph sparsification (Spielman and Srivastava, 2008), social network analysis (Liben-Nowell and Kleinberg, 2003), proximity search (Sarkar et al., 2008), collaborative filtering (Fouss et al., 2006), clustering (Yen et al., 2005), semisupervised learning (Zhou and Schölkopf, 2004), dimensionality reduction (Ham et al., 2004), image processing (Qiu and Hancock, 2005), graph labeling (Herbster andPontil, 2006, Cesa-Bianchi et al., 2009), theoretical computer science (Aleliunas et al., 1979, Chandra et al., 1989, Avin and Ercal, 2007, Cooper and Frieze, 2003, 2005, 2007, 2009), and various applications in chemometrics and bioinformatics (Klein and Randic, 1993, Ivanciuc, 2000, Fowler, 2002, Roy, 2004, Guillot et al., 2009). The commute distance has many nice properties, both from a theoretical and a practical point of view. It is a Euclidean distance function and can be computed in closed form. As opposed to the shortest path distance, it takes into account all paths between u and v, not just the shortest one. As a rule of thumb, the more paths connect u with v, the smaller their commute distance becomes. Hence it supposedly satisfies the following, highly desirable property: Property ( ): Vertices in the same "cluster" of the graph have a small commute distance, whereas vertices in different clusters of the graph have a large commute distance to each other. Consequently, the commute distance is considered a convenient tool to encode the cluster structure of the graph. In this paper we study how the commute distance behaves when the size of the graph increases. Our main result is that if the graph is large enough, then in many graphs the hitting times and commute distances can be approximated by an extremely simple formula with very high accuracy. Namely, denoting by H uv the expected hitting time and by C uv the commute distance between two vertices u and v, by d u the degree of vertex u, and by vol(G) the volume of the graph, we show that if the graph gets large enough, for all vertices u = v, On the one hand, we prove these results for arbitrary fixed, large graphs (Proposition 5). Here the quality of the approximation depends on geometric quantities describing the graph (such as minimal and maximal degrees, the spectral gap, and so on). The main part of the paper prove that results hold with probability tending to 1, as n → ∞, in all major classes of random graphs: random geometric graphs (k-nearest neighbor graphs, ε-graphs, and Gaussian similarity graphs) and for random graphs with given expected degrees (in particular, also Erdős-Rényi graphs with and without planted partitions). As a rule of thumb, our approximation results hold whenever the minimal degree in the graph increases with n (for example, as log(n) in random geometric graphs or as log 2 (n) in random graphs with given expected degrees). In order to make our results as accessible as possible to a wide range of computer scientists, we present two different strategies to prove our results: one based on flow arguments on electrical networks and another based on spectral arguments. While the former approach leads to tighter bounds, the latter is more general. An important step on the way is that we prove bounds on the spectral gap in all classes of random geometric graphs. This is interesting by itself as the spectral gap governs many important properties and processes on graphs. In this generality, the bounds on the spectral gaps are new. Our results have important consequences. Hitting and commute times in large graphs are often misleading. On the negative side, our approximation result shows that contrary to popular belief, the commute distance does not take into account any global properties of the data, at least if the graph is "large enough". It just considers the local density (the degree of the vertex) at the two vertices, nothing else. The resulting large sample commute distance dist(u, v) = 1/d u + 1/d v is completely meaningless as a distance on a graph. For example, all data points have the same nearest neighbor (namely, the vertex with the largest degree), the same second-nearest neighbor (the vertex with the second-largest degree), and so on. In particular, one of the main motivations to use the commute distance, Property ( ), no longer holds when the graph becomes large enough. Even more disappointingly, computer simulations show that n does not even need to be very large before ( ) breaks down. Often, n in the order of 1000 is already enough to make the commute distance very close to its approximation expression. This effect is even stronger if the dimensionality of the underlying data space is large. Consequently, even on moderate-sized graphs, the use of the raw commute distance should be discouraged. Efficient computation of approximate commute distances. In some applications the commute distance is not used as a distance function, but as a tool to encode the connectivity properties of a graph, for example in graph sparsification (Spielman and Srivastava, 2008) or when computing bounds on mixing or cover times (Aleliunas et al., 1979, Chandra et al., 1989, Avin and Ercal, 2007, Cooper and Frieze, 2009) or graph labeling (Herbster andPontil, 2006, Cesa-Bianchi et al., 2009). To obtain the commute distance between all points in a graph one has to compute the pseudo-inverse of the graph Laplacian matrix, an operation of time complexity O(n 3 ). This is very many paths resistance 1 edges, each with many outgoing d t many outgoing edges, each with resistance 1 d s s t Figure 1: Electrical network intuition: The effective resistance between s and t is dominated by the edges adjacent to s and t. prohibitive in large graphs. To circumvent the matrix inversion, several approximations of the commute distance have been suggested in the literature (Spielman and Srivastava, 2008, Sarkar and Moore, 2007, Brand, 2005). Our results lead to a much simpler and well-justified way of approximating the commute distance on large random geometric graphs. We start our paper with Section 2 that tries to convey our main results and techniques on a very high level. Then, after introducing general definitions and notation (Section 3), we present our main results in Section 4. This section is divided into two parts (flow based part and spectral part). All proofs are presented in Sections 5 and 6. A final discussion can be found in Section 7. For the convenience of the reader, some basic facts on random geometric graphs are presented in the appendix. Parts of this work is built on our conference paper von Luxburg et al. (2010). Before diving into technicalities, we would like to present our results in an intuitive, non-technical way. Readers interested in crisp theorems are encouraged to skip this section right away. Informally the main result of our paper is the following: Main result: Consider a "large" graph that is "reasonably strongly' connected. In such a graph, the hitting times and commute distances between any two vertices u and v can be approximated by the simple expressions In this section we want to present some intuitive arguments to understand why this makes sense. In order to show a broad picture and to make our results accessible to a general audience, we are going to present two completely different approaches in our paper. Consider an unweighted graph as an electrical network where each edge has resistance 1. We want to compute the effective resistance between two fixed vertices s and t by exploiting the electrical laws. Resistances in series add up, that is for two resistances R 1 and R 2 in series we get the overall resistance R = R 1 + R 2 . Resistances in parallel lines satisfy 1/R = 1/R 1 + 1/R 2 . Now consult the unweighted electrical network in Figure 1. Consider the vertex s and all edges from s to its d s neighbors. The resistance "spanned" by these d s parallel edges satisfies 1/R = ds i=1 1 = d s , that is R = 1/d s . Similarly for t. Between the neighbors of s and the ones of t there are very many paths. It turns out that the contribution of these paths to the resistance is negligible (essentially, we have so many wires between the two neighborhoods that electricity can flow nearly freely). So the overall effective resistance between s and t is dominated by the edges adjacent to i and j with contributions 1/d s + 1/d t . The main theorems derived from the electrical network approach are Theorems 3 and 4. In order to prove them, we bound the electrical resistance between two vertices using flow arguments. The Figure 2: Random walk intuition: Between its start and target vertex (black crosses), the random walk wanders around so long that by the time it finally arrives at its target it has already "forgotten" where it started from. overall idea is that we construct a unit flow between s and t that uses as many paths as possible. From the technical side, this approach has the advantage that we can throw away irrelevant parts of the graph -we can concentrate on a "valid region" that contains s, t, and many paths between s and t. For this reason, we need less assumptions on the geometry of the underlying space "close to its boundary". We explicitly construct such flows for random geometric graphs. The idea is to place a grid on the underlying space and control the flow between different cells of the grid. As far as we can see, this technique can only be used to bound the resistance distance R ij , it does not work for the individual hitting times H ij or H ji . Another approach to understand our convergence results is based on random walks. Essentially, our results for the hitting times H uv say that regardless at which vertex u we start, the time to hit vertex v just depends on the degree of v. What happens is that as the graph gets large, the random walk can explore so many paths that by the time it is close to v it "has forgotten" where it came from (cf. Figure 2). This is why the hitting time does not depend on u. Once the random walk is in the vicinity of v, the question is just whether it exactly hits v or whether it passes close to v without hitting it. Intuitively, the likelihood to hit v is inversely proportional to the density of the graph close to v: if there are many edges in the neighborhood of v, then it is easier to hit v than if there are only few edges. This is how the inverse degree comes into play. Stated slightly differently, the random walk has already mixed before it hits v. For this reason, the hitting time does not depend on u. All that is left is some component depending on v. Notably, this component exactly coincides with the mean return time of v (the expected time it takes a random walk that starts at v to return to v), which is given as vol(G)/d v . In the light of our explanation it is reasonable to expect that the quality of our approximation depends on the mixing time of the random walk, and the latter is known to be governed by the size of the spectral gap, in particular the quantity 1 -λ 2 (see below for exact definitions). Indeed, we will see in our Key Proposition 5 that 1 -λ 2 is exactly the quantity that governs the deviation bound for the hitting and commute times. If 1 -λ 2 is small, then the graph is too well-clustered, has a large mixing time, and our approximation guarantee gets worse. The spectral approach leads to the main theorems in Section 4.2. We first have to express the commute time in terms of a spectral representation of the graph (Proposition 5). To make use of this proposition we need a lower bound on the spectral gap 1 -λ 2 of the graph. To bound the spectral gap in random geometric graphs we use path-based arguments as well, namely we use the canonical path technique of Diaconis and Stroock (1991). Here one has to construct a set of "canonical paths" between each pair of vertices in the graph. The goal is to distribute these paths "as well as possible" over the graph. As in the case above we use a grid to control the paths between different cells of this grid. This is very reminiscent of the technique described above. However, an important difference is that we now need to consider paths between all pairs of points (we have to bound the spectral gap of the whole graph) instead of just paths between s and t. In the language of flows, instead of looking at a unit flow from s to t we would have to use multi-commodity flows between all pairs of vertices instead of a single flow from s to t (cf. Sinclair, 1992, Diaconis andSaloff-Coste, 1993). For this reason, we need stronger assumptions on the geometry of the underlying space. For the case of random graphs with expected degrees, we build on results about the spectral gap from the literature. As in the electrical network approach, we need to ensure that these graphs are "strongly enough" connected. This will be achieved by requiring that the minimal vertex degree in the graph is "large enough" (with respect to the number n of vertices). Our results hold for any arbitrary degree distribution, as soon as the minimal degree grows slowly with n. The advantage of the spectral approach is that it is very general. It works for any kind of graph, and as opposed to the electrical network approach can also be used to treat the hitting times directly. The technical disadvantage is that we cannot "throw away" irrelevant parts of the graph as in the electrical network approach (because no part of the graph is irrelevant to the gap), leading to slightly worse bounds. There are two major limitations to our results: • Our approximation results only hold if the graph is "reasonably strongly" connected and does not have too large a bottleneck. This ensures that the overall behavior of the commute distance cannot be dominated by a single edge. We can see this in both approaches. In the electrical network approach, the argument that "electricity can nearly flow without resistance" on the "many paths" breaks down if there is a strong bottleneck between u and v which all electricity has to pass. In the spectral approach, a strong bottleneck leads to a small spectral gap, and then the bounds become meaningless as well. • Our results only hold if the minimal degree in the graph is "reasonably large", compared to the number n of vertices. For example, in the random graph models the minimal degree has to grow slowly with n, say as log n. This is to ensure that there are no single vertices that can have extremely high influence on the commute distance. The downside of this condition is that our results do not hold for power law graphs in which the smallest degree is constant. As presented in this intuitive section, it nearly sounds as if our results were obvious. Indeed, in hindsight they seem to be obvious, and this is part of why we like our results so much: they were very surprising when we found them, but can be made plausible to a wide range of people. We would like to stress that all these results were not known before our work, and that the "intuitive explanations" have to be seen as the succession of our technical work. In particular, the technical work presented in the rest of this paper makes explicit all the sloppy terms like "reasonably connected" and "large enough". We consider undirected graphs G = (V, E) that are connected and not bipartite. By n we denote the number of vertices. The adjacency matrix is denoted by W := (w ij ) i,j=1,...,n . In case the graph is weighted, this matrix is also called the weight matrix. All weights are assumed to be non-negative. The minimal and maximal weights in the graph are denoted by w min and w max . By d i := n j=1 w ij we denote the degree of vertex v i . The diagonal matrix D with diagonal entries d 1 , . . . , d n is called the degree matrix, the minimal and maximal degrees are denoted d min and d max . The unnormalized graph Laplacian is given as L := D -W , the normalized one as L sym = D -1/2 LD -1/2 . Consider the natural random walk on G. Its transition matrix is given as P = D -1 W . It is well-known that λ is an eigenvalue of L sym if and only if 1 -λ is an eigenvalue of P . By 1 = λ 1 ≥ λ 2 ≥ . . . ≥ λ n > -1 we denote the eigenvalues of P . The quantity 1 -max{λ 2 , |λ n |} is called the spectral gap of P . The hitting time H uv is defined as the expected time it takes a random walk starting in vertex u to travel to vertex v (where H uu = 0 by definition). The commute distance (commute time) between u and v is defined as C uv := H uv + H vu . Recall that for a symmetric, non-invertible matrix A its Moore-Penrose inverse is defined as A † := (A + U ) -1 -U where U is the projection on the eigenspace corresponding to eigenvalue 0. It is well known that commute times can be expressed in terms of the Moore-Penrose inverse L † of the unnormalized graph Laplacian (e.g., Klein and Randic, 1993, Xiao and Gutman, 2003, Fouss et al., 2006): , where e i is the i-th unit vector in R n . The following representations for commute and hitting times involving the pseudo-inverse L † sym of the normalized graph Laplacian are less well known. Proposition 1 (Closed form expression for hitting and commute times) Let G be a connected, undirected graph with n vertices. The hitting times H ij , i = j, can be computed by and the commute times satisfy Closely related to the commute distance is the resistance distance. Here one interprets the graph as an electrical network where the edges represent resistors. The conductance of a resistor is given by the corresponding edge weight. The resistance distance R uv between two vertices u and v is defined as the effective resistance between u and v in the network. It is well known that the resistance distance coincides with the commute distance up to a constant: C uv = vol(G)R uv . For background reading on resistance and commute distances see Doyle and Snell (1984), Klein and Randic (1993), Xiao and Gutman (2003), Fouss et al. (2006). Our main focus in this paper is the class of geometric graphs. For a deterministic (fixed) geometric graph we consider a fixed set of points X 1 , . . . , X n ∈ R d . These points form the vertices v 1 , . . . , v n of the graph. The edges in the graph are defined such that "neighboring points" are connected. We consider the most popular types of random geometric graphs. In the ε-graph we connect two points whenever their Euclidean distance is less than or equal to ε. In the undirected, symmetric k-nearest neighbor graph we connect v i to v j if X i is among the k nearest neighbors of X j or vice versa. In the mutual k-nearest neighbor graph we connect v i to v j if X i is among the k nearest neighbors of X j and vice versa. Note that by default, the terms ε-and kNN-graph refer to unweighted graphs in our paper. When we treat weighted graphs, we always make it explicit. For a general similarity graph we build a weight matrix between all points based on a similarity function k : R d × R d → R ≥0 , that is we define the weight matrix W with entries w ij = k(X i , X j ) and consider the fully connected graph with weight matrix W . The most popular weight function in applications is the Gaussian similarity function w ij = exp(-X i -X j 2 /h 2 ), where h > 0 is a bandwidth parameter. While these definitions make sense with any fixed set of vertices, we are most interested in the case of random geometric graphs. Here we assume that the underlying set of vertices X 1 , ..., X n has been drawn i.i.d. according to some probability density p on R d . Once the vertices are known, the edges in the graphs are constructed as described above. In the random setting it is convenient to make regularity assumptions in order to be able to control quantities such as the minimal and maximal degrees. Sometimes we need to make these assumptions about the whole underlying space, sometimes just for a selected subset of it. Thus we introduce the following general definition. Definition 2 (Valid region) Let p be any density on R d . We call a connected subset X ⊂ R d a valid region if the following properties are satisfied: 1. The density on X is bounded away from 0, that is for all x ∈ X we have that p(x) ≥ p min > 0 for some constant p min . 2. X has "bottleneck" larger than some value h > 0: the set {x ∈ X : dist(x, ∂X ) > h/2} is connected (here ∂X denotes the topological boundary of X ). 3. The boundary of X is regular in the following sense. We assume that there exist positive constants α > 0 and ε 0 > 0 such that if ε < ε 0 , then for all points x ∈ ∂X we have vol(B ε (x) ∩ X ) ≥ α vol(B ε (x)) (where vol denotes the Lebesgue volume). Essentially this condition just excludes the situation where the boundary has arbitrarily thin spikes. Sometimes we consider a valid region with respect to two points s, t. Here we additionally assume that s and t are interior points of X . In the spectral part of our paper, we always have to make a couple of assumptions that will be summarized by the term general assumptions. They are as follows: First we assume that X := supp(p) is a valid region according to Definition 2. Second, we assume that X does not contain any holes and does not become arbitrarily narrow: there exists a homeomorphism h : X → [0, 1] d and constants 0 < L min < L max < ∞ such that for all x, y ∈ X we have This condition restricts X to be topologically equivalent to the cube. In applications this is not a strong assumption, as the occurrence of "holes" with vanishing probability density is unrealistic due to the presence of noise in the data generating process. More generally we believe that our results can be generalized to other homeomorphism classes, but refrain from doing so as it would substantially increase the amount of technicalities. In the following we denote the volume of the unit ball in R d by η d . For readability reasons, we are going to state our main results using constants c i > 0. These constants are independent of n and the graph connectivity parameter (ε or k or h, respectively) but depend on the dimension, the geometry of X , and p. The values of all constants are determined explicitly in the proofs. They are not the same in different propositions. Our paper comprises two different approaches. In the first approach we analyze the resistance distance by flow based arguments. This technique is somewhat restrictive in the sense that it only works for the resistance distance itself (not the hitting times) and we only apply it to random geometric graphs. The advantage is that in this setting we obtain good convergence conditions and rates. The second approach is based on spectral arguments and is more general. It works for various kinds of graphs and can treat hitting times as well. This comes at the price of slightly stronger assumptions and worse convergence rates. Theorem 3 (Commute distance on ε-graphs) Let X be a valid region with bottleneck h and minimal density p min . For ε ≤ h, consider an unweighted ε-graph built from the sequence X 1 , . . . , X n that has been drawn i.i.d. from the density p. Fix i and j. Assume that X i and X j have distance at least h from the boundary of X , and that the distance between X i and X j is at least 8ε. Then there exist constants c 1 , . . . , c 7 > 0 (depending on the dimension and geometry of X ) such that with probability at least The probability converges to 1 if n → ∞ and nε d / log(n) → ∞. The right hand side of the deviation bound converges to 0 as n → ∞, if Under these conditions, if the density p is continuous and if ε → 0, then Theorem 4 (Commute distance on kNN-graphs) Let X be a valid region with bottleneck h and density bounds p min and p max . Consider an unweighted kNN-graph (either symmetric or mutual) such that (k/n) 1/d /2p max ≤ h, built from the sequence X 1 , . . . , X n that has been drawn i.i.d. from the density p. Fix i and j. Assume that X i and X j have distance at least h from the boundary of X , and that the distance between X i and X j is at least 4(k/n) 1/d /p max . Then there exist constants c 1 , . . . , c 5 > 0 such that with probability at least 1 -c 1 n exp(-c 2 k) the commute distance on both the symmetric and the mutual kNN-graph satisfies The probability converges to 1 if n → ∞ and k/ log(n) → ∞. In case d > 3, the right hand side of the deviation bound converges to 0 if k → ∞ (and under slightly worse conditions in cases d = 3 and d = 2). Under these conditions, if the density p is continuous and if additionally k/n → 0, Let us make a couple of technical remarks about these theorems. To achieve the convergence of the commute distance we have to rescale it appropriately (for example, in the ε-graph we scale by a factor of nε d ). Our rescaling is exactly chosen such that the limit expressions are finite, positive values. Scaling by any other factor in terms of n, ε or k either leads to divergence or to convergence to zero. In case d > 3, all convergence conditions on n and ε (or k, respectively) are the ones to be expected for random geometric graphs. They are satisfied as soon as the degrees grow faster than log(n) (for degrees of order smaller than log(n), the graphs are not connected anyway, see e.g. Penrose, 1999). Hence, our results hold for sparse as well as for dense connected random geometric graphs. In dimensions 3 and 2, our rates are not of the same flavor as in the higher dimensions. For example, in dimension 2 we need nε 3 → ∞ instead of nε 2 → ∞. On the one hand we are not too surprised to get systematic differences between the lowest few dimensions. The same happens in many situations, just consider the example of Polya's theorem about the recurrence/ transience of random walks on grids. On the other hand, these differences might as well be an artifact of our proof methods (and we suspect so at least for the case d = 3; but even though we tried, we did not get rid of the log factor in this case). It is a matter of future work to clarify this. The valid region X has been introduced for technical reasons. We need to operate in such a region in order to be able to control the behavior of the graph, e.g. the minimal and maximal degrees. The assumptions on X are the standard assumptions used regularly in the random geometric graph literature. In our setting, we have the freedom of choosing X ⊂ R d as we want. In order to obtain the tightest bounds one should aim for a valid X that has a wide bottleneck h and a high minimal density p min . In general this freedom of choosing X shows that if two points are in the same high-density region of the space, the convergence of the commute distance is very fast, while it gets slower if the two points are in different regions of high density separated by a bottleneck. We stated the theorems above for a fixed pair i, j. However, they also hold uniformly over all pairs i, j that satisfy the conditions in the theorem (with exactly the same statement). The reason is that the main probabilistic quantities that enter the proofs are bound on the minimal and maximal degrees, which of course hold uniformly. The representation of the hitting and commute times in terms of the Moore-Penrose inverse of the normalized graph Laplacian (Proposition 1) can be used to derive the following key proposition that is the basis for all further results in this section. Proposition 5 (Absolute and relative bounds in any fixed graph) Let G be a finite, connected, undirected, possibly weighted graph that is not bipartite. (1) (2) We would like to point out that even though the bound in Part 3 of the proposition is reminiscent to statements in the literature, it is much tighter. Consider the following formula from Lovász (1993) 1 2 that can easily be rearranged to the following bound: The major difference between our bound (2) and Lovasz' bound (3) is that while the latter has the term d min in the denominator, our bound has the term d 2 min in the denominator. This makes all of a difference: in the graphs under considerations our bound converges to 0 whereas Lovasz' bound diverges. In the following we are going to apply Proposition 5 to various random geometric graphs. Next to some standard results about the degrees and number of edges in random geometric graphs, the main ingredients are the following bounds on the spectral gap in random geometric graphs. These bounds are of independent interest because the spectral gap governs many important properties and processes on graphs. Theorem 6 (Spectral gap of the ε-graph) Suppose that the general assumptions hold. Then there exist constants c 1 , . . . , c 6 > 0 such that with probability at least If nε d / log n → ∞, then this probability converges to 1. Theorem 7 (Spectral gap of the kNN-graph) Suppose that the general assumptions hold. Then for both the symmetric and the mutual kNN-graph there exist constants c 1 , . . . , c 4 > 0 such that with probability at least 1 -c 1 n exp(-c 2 k), If k/ log n → ∞, then the probability converges to 1. At first glance it seems surprising that the geometry of the underlying space X does not affect the order of magnitude of the spectral gap, these quantities only enter the bound in terms of the constants (as can be seen in the proofs below). In particular, for large n the spectral gap does not depend on whether X has a "bottleneck" or not. Intuitively this is the case because if the sample size is large, even a bottleneck with very small diameter contains many sample points and "appears wide" to the random walk. The following theorems characterize the hitting and commute times for ε-and kNN-graphs. They are direct consequences of plugging the results about the spectral gap into Proposition 5. Corollary 8 (Hitting and commute times on ε-graphs) Assume that the general assumptions hold. Consider an unweighted ε-graph built from the sequence X 1 , . . . , X n drawn i.i.d. from the density p. Then there exist constants c 1 , . . . , c 5 > 0 such that with probability at least 1 - If the density p is continuous and n → ∞, ε → 0 and nε d+2 → ∞, then almost surely. For the commute times, the analogous results hold due to Corollary 9 (Hitting and commute times on kNN-graphs) Assume that the general assumptions hold. Consider an unweighted kNN-graph built from the sequence X 1 , . . . , X n drawn i.i.d. from the density p. Then for both the symmetric and mutual kNN-graph there exist constants c 1 , c 2 , c 3 > 0 such that with probability at least If the density p is continuous and For the commute times, the analogous results hold due to In several applications, ε-graphs or kNN graphs are not used as unweighted graphs, but additionally endowed with edge weights. For example, in the field of machine learning it is common to use Gaussian weights w ij = exp(-X i -X j 2 /h 2 ), where h > 0 is a bandwidth parameter. We can use standard spectral results to prove approximation theorems in such cases. Theorem 10 (Results on fully connected weighted graphs) Consider a fixed, fully connected weighted graph with weight matrix W . Assume that its entries are upper and lower bounded by some constants w min , w max , that is 0 < w min ≤ w ij ≤ w max for all i, j. Then, uniformly for all i, j ∈ {1, ..., n}, i = j, For example, this result can be applied directly to a Gaussian similarity graph (for fixed bandwidth h). The next theorem treats the case of Gaussian similarity graphs with adapted bandwidth h. almost surely. Note that in this theorem, we introduced the scaling factor 1/h d already in the definition of the Gaussian similarity function to obtain the correct density estimate p(X j ) in the limit. For this reason, the resistance results are rescaled with factor n instead of nh d . Consider the general random graph model where the edge between vertices i and j is chosen independently with a certain probability p ij that is allowed to depend on i and j. This model contains very popular random graph models such as the Erdős-Rényi random graph, planted partition graphs, and random graphs with given expected degrees. For this class of random graphs, the following result has been proved recently by Chung and Radcliffe (2011). Theorem 12 (Chung and Radcliffe, 2011) Let G be a random graph where edges between vertices i and j are put independently with probabilities p ij .Consider the normalized Laplacian L sym , and define the expected normalized Laplacian as the matrix Lsym : Here all edges have constant probabilities p ij = p (for simplicity, we also allow for self-edges). Corollary 13 (Results on Erdős-Rényi graphs) Let n → ∞, p = ω(log n/n). Then the rescaled hitting times on the Erdős-Rényi graph converge to a constant: for all vertices u, v in the graph we have Application to planted partition graphs. Next we consider a simple model of an Erdős-Rényi-graph with planted partitions, the planted bisection case. Assume that the n vertices are split into two "clusters" of equal size. We put an edge between two vertices u and v with probability p within if they are in the same cluster and with probability p between < p within if they are in different clusters. For simplicity we allow self-loops. Corollary 14 (Random graph with planted partitions) Consider an Erdős-Rényi graph with planted bisection. Assume that p within = ω(log(n)/n) and p between such that np between → ∞ (arbitrarily slow). Then, for all vertices u, v in the graph This result is a prime example to show how that even though there is a strong cluster structure in the graph, the hitting times and commute distances cannot see this cluster structure any more, once the graph gets too large. Note that the corollary even holds if p between grows much slower than p within . That is, the larger our graph, the more pronounced is the cluster structure. Nevertheless, the commute distance converges to a trivial result. On the other hand, we also see that the speed of convergence is O(np between ), that is, if p between = g(n)/n with a very slow growing function g, then convergence can be very slow. We might need very large graphs before the degeneracy of the commute time will be visible. Application to random graphs with given expected degrees. For a graph of n vertices we have n parameters d1 , ..., dn > 0. For each pair of vertices v i and v j , we independently place an edge between these two vertices with probability di dj / n k=1 dk . It is easy to see that in this model, vertex v i has expected degree di (cf. Section 5.3. in Chung and Lu, 2006 for background reading). Corollary 15 (Results on random graphs with expected degrees) Consider any sequence of random graphs with expected degrees such that d min = ω(log n). Then the commute distances satisfy for all i = j, -→ 0, almost surely. For notational convenience, in this section we work with the resistance distance R uv = C uv / vol(G) instead of the commute distance C uv , then we do not have to carry the factor 1/ vol(G) everywhere. It is easy to prove that the resistance distance between two points is lower bounded by the sum of the inverse degrees. Proposition 16 (Lower bound) Let G be a weighted, undirected, connected graph and consider two vertices s and t, s = t. Assume that G remains connected if we remove s and t. Then the effective resistance between s and t is bounded by . Note that if s and t are not connected by a direct edge (that is, w st = 0), then the right hand side simplifies to 1/d s + 1/d t . Proof. The proof is based on Rayleigh's monotonicity principle that states that increasing edge weights in the graph can never increase the effective resistance between two vertices (cf. Corollary 7 in Section IX.2 of Bollobás, 1998). Given our original graph G, we build a new graph G by setting the weight of all edges to infinity, except the edges that are adjacent to s or t (setting the weight of an edge to infinity means that this edge has no resistance any more). This can also be interpreted as taking all vertices except s and t and merging them to one super-node a. Now our graph G consists of three vertices s, a, t with several parallel edges from s to a, several parallel edges from a to t, and potentially the original edge between s and t (if it existed in G). Exploiting the laws in electrical networks (resistances add along edges in series, conductances add along edges in parallel; see Section 2.3 in Lyons and Peres (2010) for detailed instructions and examples) leads to the desired result. This is the part that requires the hard work. Our proof is based on a theorem that shows how the resistance between two points in the graph can be computed in terms of flows on the graph. The following result is taken from Corollary 6 in Section IX.2 of Bollobás (1998). Theorem 17 (Resistance in terms of flows, cf. Bollobás, 1998) Let G = (V, E) be a weighted graph with edge weights w e (e ∈ E). The effective resistance R st between two fixed vertices s and t can be expressed as Note that evaluating the formula in the above theorem for any fixed flow leads to an upper bound on the effective resistance. The key to obtaining a tight bound is to distribute the flow as widely and uniformly over the graph as possible. For the case of geometric graphs we are going to use a grid on the underlying space to construct an efficient flow between two vertices. Let X 1 , ..., X n be a fixed set of points in R d and consider a geometric graph G with vertices X 1 , ..., X n . Fix any two of them, say s := X 1 and t := X 2 . Let X ⊂ R d be a connected set that contains both s and t. Consider a regular grid with grid width g on X . We say that grid cells are neighbors of each other if they touch each other in at least one edge. Definition 18 (Valid grid) We call the grid valid if the following properties are satisfied: 1. The grid width is not too small: Each cell of the grid contains at least one of the points X 1 , ..., X n . 2. The grid width g is not too large: Points in the same or neighboring cells of the grid are always connected in the graph G. 3. Relation between grid width and geometry of X : Define the bottleneck h of the region X as the largest u such that the set {x ∈ X dist(x, ∂X ) > u/2} is connected. We require that √ d g ≤ h (a cube of side length g should fit in the bottleneck). We now prove the following general proposition that gives an upper bound on the resistance distance between vertices in a fixed geometric graph. Proposition 19 (Resistance on a fixed geometric graph) Consider a fixed set of points X 1 , ..., X n in some connected region X ⊂ R d and a geometric graph on X 1 , ..., X n . Assume that X has bottleneck not smaller than h (where the bottleneck is defined as in the definition of a valid grid). Denote s = X 1 and t = X 2 . Assume that s and t can be connected by a straight line that stays inside X and has distance at least h/2 to ∂X . Denote the distance between s and t by d(s, t). Let g be the width of a valid grid on X and assume that d(s, t) > 4 √ d g. By N min denote the minimal number of points in each grid cell, and define a as Assume that points that are connected in the graph are at most Q grid cells apart from each other (for example, two points in the two grey cells in Figure 3b are 5 cells apart from each other). Then the effective resistance between s and t can be bounded as follows: Step 1. Distributing the flow from s (black dot) to all its neighbors (grey dots). Step 2. We bring back all flow from p to C(s). Also shown in the figure is the hypercube to which the flow will be expanded in Step 3. The general idea of the proof is to construct a flow from s to t with the help of the underlying grid. On a high level, the construction of the proof is not so difficult, but the details are lengthy and a bit tedious. The rest of this section is devoted to it. Construction of the flow -overview. Without loss of generality we assume that there exists a straight line connecting s and t which is along the first dimension of the space. Step 0: We start a unit flow in vertex s. Step 1: We make a step to all neighbors Neigh(s) of s and distribute the flow uniformly over all edges. That is, we traverse d s edges and send flow 1/d s over each edge (see Figure 3a). Step 2: Some of the flow now sits inside C(s), but some of it might sit outside of C(s). In this step, we bring back all flow to C(s) in order to control it later on (see Figure 3b). Step 3: We now distribute the flow from C(s) to a larger region, namely to a hypercube H(s) of side length h that is perpendicular to the linear path from s to t and centered at C(s) (see the hypercubes in Figure 3c). This can be achieved in several substeps that will be defined below. Step 4: We now traverse from H(s) to an analogous hypercube H(t) located at t using parallel paths, see Figure 3c. Step 5: From the hypercube H(t) we send the flow to the neighborhood Neigh(t) (this is the "reverse" of steps 2 and 3). Step 6: From Neigh(t) we finally send the flow to the destination t ("reverse" of step 1). Details of the flow construction and computation of the resistance beween s and t in the general case d > 3. We now describe the individual steps and their contribution to the bound on the resistance. We start with the general case d > 3. We will discuss the special cases d = 2 and d = 3 below. In the computations below, by the "contribution of a step" we mean the part of the sum in Theorem 17 that goes over the edges considered in the current step. Step 1 We start with a unit flow at s that we send over all d s adjacent edges. This leads to flow 1/d s over d s edges. According to the formula in Theorem 17 this contributes to the overall resistance R st . Step 2: After Step 1, the flow sits on all neighbors of s, and these neighbors are not necessarily all contained in C(s). To proceed we want to re-concentrate all flow in C(s). For each neighbor p of s, we thus carry the flow along a Hamming path of cells from p back to C(s), see Figure 3b for an illustration. To compute an upper bound for Step 2 we exploit that each neighbor p of s has to traverse at most Q cells to reach C(s) (recall the definition of Q from the proposition). Let us fix p. After Step 1, we have flow of size 1/d s in p. We now move this flow from p to all points in the neighboring cell C(2) (cf. Figure 3b). For this we can use at least N min edges. Thus we send flow of size 1/d s over N min edges, that is each edge receives flow 1/(d s N min ). Summing the flow from C(p) to C(2), for all points p, gives Then we transport the flow from C(2) along to C(s). Between each two cells on the way we can use N 2 min edges. Note, however, that we need to take into account that some of these edges might be used several times (for different points p). In the worst case, C(2) is the same for all points p, in which case we send the whole unit flow over these edges. This amounts to flow of size 1/(N 2 min ) over (Q -1)N 2 min edges, that is a contribution of Altogether we obtain Step 3: At the beginning of this step, the complete unit flow resides in the cube C(s). We now want to distribute this flow to a "hypercube" of three dimensions (no matter what d is, as long as d > 3) that is perpendicular to the line that connects s and t (see Figure 3c, where the case of d = 3 and a 2-dimensional "hypercube" are shown). To distribute the flow to this cube we divide it into layers (see Figure 4a). Layer 0 consists of the cell C(s) itself, the first layer consists of all cells adjacent to C(s), and so on. Each side of Layer i consists of cells. For the 3-dimensional cube, the number z i of grid cells in Layer i, i ≥ 1, is given as interior cells of the faces cells along the edges (excluding corners) + 8 All in all we consider layers, so that the final layer has diameter just a bit smaller than the bottleneck h. We now distribute the flow stepwise through all layers, starting with unit flow in Layer 0. To send the flow from Layer i -1 to Layer i we use two phases, see Figure 4 for details. In the "expansion phase" 3a(i) we transmit the flow from Layer i -1 to all adjacent cells in Layer i. In the "redistribution phase" 3b(i) we then redistribute the flow in Layer i to achieve that it is uniformly distributed in Layer i. In all phases, the aim is to use as many edges as possible. Expansion phase 3a(i). We can lower bound the number of edges between Layer i -1 and Layer i by z i-1 N 2 min : each of the z i-1 cells in Layer i -1 is adjacent to at least one of the cells in Layer i, and each cell contains at least N min points. Consequently, we can upper bound the contribution of the edges in the expansion phase 3a(i) to the resistance by Redistribution phase 3b(i). We make a crude upper bound for the redistribution phase. In this phase we have to move some part of the flow from each cell to its neighboring cells. For simplicity we bound this by assuming that for each cell, we had to move all its flow to neighboring cells. By a similar argument as for Step 3a(i), the contribution of the redistribution step can be bounded by All of Step 3. All in all we have a layers. Thus the overall contribution of Step 3 to the resistance can be bounded by To see the last inequality, note that the sum a-1 i=1 1/i 2 is a partial sum of the over-harmonic series that converges to a constant smaller than 2. Step 4: Now we transfer all flow in "parallel cell paths" from H(s) to H(t). We have (2a + 1) 3 parallel rows of cells going from H(s) to H(t), each of them contains d(s, t)/g cells. Thus all in all we traverse (2a + 1) 3 N 2 min d(s, t)/g edges, and each edge carries flow 1/((2a + 1) 3 N 2 min ). Thus step 4 contributes Step 5 is completely analogous to steps 2 and 3, with the analogous contribution r 5 = 1 dtNmin + r 3 . Step 6 is completely analogous to step 1 with overall contribution of r 6 = 1/d t . Summing up the general case d > 3. All these contributions leads to the following overall bound on the resistance in case d > 3: with a and Q as defined in Proposition 19. This is the result stated in the proposition for case d > 3. Note that as spelled out above, the proof works whenever the dimension of the space satisfies d > 3. In particular, note that even if d is large, we only use a 3-dimensional "hypercube" in Step 3. It is sufficient to give the rate we need, and carrying out the construction for higher-dimensional hypercube (in particular Step 3b) is a pain that we wanted to avoid. The special case d = 3. In this case, everything works very similar to above, except that we we only use a 2-dimensional "hypercube" (this is what we always show in the figures). The only place in the proof where this really makes a difference is in Step 3. The number z i of grid cells in Layer i is given as z i = 8i. Consequently, instead of obtaining an over-harmonic sum in r 3 we obtain a harmonic sum. Using the well-known fact that a i=1 1/i ≤ log(a) + 1 we obtain In Step 4 we just have to replace the terms (2a + 1) 3 by (2a + 1) 2 . This leads to the result in Proposition 19. The special case d = 2. Here our "hypercube" only consists of a "pillar" of 2a + 1 cells. The fundamental difference to higher dimensions is that in Step 3, the flow does not have so much "space" to be distributed. Essentially, we have to distribute all unit flow through a "pillar", which results in contributions This concludes the proof of Proposition 19. Let us make a couple of technical remarks about this proof. For the ease of presentation we simplified the proof in a couple of respects. Strictly speaking, we do not need to distribute the whole unit flow to the outmost Layer a. The reason is that in each layer, a fraction of the flow already "branches off" in direction of t. We simply ignore this leaving flow when bounding the flow in Step 3, our construction leads to an upper bound. It is not difficult to take the outbound flow into account, but it does not change the order of magnitude of the final result. So for the ease of presentation we drop this additional complication and stick to our rough upper bound. When we consider Steps 2 and 3 together, it turns out that we might have introduced some loops in the flow. To construct a proper flow, we can simply remove these loops. This would then just reduce the contribution of Steps 2 and 3, so that our current estimate is an overestimation of the whole resistance. The proof as it is spelled out above considers the case where s and t are connected by a straight line. It can be generalized to the case where they are connected by a piecewise linear path. This does not change the result by more than constants, but adds some technicality at the corners of the paths. The construction of the flow only works if the bottleneck of X is not smaller than the diameter of one grid cell, if s and t are at least a couple of grid cells apart from each other, and if s and t are not too close to the boundary of X . We took care of these conditions in Part 3 of the definition of a valid grid. First of all, note that by Rayleigh's principle (cf. Corollary 7 in Section IX.2 of Bollobás, 1998) the effective resistance between vertices cannot decrease if we delete edges from the graph. Given a sample from the underlying density p, a random geometric graph based on this sample, and some valid region X , we first delete all points that are not in X . Then we consider the remaining geometric graph. The effective resistances on this graph are upper bounds on the resistances of the original graph. Then we conclude the proofs with the following arguments: Proof of Theorem 3. The lower bound on the deviation follows immediately from Proposition 16. The upper bound is a consequence of Proposition 19 and well known properties of random geometric graphs (summarized in the appendix). In particular, note that we can choose the grid width g := ε/(2 √ d -1) to obtain a valid grid. The quantity N min can be bounded as stated in Proposition 29 and is of order nε d , the degrees behave as described in Proposition 30 and are also of order nε d (we use δ = 1/2 in these results for simplicity). The quantity a in Proposition 19 is of the order 1/ε, and Q can be bounded by Q = ε/g and by the choice of g is indeed a constant. Plugging all these results together leads to the final statement of the theorem. Proof of Theorem 4. This proof is analogous to the ε-graph. As grid width g we choose g = R k,min /(2 √ d -1) where R k,min is the minimal k-nearest neighbor distance (note that this works for both the symmetric and the mutual kNN-graph). Exploiting Propositions 29 and 31 we can see that R k,min and R k,max are of order (k/n) 1/d , the degrees and N min are of order k, a is of the order (n/k) 1/d and Q a constant. Now the statements of the theorem follow from Proposition 19. 6 Proofs for the spectral approach 6.1 Proof of the key propositions 1 and 5 In this section we prove the general formulas to compute and approximate the hitting times. Proof of Proposition 1: For the hitting time formula, let u 1 , . . . , u n be an orthonormal set of eigenvectors of L sym corresponding to the eigenvalues λ 1 , . . . , λ n . Let u ij denote the j-th entry of u i . According to Lovász (1993) the hitting time is given by A straightforward calculation using the spectral representation of L sym yields Exploiting that (A -P 1 ) commutes with all P r gives Applying the Cauchy-Schwarz inequality and the fact n r=2 1 1-λr P r 2 = 1/(1 -λ 2 ) leads to the desired statement. Proof of Proposition 5. This proposition now follows easily from the Lemma above. Observe that Exploiting that P 1 (u i -u j ) = 0 we get for the hitting time For the commute time, we note that We would like to point out that the key to achieving this bound is not to give in to the temptation to manipulate Eq. ( 12) directly, but to bound Eq. ( 13). The reason is that we can compute terms of the form u i , Au j and related terms explicitly, whereas we do not have any explicit formulas for the eigenvalues and eigenvectors in (12). As we have seen above, a key ingredient in the approximation result for hitting times and commute distances is the spectral gap. In this section we show how the spectral gap can be lower bounded for random geometric graphs. We first consider the case of a fixed geometric graph. From this general result we then derive the results for the special cases of the ε-graph and the kNN-graphs. All graphs considered in this section are unweighted and undirected. We follow the strategy in Boyd et al. (2005) where the spectral gap is bounded by means of the Poincaré inequality (see Diaconis and Stroock (1991) for a general introduction to this technique; see Cooper and Frieze (2009) for a related approach in simpler settings). The outline of this technique is as follows: for Figure 5: Canonical path between a and b. We first consider a "Hamming path of cells" between a and b. In all intermediate cells, we randomly pick a point. each pair (X, Y ) of vertices in the graph we need to select a path γ XY in the graph that connects these two vertices. In our case, this selection is made in a random manner. Then we need to consider all edges in the graph and investigate how many of the paths γ XY , on average, traverse this edge. We need to control the maximum of this "load" over all edges. The higher this load is, the more pronounced is the bottleneck in the graph, and the smaller the spectral gap is. Formally, the spectral gap is related to the maximum average load b as follows. Proposition 21 (Spectral gap, Diaconis and Stroock, 1991) Consider a finite, connected, undirected, unweighted graph that is not bipartite. For each pair of vertices X =Y let P XY be a probability distribution over all paths that connect X and Y and have uneven length. Let (γ XY ) X,Y be a family of paths independently drawn from the respective P XY . Define b := max {e edge} E|{γ XY e ∈ γ XY }|. Denote by |γ max | the maximum path length (where the length of the path is the number of edges in the path). Then the spectral gap in the graph is bounded as follows: For deterministic sets Γ, this proposition has been derived as Corollary 1 and 2 in Diaconis and Stroock (1991). The adaptation for random selection of paths is straightforward, see Boyd et al. (2005). The key to tight bounds based on Proposition 21 is a clever choice of the paths. We need to make sure that we distribute the paths as "uniformly" as possible over the whole graph. This is relatively easy to achieve in the special situation where X is a torus with uniform distribution (as studied in Boyd et al., 2005, Cooper andFrieze, 2009) because of symmetry arguments and the absence of boundary effects. However, in our setting with general X and p we have to invest quite some work. We first treat the special case of a fixed geometric graph with vertices in the unit cube [0, 1] d in R d . Consider a grid on the cube with grid width g. For now we assume that the grid cells are so small that points in neighboring cells are always connected in the geometric graph, and so large that each cell contains a minimal number of data points. We will specify the exact value of g later. In the following, cells of the grid are identified with their center points. Construction of the paths. Assume we want to construct a path between two vertices a and b that correspond to the points a = (a 1 , . . . . Now we transform this deterministic "cell path" to a random path on the graph. In the special cases where a and b are in the same cell or in neighboring cells, we directly connect a and b by an edge. In the general case, we select one data point uniformly at random in each of the interior cells on the cell path. Then we connect the selected points to form a path. Note that we can always force the paths to have uneven lengths by adding one more point somewhere in between. Proposition 22 (Path construction is valid) Assume that (1) Each cell of the grid contains at least one data point. (2) Data points in the same and in neighboring cells are always connected in the graph. Then the graph is connected, and the paths constructed above are paths in the graph. Proof. Obvious, by construction of the paths. In order to apply Proposition 21 we now need to compute the maximal average load of all paths. Proposition 23 (Maximum average load for fixed graph on cube) Consider a geometric graph on [0, 1] d and the grid of width g on [0, 1] d . Denote by N min and N max the minimal and maximal number of points per grid cell. Construct a random set of paths as described above. 1. Let C be any fixed cell in the grid. Then there exist at most d/g d+1 pairs of cells (A, B) such that cell paths starting in cell A and ending in cell B pass through C. That is, there exists some l ∈ {1, . . . , d} such that (I) b i = c i for all i = 1, . . . , l -1 and (II) a i = c i for all i = l + 1, . . . , d. For the given grid size g there are 1/g different cell centers per dimension. For fixed l there thus exist 1/g d-l+1 cell centers that satisfy (I) and 1/g l cell centers that satisfy (II). So all in all there are 1/g d+1 pairs of cells A and B such that both (I) and (II) are satisfied for a fixed value of l. Adding up the possibilities for all choices of l ∈ {1, . . . , d} leads to the factor d. Part 2. Fix an edge e in the graph and consider its two adjacent vertices v 1 and v 2 . If v 1 and v 2 are in two different cells that are not neighbors to each other, then by construction none of the paths traverses the edge. If they are in the same cell, by construction at most one of the paths can traverse this edge, namely the one directly connecting the two points. The interesting case is the one where v 1 and v 2 lie in two neighboring grid cells C and C. If both cells are "interior" cells of the path, then by construction each edge connecting the two cells has equal probability of being selected. As there are at least N min points in each cell, there are at least N 2 min different edges between these cells. Thus each of the edges between the cells is selected with probability at most 1/N 2 min . We know by Part 1 that there are at most d/g d+1 pairs of start/end cells. As each cell contains at most N max points, this leads to N 2 max d/g d+1 different paths passing through C. This is also an upper bound on the number of paths passing through both C and C. Thus, each edge is selected by at most dN 2 max /(g d+1 N 2 min ) paths. If at least one of the cells is the start cell of the path, then the corresponding vertex, say v 1 , is the start point of the path. If v 2 is an intermediate point, then it is selected with probability at most 1/N min (the case where v 2 is an end point has already been treated at the beginning). Similarly to the last case, there are at most N max d/g d+1 paths that start in v 1 and pass through C. This leads to an average load of dN max /(g d+1 N min ) on edge e. The same holds with the roles of v 1 and v 2 exchanged, leading to a factor 2. The overall average load is now the sum of the average loads in the different cases. 6.2.2 Fixed geometric graph on a domain X that is homeomorphic to a cube Now assume that X ⊂ R d is a compact subset that is homeomorphic to the cube [0, 1] d in the following sense: we assume that there exists a homeomorphism h : X → [0, 1] d and constants 0 < L min < L max < ∞ such that for all x, y ∈ X we have The general idea is now as follows. Assume we are given a geometric graph on X 1 , . . . , X n ∈ X . In order to construct the paths we first map the points in the cube using h. Then we construct the paths on h(X 1 ), . . . , h(X n ) ∈ [0, 1] d as in the last section. Finally, we map the path back to X . Proposition 24 (Maximum average load for fixed graph on general domain) Let G be a geometric graph based on X 1 , . . . , X n ∈ X . Assume that there exists some g > 0 such that points of distance smaller than g are always connected in the graph. Consider a mapping h : X → [0, 1] d as in Equation ( 17) and a grid of width g on [0, 1] d . Let (C i ) i be the cells of the g-grid on [0, 1] d , denote their centers by c i . Let B i and B i be balls in X with radius r = g/(2L max ) and R = √ d g/L min centered at h -1 (c i ). 2. Denote by Ñmin the minimal number of points in B i and Ñmax the maximal number of points in B i . Construct paths between the points h(X i ) ∈ [0, 1] d as described in the previous subsection. If Ñmin ≥ 1 and g ≤ L min g/ √ d + 3, then these paths are valid. 3. In this case, the maximal average load can be upper bounded by Proof. Part 1. Let c i be the center of cell C i and consider the ball B i centered at h -1 (c i ) with radius g/(2L max ). Clearly, h -1 (c i ) is an interior point of h -1 (C i ). Suppose that there exists x ∈ B i ∩ ∂h -1 (C i ). Since h maps the boundary of h -1 (C i ) onto the boundary of C i , we conclude that h(x) ∈ ∂C i and thus h(x) -c i ≥ g/2. By our assumption on the homeomorphism we can estimate Part 2. By the definition of Ñmin it is clear that each cell of the grid contains at least one point. Consider two points X i , X j ∈ X such that h(X i ) and h(X j ) are in neighboring cells of the g-grid. Then h(X i ) -h(X j ) ≤ g √ d + 3. By the properties of h, Thus, by the definition of g the points X i and X j are connected in G. Part 3. Follows directly from Proposition 23. Now we are going to apply Proposition 24 to ε-graphs. We will use the general results on ε-graphs summarized in the appendix. Proposition 25 (Maximal average load for ε-graph) Assume that X is homeomorphic to the cube with a mapping h as described in Equation (17). Then there exist constants c 1 , c 2 , c 3 > 0 such that with probability at least 1 -c 1 exp(-c 2 nε d )/ε d , the maximum average load is upper bounded by c 3 /ε d+1 . If nε d / log n → ∞, then this probability tends to 1 as n → ∞. Proof. The proof is based on Proposition 24. By construction we know that points with distance at most g = ε are always connected in the ε-graph. By Part 2 of Proposition 24, to ensure that points in neighboring grid cells are always connected in the graph we thus need to choose the grid width g = ε • L min / √ d + 3. The radius r defined in Proposition 24 is then given as The probability mass of the balls B i is thus bounded by We have grid cells and thus the same number of balls B i . We can now apply Proposition 29 (with δ := 1/2) to deduce the bound for the quantity Ñmin used in Proposition 24: Analogously Plugging these values into Proposition 24 leads to the result. We are now ready to prove Theorem 6 by applying Proposition 21. With probability at least 1 -c 1 n exp(-c 2 nε d ), both the minimal and maximal degrees in the graph are of the order Θ(nε d ) (cf. Proposition 30), and the volume of G is of order Θ(n 2 ε d ). To compute the maximal number |γ max | of edges in each of the paths constructed above, observe that each path can traverse at most d•1/g = (d √ d + 3/L min )•(1/ε) cubes, and a path contains just one edge per cube. Thus |γ max | is of the order Θ(1/ε). In Proposition 25 we have seen that with probability at least c 4 exp(-c 5 nε d )/ε d the maximum average load b is of the order Ω(1/ε d+1 ). Plugging all these quantities in Proposition 21 leads to the result. As in the case of the flow proofs, the techniques in the case of the kNN-graphs are identical to the ones for the ε-graph, we just have to replace the deterministic radius ε by the minimal kNN-radius. As before we exploit that if two sample points have distance less than R k,min from each other, then they are always connected both in the symmetric and mutual kNN-graph. Proposition 26 (Maximal average load in the kNN-graph) Under the general assumptions, with probability at least 1 -c 1 • n • exp(-c 2 k) the maximal average load in both the symmetric and mutual kNN-graph is bounded from above by c 3 (n/k) 1+1/d . If k/ log n → ∞, then this probability converges to 1. Proof. This proof is completely parallel to the one of Proposition 25, the role of ε is now taken over by R k,min . Finally, the proof of Theorem 7 goes as follows. With probabilities at least 1 -n exp(-c 1 k) the following statements hold: the minimal and maximal degree are of order Θ(k), thus the number of edges in the graph is of order Θ(nk). Analogously to the proof for the ε-graph, the maximal path length |γ max | is of the order 1/R k,min = Θ((k/n) 1/d ). The maximal average load is of the order O((n/k) d+1/d ). Plugging all these quantities in Proposition 21 leads to the result. Now we collected all ingredients to finally present the following proofs. This is a direct consequence of the results on the minimal degree (Proposition 30) and the spectral gap (Theorem 6). Plugging these results into Proposition 5 leads to the first result. The last statement in the theorem follows by a standard density estimation argument, as the degree of a vertex in the ε-graph is a consistent density estimator (see Proposition 30). Follows similarly as Theorem 8 by applying Proposition 5. The results on the minimal degree and the spectral gap can be found in Proposition 31 and Theorem 7. The last statement follows from the convergence of the degrees, see Proposition 31. For weighted graphs, we use the following results from the literature. Proposition 27 (Spectral gap in weighted graphs) 1. For any row-stochastic matrix P , 2. Consider a weighted graph G with edge weights 0 < w min ≤ w ij ≤ w max and denote its second eigenvalue by λ 2,weighted . Consider the corresponding unweighted graph where all edge weights are replaced by 1, and denote its second eigenvalue by λ 2,unweighted . Then we have 1. This bound was obtained by Zenger (1972), see also Section 2.5 of Seneta (2006) for a discussion. Note that the second inequality is far from being tight. But in our application, both bounds lead to similar results. 2. This statement follows directly from the well-known representation of the second eigenvalue µ 2 of the normalized graph Laplacian L sym (see Sec. 1.2 in Chung, 1997), Note that the eigenvalue µ 2 of the normalized Laplacian and the eigenvalue λ 2 of the random walk matrix P are in relation 1 -λ 2 = µ 2 . We will now show to examples how this proposition can be used. The first application of Proposition 27 is the Proof of Theorem 10, which follows directly from plugging in the first part of Proposition 27 in Theorem 5. The second application of Proposition 27 is the following proof. Proof of Theorem 11. We split . Under the given assumption, the second term on the right hand side converges to 0 a.s. by a standard kernel density estimation argument. The main work is the first term on the right hand side. We treat upper and lower bounds of R ij -1/d i -1/d j separately. To get a lower bound, recall that by Proposition 16 we have ) and w ij is the weight of the edge between i and j. It is straightforward to see that under the given conditions, To treat the upper bound, we define the ε-truncated Gauss graph G ε as the graph with edge weights , where R ε denotes the resistance of the ε-truncated Gauss graph. Obviously, . To bound term ( * * ) we show that the degrees in the truncated graph converge to the ones in the non-truncated graph. To see this, note that E we obtain the convergence of the expectations: under the assumptions on n and h from the theorem, E Now, a probabilistic bound for term ( * * ) can be obtained by standard concentration arguments. We now bound term ( * ). In the following we implicitly define ε via h 2 = ε 2 / log(nε d+2 ). Note that for the given choice of ε, the truncated Gaussian graph "converges" to the non-truncated graph, as we truncate less and less weight. Denote by λ ε,weighted the eigenvalues of the ε-truncated Gauss graph, and by w ε min , w ε max its minimal and maximal edge weights. Also consider the graph G that is the unweighted version of the ε-truncated Gauss graph G ε . Note that G coincides with the standard ε-graph. We denote its eigenvalues by λ ε,unweighted . By applying Proposition 5 and Corollary 8 we get where the first inequality holds with probability at least 1 -c 1 n exp(-c 2 nh d ) -c 3 exp(-c 4 nh d )/h d . By ( * * ) we already know that the last factor of Term (20) converges to a constant: For the other factors of Term (20) we use the following quantities: Plugging these quantities in (20) we obtain the convergence of ( * ). Proof. It is well known that under the given assumptions, the following properties hold with high probability: the graph is connected and the minimal and average degrees are of the order np, in particular np/d i converges to 1 in probability. The volume of the graph is of the order n 2 p. To use Theorem 12, observe that the matrix A = pJ where J is the (n × n)-matrix of all ones. The expected degree of all vertices is np. Hence, D -1/2 AD -1/2 = 1 np • A. This matrix has rank 1, its non-zero eigenvalue is 1 with the constant one vector as corresponding eigenvector. Hence the expected spectral gap in this model is 1. It is easy to see that as soon as p/ log(n) → ∞, the deviations in Theorem 12 converge to 0. Plugging all this into our Proposition 5 shows that with high probability, Proof. The expected degree of each vertex is n(p within + p between )/2, the expected volume of the graph is n 2 (p within + p between )/2. The matrix A has the form pJ qJ qJ pJ where J is the (n/2 × n/2)matrix of all ones. The expected degree of all vertices is n(p + q)/2. Hence, D -1/2 AD -1/2 = 2 n(p+q) • A. This matrix has rank 2, its largest eigenvalue is 1 (with eigenvector the constant 1 vector), the other eigenvalue is (p -q)/(p + q) with eigenvector (1, ..., 1, -1, ..., -1). Hence, the spectral gap in this model is 2q/(p + q). Under the assumption that p = ω(log(n)/n), the deviations in Theorem 12 converge to 0. Proof. we use the result from Theorem 4 in Chung and Radcliffe (2011) which states that under the assumption that the minimal expected degree d min satisfies d min / log(n) → ∞, then with probability at least 1 -1/n the spectral gap is bounded by a term of the order O(log(2n)/d min ). Plugging this in Proposition 5 shows that with high probability, We have presented different strategies to prove that in many large graphs the commute distance can be approximated by 1/d i + 1/d j . Both our approaches tell a similar story. Our result holds as soon as there are "enough disjoint paths" between i and j, compared to the size of the graph, and the minimal degree is "large enough" compared to n. We would like to point out that our results on the degeneracy of the hitting and commute times are not due to pathologies such as a "misconstruction" of the graphs. For example, in the random geometric graph setting the graph Laplacian can be proved to converge to the Laplace-Beltrami operator on the underlying space under similar assumptions as the ones above (Hein et al., 2007). But even though the Laplacian itself converges to a meaningful limit, the resistance distance, which is computed based on point evaluations of the inverse of this Laplacian, does not converge to a useful limit. The limit distance function dist(i, j) = 1/d i +1/d j is completely meaningless as a distance function. It just considers the local density (the degree) at the two vertices, but does not take into account any global property such as the cluster structure of the graph. As the speed of convergence is very fast (for example, of the order 1/n in the case of Gaussian similarity graphs), the use of the raw commute distance should be discouraged even on moderate sized graphs. However, there might be ways how useful information can be extracted from the commute distance, namely in the form of the remainder terms S ij -1/d i -1/d j . Exploring this idea in depth is a project for future research. There are two important classes of graphs that are not covered in our approach. In power law graphs as well as in grid-like graphs, the minimal degree is constant, thus our results do not lead to tight enough bounds. The resistance distances on grid-like graphs has been studied in some particular cases. For example, Cserti (2000) and Wu (2004) prove explicit formulas for the resistance on regular one-and two-dimensional grids, and Benjamini and Rossignol (2008) characterize the variance of the resistance on random Bernoulli grids. To the best of our knowledge, general results about the convergence of the resistance distance on grid-like graphs do not exist. For the symmetric graph, observe that the maximal degree in the graph is bounded by the maximal number of points in the balls of radius R k,max centered at the data points. We know that with high probability, a ball of radius R k,max contains of the order Θ(nR d k,max ) points. Using Part 1 we know that with high probability, R k,max is of the order (k/n) 1/d . Thus the maximal degree in the symmetric kNN-graph is of the order Θ(k), with high probability. In the mutual graph, observe that the minimal degree in the graph is bounded by the minimal number of points in the balls of radius R k,min centered at the data points. Then the statement follows analogously to the last one. Part 3, proof sketch. Consider a fixed point x in the interior of X . We know that both in the symmetric and mutual kNN-graph, two points cannot be connected if their distance is larger than R k,max . As we know that R k,max is of the order (k/n) 1/d , under the growth conditions on n and k this radius becomes arbitrarily small. Thus, because of the continuity of the density, if n is large enough we can assume that the density in the ball B(x, R k,max ) of radius R k,max around x is approximately constant. Thus, all points y ∈ B(x, R k,max ) have approximately the same expected k-nearest neighbor radius R := (k/(n • p(x)η d )) 1/d . Moreover, by concentration arguments it is easy to see that the actual kNN-radii only deviate by a factor 1 ± δ from their expected values. Then, with high probability, all points inside of B(x, R(1 -δ)) are among the k nearest neighbors of x, and all k nearest neighbors of x are inside B(x, R(1 + δ)). On the other hand, with high probability x is among the k nearest neighbors of all points y ∈ B(x, R(1 -δ)), and not among the k nearest neighbors of any point outside of B(x, R(1 + δ)). Hence, in the mutual kNN-graph, with high probability x is connected exactly to all points y ∈ B(x, R(1 -δ)). In the symmetric kNN-graph, x might additionally be connected to the points in B(x, R(1 + δ)) \ B(x, R(1 -δ)). By construction, with high probability the number of sample points in these balls is (1 + δ)k and (1 -δ)k. Driving δ to 0 leads to the result. The technique we use to prove this theorem is very general. Using the Rayleigh principle, we reduce the case of the fully connected Gaussian graph to a truncated graph where edges beyond a certain length are removed. Bounds for this truncated graph, in turn, can be reduced to bounds of the unweighted ε-graph. With this technique it is possible to treat very general classes of graphs.