Entropy Rate of Diffusion Processes on Complex Networks

Entropy is a key concept in statistical thermodynamics [1], in the theory of dynamical systems [2], and in information theory [3]. In the realm of complex networks [4,5], the entropy has been used as a measure to characterize properties of the topology, such as the degree distribution of a graph [6], or the shortest paths between couples of nodes (with the main interest in quantifying the information associated with locating specific addresses [7], or to send signals in the network [8]. Alternatively, various authors have studied the entropy associated with ensembles of graphs, and provided, via the application of the maximum entropy principle, the best prediction of network properties subject to the constraints imposed by a given set of observations [9,10,11]. An approach of this type plays the same role in networks as is played by the Boltzmann distribution in statistical thermodynamics [1].

The main theoretical and empirical interest in the study of complex networks is in understanding the relations between structure and function. Besides, many of the interaction dynamics that takes place in social, biological and technological systems can be analyzed in terms of diffusion processes on top of complex networks, e.g. data search and routing, information and disease spreading [4,5]. It is therefore of outmost importance to relate the properties of a diffusion process with the structure of the underlying network.

In this Letter, we show how to associate an entropy rate to a diffusion process on a graph. In particular, we consider processes such as biased random walks on the graph that can be represented as ergodic Markov chains. In this context, the entropy rate is a quantity more similar to the Kolmogorov-ÂSinai entropy rate of a dynamical system [12,13], than to the entropy of a statistical ensemble [1,4]. Differently from the network entropies previously defined, the entropy rate of diffusion processes depends both on the dynamical process (the kind of bias in the random walker) and on the graph topology. We provide the analytical expression that describes the entropy rate in scale-free networks as a function of the bias in the walk, and of the degree distribution and correlations. We show how the values of the entropy rate can provide useful information to characterize diffusion processes in real-world networks. In particular, a maximum value of entropy is found for different types of the bias in the diffusion processes, depending on the network structure.

Let us consider a connected undirected graph with N nodes (labelled as 1, 2, …, N ) and K links, described by the adjacency matrix A = {a ij } We limit our discussion to diffusion processes on the graph that can be represented as Markov chains [3]. In particular, we consider the case of biased random walks in which, at each time step, the walker at node i chooses one of the first neighbors of i, let say j, with a probability proportional to the power α (α ∈ R) of the degree k j . Such biased random walk corresponds to a time-invariant (the rule does not change in time) Markov chain with a transition probability matrix Π, with elements:

Notice that Π depends on either the graph topology and the kind of stochastic process we are considering. The exponent α allows to tune the dependence of the diffusion process on the nodes’ degree. When α = 0 we are introducing in the random movement of the particle a bias towards high-(α > 0) or low-degree (when α < 0) neighbors. On the other hand, when α = 0 the standard (unbiased) random walk is recovered. Since the walker must move from a node to somewhere, we have j π ji = 1, thus Π is a stochastic matrix. If w i (t) is the probability that the random walker is at node i at time t (with N i=1 w i (t) = 1 ∀t), then the probability w j (t + 1) of its being at j one step later is: w j (t+1) = i π ji w i (t). Writing the probabilities w i (t) as a N -dimensional column vector w(t) = ( w 1 (t), w 2 (t) . . . w N (t) ) ⊤ , the rule of the walk can be expressed in matricial form as: w(t + 1) = Πw(t). In the case of an undirected and connected network, the Perron-Frobenius theorem [14] assures that the dynamics described by Eq. ( 1) is an ergodic Markov chain [3]. This means that the Markov chain has a unique stationary distribution w * , such that lim t→∞ Π t w(0) = w * for any initial distribution w(0). In other words, any initial distribution of the random walker over the nodes of the graph will converge, under the dynamics of Eq. ( 1), to the same distribution w * .

The dynamical properties of the above diffusion processes over the graph can be accounted by evaluating the entropy rate of the associated Markov chain that, in the case of an ergodic Markov chain, is given by [3]:

The value of h measures how the entropy of the biased random walk grows with the number of hops. This means that we can practically represent the typical sequences of length n generated by the diffusion process by using approximately n • h

…(Full text truncated)…

This content is AI-processed based on ArXiv data.