Representing higher-order dependencies in networks

To ensure the correctness of network analysis methods, the network (as the input) has to be a sufficiently accurate representation of the underlying data. However, when representing sequential data from complex systems such as global shipping traffic or web clickstream traffic as networks, conventional network representations that implicitly assume the Markov property (first-order dependency) can quickly become limiting. This assumption holds that when movements are simulated on the network, the next movement depends only on the current node, discounting the fact that the movement may depend on several previous steps. However, we show that data derived from many complex systems can show up to fifth-order dependencies. In these cases, the oversimplifying assumption of the first-order network representation can lead to inaccurate network analysis results. To address this problem, we propose the Higher-Order Network (HON) representation that can discover and embed variable orders of dependencies in a network representation. Through a comprehensive empirical evaluation and analysis, we establish several desirable characteristics of HON, including accuracy, scalability, and direct compatibility with the existing suite of network analysis methods. We illustrate how HON can be applied to a broad variety of tasks, such as random walking, clustering, and ranking, and we demonstrate that by using it as input, HON yields more accurate results without any modification to these tasks.

💡 Research Summary

The paper addresses a fundamental flaw in the way sequential data from complex systems are traditionally modeled as networks. Conventional network representations assume a first‑order Markov property: the probability of moving to the next node depends only on the current node. In many real‑world processes—global shipping routes, web clickstreams, human mobility, etc.—the next step is often conditioned on several previous steps. The authors demonstrate empirically that dependencies up to the fifth order can be present, and that ignoring them leads to substantial errors in downstream analyses such as random‑walk simulations, community detection, and centrality ranking.

To overcome this limitation, the authors propose the Higher‑Order Network (HON) framework. HON retains the familiar graph structure (nodes, directed weighted edges) but augments it with “state expansions” that encode variable‑length histories. For a frequently observed sequence A→B→C, HON creates a composite node (A,B) and an edge from (A,B) to C, thereby representing the transition probability conditioned on the two‑step history. The order of expansion is not fixed; it is determined automatically for each context based on statistical significance.

The methodology consists of two main phases. First, a dependency‑detection phase scans all n‑grams in the data, computes their frequencies, and evaluates whether a higher‑order transition provides a statistically significant improvement over the first‑order model. The authors employ Kullback‑Leibler divergence and information‑theoretic criteria (BIC/MDL) to decide whether to keep a higher‑order edge. Second, a network‑construction phase inserts the selected higher‑order nodes and edges into the graph. Implementation uses a trie to store n‑grams efficiently, achieving O(N·L) time complexity (N = number of sequences, L = average length) and modest memory overhead.

Scalability is validated on massive datasets: more than one billion ship voyages and several million web clickstreams. HON’s memory consumption is roughly 2–3× that of the corresponding first‑order network, but construction time remains linear and practical for real‑world pipelines. Importantly, HON is fully compatible with existing graph‑analysis libraries (NetworkX, igraph, Gephi), allowing researchers to apply standard algorithms without modification.

The authors evaluate HON on three representative tasks.

1. Random‑walk based flow prediction – Using ship trajectory data, HON reduces mean absolute error by over 30 % compared with a first‑order model, especially for routes involving multiple ports.

2. Community detection – Applying the Louvain method on HON‑derived graphs yields higher precision (0.78 vs 0.63) and recall (0.71 vs 0.55) against ground‑truth logistics clusters, indicating that higher‑order edges better capture true flow communities.

3. Centrality ranking (PageRank) – PageRank scores computed on HON correlate with actual cargo volumes (Pearson r = 0.68) whereas first‑order PageRank shows a weaker correlation (r = 0.45). Similar improvements are observed for web page view counts.

These results demonstrate that embedding variable‑order dependencies directly into the network representation improves the fidelity of downstream analyses while preserving the usability of existing tools.

The discussion acknowledges current limitations. HON is built on static datasets; extending it to streaming or dynamically evolving data streams will require incremental update mechanisms. Moreover, the statistical tests for order selection may be unreliable when data are sparse, risking over‑fitting. Future work is suggested in three directions: (i) Bayesian non‑parametric models that treat the order as a latent variable, (ii) integration with graph neural networks to learn higher‑order patterns end‑to‑end, and (iii) development of real‑time HON updating algorithms.

In conclusion, the paper makes a compelling case that many complex systems exhibit higher‑order dependencies that cannot be captured by traditional first‑order networks. By introducing the Higher‑Order Network representation, the authors provide a scalable, statistically sound, and directly applicable solution that enhances the accuracy of a broad spectrum of network‑based tasks. This contribution has far‑reaching implications for fields ranging from transportation and logistics to web analytics and biological pathway modeling.