Dynamic Node Embeddings from Edge Streams

Networks evolve continuously over time with the addition, deletion, and changing of links and nodes. Such temporal networks (or edge streams) consist of a sequence of timestamped edges and are seemingly ubiquitous. Despite the importance of accurately modeling the temporal information, most embedding methods ignore it entirely or approximate the temporal network using a sequence of static snapshot graphs. In this work, we propose using the notion of temporal walks for learning dynamic embeddings from temporal networks. Temporal walks capture the temporally valid interactions (e.g., flow of information, spread of disease) in the dynamic network in a lossless fashion. Based on the notion of temporal walks, we describe a general class of embeddings called continuous-time dynamic network embeddings (CTDNEs) that completely avoid the issues and problems that arise when approximating the temporal network as a sequence of static snapshot graphs. Unlike previous work, CTDNEs learn dynamic node embeddings directly from the temporal network at the finest temporal granularity and thus use only temporally valid information. As such CTDNEs naturally support online learning of the node embeddings in a streaming real-time fashion. Finally, the experiments demonstrate the effectiveness of this class of embedding methods that leverage temporal walks as it achieves an average gain in AUC of 11.9% across all methods and graphs.

💡 Research Summary

This paper introduces a novel framework for learning node embeddings from temporal networks, which are dynamic structures that evolve continuously over time through the addition, deletion, and modification of timestamped edges (edge streams). The authors identify a fundamental flaw in prior network embedding approaches: most methods either completely ignore temporal information or approximate the dynamic network as a discrete-time dynamic network (DTDN), i.e., a sequence of static snapshot graphs aggregated over fixed time windows. These approximations lead to significant information loss, the creation of temporally invalid sequences (e.g., walks that travel backward in time), and sensitivity to the often-arbitrary choice of aggregation granularity.

To address these issues, the paper proposes the concept of temporal walks. A temporal walk is a sequence of edges where each subsequent edge occurs at a time equal to or later than the previous edge. This simple but powerful definition ensures that the walk respects the natural flow of time, capturing only sequences that are physically possible for the diffusion of information, influence, or disease within the network. This stands in stark contrast to traditional random walks on static graphs, which can generate paths that are impossible in a temporal context.

Building upon temporal walks, the authors define a new class of methods called Continuous-Time Dynamic Network Embeddings (CTDNEs). CTDNE methods learn node embeddings directly from the raw edge stream at the finest temporal granularity, completely avoiding the DTDN approximation. The core framework involves: 1) sampling temporally valid random walks from the continuous-time network, often with a bias towards more recent and frequent edges; 2) using these temporally coherent sequences of nodes as input to representation learning algorithms like Skip-gram (used in Word2Vec, DeepWalk, Node2Vec). This framework is highly generalizable; it provides a plug-and-play mechanism to convert any existing random walk-based static embedding method into a temporal one by simply replacing its walk sampling module with a temporal walk sampler.

Key advantages of CTDNE include:

• Lossless Temporal Modeling: It uses all temporal information without aggregation loss.
• Temporal Validity: All learned node contexts come from feasible, time-respecting paths.
• Online Learning Support: The method naturally supports streaming scenarios. When a new edge arrives, only walks involving the affected nodes need to be updated, enabling efficient real-time updates of node embeddings without full retraining.
• Generalization: It serves as a foundational framework for temporalizing a wide range of existing graph learning models.

The paper validates the effectiveness of CTDNE through extensive experiments on real-world temporal networks from diverse domains (e.g., academic collaboration, social media). The task is temporal link prediction: predicting future edges given past observations. The results demonstrate that CTDNE consistently and significantly outperforms both static embedding methods and state-of-the-art DTDN-based dynamic embedding methods, achieving an average gain of 11.9% in AUC across all datasets and baselines. This work establishes that precisely modeling the direction and sequence of time is not merely beneficial but critical for learning accurate and meaningful representations in dynamic networks.