How Expressive Are Graph Neural Networks in the Presence of Node Identifiers?

Graph neural networks (GNNs) are a widely used class of machine learning models for graph-structured data, based on local aggregation over neighbors. GNNs have close connections to logic. In particular, their expressive power is linked to that of modal logics and bounded-variable logics with counting. In many practical scenarios, graphs processed by GNNs have node features that act as unique identifiers. In this work, we study how such identifiers affect the expressive power of GNNs. We initiate a study of the key-invariant expressive power of GNNs, inspired by the notion of order-invariant definability in finite model theory: which node queries that depend only on the underlying graph structure can GNNs express on graphs with unique node identifiers? We provide answers for various classes of GNNs with local max- or sum-aggregation.

💡 Research Summary

This paper investigates the expressive power of graph neural networks (GNNs) when each node in a graph is equipped with a unique identifier, or “key”. In many practical settings—such as geometric graphs with coordinates, transformer‑based positional encodings, or randomly generated node features—nodes carry real‑valued attributes that are distinct across the graph. While such identifiers can dramatically increase the discriminative capacity of a GNN, they also break the natural isomorphism‑invariance of the model: the output may depend on the particular choice of identifiers rather than solely on the underlying graph structure.

To address this tension, the authors introduce the notion of key‑invariant GNNs. A key‑invariant GNN operates on a keyed graph but its predictions must be independent of the specific key values, provided the keys are all distinct. In other words, the GNN may read the keys as part of its input, but the induced node query must be invariant under any bijection that preserves the graph’s edges, labels, and the distinctness of keys. This concept mirrors the idea of order‑invariant definability from finite model theory, where formulas may refer to an auxiliary linear order but the truth of the formula must not depend on the particular order chosen.

The study focuses on two families of local aggregation operators: LocalMax, which aggregates neighbor embeddings by coordinate‑wise maximum, and LocalSum, which aggregates by coordinate‑wise sum. For each aggregation family the authors consider four classes of combination (or “readout”) functions:

1. FFN(ReLU) – feed‑forward neural networks with ReLU activations (the standard GNN setting).
2. Continuous – arbitrary continuous real‑valued functions.
3. Semilinear / piecewise‑linear – functions that may include the Heaviside step (i.e., discontinuous but semilinear).
4. Arbitrary – any real‑valued function without restrictions.

The expressive power of key‑invariant GNNs is measured in terms of the node queries they can realize. A node query is an isomorphism‑closed set of pointed graphs; the GNN “expresses” a query if it accepts exactly the pointed graphs whose keyed extensions are accepted. The authors relate these query classes to well‑studied logical formalisms: graded modal logic (GML), its weak variant (WGML), first‑order logic with counting (FO+C), and order‑invariant first‑order logic (OrdInvFO).

Main Results for LocalMax GNNs

• FFN(ReLU) GNNs with LocalMax aggregation capture exactly the weak graded modal logic (WGML). This matches prior results for unkeyed GNNs.
• Continuous combination functions are strictly weaker: they correspond to the “modal” fragment of WGML, i.e., formulas that do not use the ≥2 ⊤ operator.
• Semilinear (including step functions) are strictly stronger than continuous functions; they can express the ≥2 ⊤ operator and therefore a larger fragment of WGML.
• Arbitrary combination functions achieve the full power of order‑invariant first‑order logic (OrdInvFO). Consequently, any query definable in OrdInvFO can be realized by a key‑invariant LocalMax GNN with unrestricted combination functions.
• When the acceptance policy forces the network to output a strictly positive value on yes‑instances and a strictly negative value on no‑instances (the ( > 0 / < 0 ) policy), continuous LocalMax GNNs collapse to the unkeyed setting: they cannot exploit the keys to gain extra expressive power.

Main Results for LocalSum GNNs

• Continuous combination functions already yield maximal expressive power for strongly local queries: every query that is invariant under inverse coverings (a combinatorial analogue of bisimulation) can be expressed. This class coincides with all “strongly local” queries, which include many natural graph properties.
• Arbitrary combination functions extend this to the full order‑invariant FO with counting (OrdInvFO + C). Hence, any query definable in FO+C that does not depend on a particular ordering of the keys can be captured.
• Under the ( ≥ 1 / ≤ 0 ) acceptance policy, key‑invariant LocalSum GNNs collapse to their unkeyed counterparts, mirroring the LocalMax case. However, under the ( > 0 / < 0 ) policy, the presence of keys genuinely augments expressive power.

Illustrative Queries

The authors study concrete node queries to illustrate the hierarchy:

• Q_even – “the distinguished node has an even number of neighbors.” This query is not expressible by continuous LocalMax GNNs but becomes expressible with semilinear or arbitrary combination functions.
• Q_G_u – “the connected component of the distinguished node is isomorphic to a fixed graph G with a distinguished root u.” This family of isomorphism‑type queries requires at least semilinear (or arbitrary) combination functions for LocalMax, while LocalSum with continuous functions already captures them.

Implications and Applications

The results have several practical implications. In geometric GNNs, node coordinates act as unique identifiers; the analysis shows that, depending on the aggregation and activation choices, such identifiers can either be harmless (if the model collapses to the unkeyed case) or can dramatically increase the set of graph properties the network can learn. Similarly, positional encodings used in transformer‑style graph models often provide (near‑)unique identifiers; understanding the logical limits of key‑invariant GNNs informs the design of encoding schemes that preserve desirable invariance properties while enhancing expressive power.

Moreover, the paper clarifies the relationship between randomised GNNs (which use random node features to break symmetry) and deterministic key‑invariant GNNs. Randomised models achieve universal approximation only in expectation, whereas key‑invariant GNNs guarantee deterministic isomorphism‑invariance, making them more suitable for tasks where strict symmetry is required.

Technical Contributions

• Formal definition of key‑invariant GNNs and the associated node query semantics.
• Comprehensive logical characterisation of expressive power for each combination‑function class under both LocalMax and LocalSum aggregation.
• Proofs of collapse results under specific acceptance policies, establishing when keys do not increase expressive power.
• Construction of separating examples (e.g., Q_even) that demonstrate strict hierarchies between function classes.
• Extension of known bisimulation‑type closure properties (inverse functional bisimulation for LocalMax, inverse coverings for LocalSum) to the keyed setting.

Future Directions

The authors suggest several avenues for further research:

• Extending the analysis to more expressive message‑passing architectures (e.g., higher‑order GNNs, attention‑based aggregations).
• Investigating the impact of approximate or noisy keys, which arise in practice when positional encodings are not perfectly unique.
• Developing learning algorithms that explicitly enforce key‑invariance, possibly via regularisation or architectural constraints.
• Exploring connections to proof complexity and descriptive complexity theory, given the tight link to order‑invariant logics.

In summary, this work provides a rigorous logical framework for understanding how unique node identifiers influence the expressive capabilities of graph neural networks. By mapping key‑invariant GNNs to well‑studied logical fragments, it clarifies when and how identifiers can be leveraged without sacrificing the desirable symmetry properties of graph learning models.