Breaking Symmetry Bottlenecks in GNN Readouts

Graph neural networks (GNNs) are widely used for learning on structured data, yet their ability to distinguish non-isomorphic graphs is fundamentally limited. These limitations are usually attributed to message passing; in this work we show that an independent bottleneck arises at the readout stage. Using finite-dimensional representation theory, we prove that all linear permutation-invariant readouts, including sum and mean pooling, factor through the Reynolds (group-averaging) operator and therefore project node embeddings onto the fixed subspace of the permutation action, erasing all non-trivial symmetry-aware components regardless of encoder expressivity. This yields both a new expressivity barrier and an interpretable characterization of what global pooling preserves or destroys. To overcome this collapse, we introduce projector-based invariant readouts that decompose node representations into symmetry-aware channels and summarize them with nonlinear invariant statistics, preserving permutation invariance while retaining information provably invisible to averaging. Empirically, swapping only the readout enables fixed encoders to separate WL-hard graph pairs and improves performance across multiple benchmarks, demonstrating that readout design is a decisive and under-appreciated factor in GNN expressivity.

💡 Research Summary

This paper uncovers a previously overlooked source of expressivity limitation in graph neural networks (GNNs): the readout stage. While most prior work attributes GNN’s inability to distinguish certain non‑isomorphic graphs to the message‑passing (MP) layers—often analyzed through the Weisfeiler–Leman (WL) hierarchy—this work shows that even with an arbitrarily powerful MP encoder, a linear permutation‑invariant readout (such as sum or mean pooling) inevitably collapses all non‑trivial symmetry information.

Using finite‑dimensional representation theory, the authors model the action of the node‑permutation group (H) on the space of node embeddings (U). They define the Reynolds (group‑averaging) operator (p_{\text{avg}} = \frac{1}{|H|}\sum_{h\in H}\rho(h)), where (\rho) is the representation induced by permuting rows of the feature matrix. The key theoretical contribution (Theorem 3.3) proves that any linear (H)-invariant map (f:U\to U’) must factor through this averaging operator: (f = f’\circ p_{\text{avg}}). Consequently, linear readouts can only access the trivial isotypic component (U_1) (often a one‑dimensional subspace corresponding to the global average of node features) and discard all higher‑order irreducible components that encode richer structural cues.

The paper illustrates this phenomenon with a concrete example: two six‑node graphs that are WL‑equivalent. Even if a sophisticated MP encoder populates distinct non‑trivial representation components for the two graphs, any linear readout projects both embeddings onto the same trivial component, rendering the graphs indistinguishable. This demonstrates that the readout bottleneck is orthogonal to WL‑type limitations and can be the decisive factor in practice.

To overcome the bottleneck, the authors propose projector‑based invariant readouts. For a given graph (G), they compute the automorphism group (\text{Aut}(G)) and construct orthogonal projectors onto each isotypic subspace using the classic formula
\