Quantifying Explanation Quality in Graph Neural Networks using Out-of-Distribution Generalization

Evaluating the quality of post-hoc explanations for Graph Neural Networks (GNNs) remains a significant challenge. While recent years have seen an increasing development of explainability methods, current evaluation metrics (e.g., fidelity, sparsity) often fail to assess whether an explanation identifies the true underlying causal variables. To address this, we propose the Explanation-Generalization Score (EGS), a metric that quantifies the causal relevance of GNN explanations. EGS is founded on the principle of feature invariance and posits that if an explanation captures true causal drivers, it should lead to stable predictions across distribution shifts. To quantify this, we introduce a framework that trains GNNs using explanatory subgraphs and evaluates their performance in Out-of-Distribution (OOD) settings (here, OOD generalization serves as a rigorous proxy for the explanation’s causal validity). Through large-scale validation involving 11,200 model combinations across synthetic and real-world datasets, our results demonstrate that EGS provides a principled benchmark for ranking explainers based on their ability to capture causal substructures, offering a robust alternative to traditional fidelity-based metrics.

💡 Research Summary

The paper tackles a fundamental problem in graph neural network (GNN) interpretability: how to assess whether a post‑hoc explanation truly captures the causal factors that generate the target label, rather than merely reflecting spurious correlations. Existing evaluation metrics such as fidelity, sparsity, or stability measure how well an explanation aligns with a model’s predictions or how compact it is, but they do not address causal validity. To fill this gap, the authors introduce the Explanation‑Generalization Score (EGS), a metric that quantifies the causal relevance of GNN explanations by leveraging out‑of‑distribution (OOD) generalization as a proxy for causality.

The central hypothesis is rooted in the invariance principle: causal features remain predictive across different environments (or “shifts”), whereas spurious features do not. If an explanation correctly identifies the causal subgraph C within a graph G, then training a model that is constrained to use only the nodes and edges highlighted by that explanation should improve OOD performance relative to an unconstrained baseline. The authors formalize this idea in two steps. First, they demonstrate on synthetic and real‑world molecular datasets that models trained exclusively on ground‑truth causal subgraphs achieve markedly better OOD accuracy under various distribution shifts (e.g., covariate shift in node attributes, structural shift in graph topology). This empirical evidence validates the premise that causal substructures are invariant predictors.

Second, they turn the premise around to evaluate explainers. For each explainer E (e.g., GNNExplainer, GraphMask, Grad‑CAM, Integrated Gradients, etc.), they obtain a representative explanation—typically a binary mask over nodes/edges. They then train an Explanation‑Guided GNN (EG‑GNN) where the loss includes a regularization term that forces the model’s gradients to focus on the masked (explainable) components and suppress gradients on unmasked components. In effect, the explainer’s output becomes a supervisory signal that shapes the model’s internal attention to the purportedly causal region.

The OOD performance of this EG‑GNN is measured on test sets that differ from the training distribution in controlled ways (environmental variables, edge density, motif distribution, etc.). The Explanation‑Generalization Score is defined as the relative improvement of the EG‑GNN’s OOD accuracy over a vanilla GNN baseline:

EGS(E) =