Is Meta-Path Attention an Explanation? Evidence of Alignment and Decoupling in Heterogeneous GNNs

Meta-path-based heterogeneous graph neural networks aggregate over meta-path-induced views, and their semantic-level attention over meta-path channels is widely used as a narrative for ``which semantics matter.’’ We study this assumption empirically by asking: when does meta-path attention reflect meta-path importance, and when can it decouple? A key challenge is that most post-hoc GNN explainers are designed for homogeneous graphs, and naive adaptations to heterogeneous neighborhoods can mix semantics and confound perturbations. To enable a controlled empirical analysis, we introduce MetaXplain, a meta-path-aware post-hoc explanation protocol that applies existing explainers in the native meta-path view domain via (i) view-factorized explanations, (ii) schema-valid channel-wise perturbations, and (iii) fusion-aware attribution, without modifying the underlying predictor. We benchmark representative gradient-, perturbation-, and Shapley-style explainers on ACM, DBLP, and IMDB with HAN and HAN-GCN, comparing against xPath and type-matched random baselines under standard faithfulness metrics. To quantify attention reliability, we propose Meta-Path Attention–Explanation Alignment (MP-AEA), which measures rank correlation between learned attention weights and explanation-derived meta-path contribution scores across random runs. Our results show that meta-path-aware explanations typically outperform random controls, while MP-AEA reveals both high-alignment and statistically significant decoupling regimes depending on the dataset and backbone; moreover, retraining on explanation-induced subgraphs often preserves, and in some noisy regimes improves, predictive performance, suggesting an explanation-as-denoising effect.

💡 Research Summary

The paper tackles a fundamental question in heterogeneous graph neural networks (HeteroGNNs) that rely on meta‑path based views: does the semantic‑level attention over meta‑paths truly reflect the importance of those meta‑paths, or can it diverge from the actual contribution? While attention mechanisms have been widely used as post‑hoc explanations in natural‑language processing and homogeneous GNNs, recent work has shown that attention weights are not guaranteed to align with feature importance. This uncertainty is amplified in heterogeneous settings where multiple node and edge types create distinct semantic channels (meta‑paths).

To investigate this, the authors first identify a methodological gap: most existing post‑hoc explainers (gradient‑based, perturbation‑based, Shapley‑style) assume a single homogeneous neighborhood and therefore collapse the heterogeneous semantics when applied naïvely to meta‑path based models. Such collapse leads to (i) semantic mixing, (ii) confounded perturbations that affect several meta‑paths simultaneously, and (iii) attribution artifacts caused by the cross‑meta‑path fusion step.

The core contribution is MetaXplain, a protocol that lifts any homogeneous‑graph explainer into the native meta‑path view domain while preserving semantic fidelity. MetaXplain enforces three consistency conditions:

1. View‑factorized explanation domain (C1) – explanations are produced separately for each meta‑path view, preventing semantic collapse.
2. Channel‑wise, schema‑valid perturbations (C2) – masks or edits are applied only within a single meta‑path view and respect the type constraints of that view.
3. Fusion‑aware attribution (C3) – contribution scores are computed before the model’s cross‑meta‑path fusion, either via gradients on the per‑view embeddings or via occlusion/conditional fidelity on the per‑view inputs, and then aggregated.

With MetaXplain, the authors can apply five representative explainers—Grad, GNNExplainer, PGM‑Explainer, GraphSVX, and GNNShap—to three heterogeneous benchmark graphs (ACM, DBLP, IMDB) using two backbones (HAN and HAN‑GCN). They compare against two baselines: the path‑based explainer xPath and a type‑matched random mask.

To quantify the alignment between attention and explanation, they introduce Meta‑Path Attention–Explanation Alignment (MP‑AEA). MP‑AEA computes the Spearman rank correlation between the learned attention weights for each meta‑path and the contribution scores derived from the post‑hoc explanations, averaging over multiple random seeds. This turns the qualitative “is attention explanatory?” question into a reproducible metric.

Key empirical findings:

• Meta‑path‑aware explanations outperform random baselines on standard faithfulness metrics (prediction fidelity after masking, sparsity). Gradient‑based methods tend to be the most stable, while mask‑based methods benefit from the per‑view formulation for interpretability.
• MP‑AEA reveals heterogeneous alignment: on some dataset‑backbone pairs (e.g., DBLP with HAN‑GCN) the correlation is high (ρ≈0.68), indicating that attention reliably mirrors meta‑path importance. On others (e.g., ACM with HAN) the correlation is low (ρ≈0.25), showing a clear decoupling. Statistical tests confirm that these differences are significant.
• Explanation‑induced subgraph retraining acts as denoising: when the model is retrained on subgraphs consisting only of the meta‑paths highlighted by the explanations, predictive performance is preserved and, in noisy regimes (e.g., removing “Author‑Paper‑Venue” meta‑path in ACM), even improves by 2‑3 %. This suggests that explanations can identify and prune spurious semantic channels.
• Backbone sensitivity: HAN‑GCN, which uses a more complex fusion (e.g., transformer‑style), tends to maintain higher attention‑explanation alignment than the simpler attention‑only HAN, indicating that the fusion design influences how faithfully attention reflects underlying importance.

The authors discuss the implications: meta‑path attention should not be taken as a definitive explanation without validation; MP‑AEA provides a lightweight diagnostic that can be run alongside any explanation pipeline. MetaXplain offers a practical way to reuse existing explainers without redesigning them for heterogeneity, making it applicable to a broad class of future heterogeneous GNNs (e.g., MAGNN, SeHGNN).

Limitations include the reliance on a predefined set of meta‑paths (the protocol does not address automatic meta‑path discovery) and the computational cost that grows with the number of meta‑paths, as each view requires a separate explanation run. Future work could explore meta‑path selection, more scalable explanation strategies, and alternative alignment metrics (e.g., NDCG, Kendall‑tau).

In summary, the paper provides a rigorous empirical framework to assess when meta‑path attention can be interpreted as an explanation and when it diverges. By introducing MetaXplain and MP‑AEA, the authors enable systematic, reproducible analysis of semantic‑level attention in heterogeneous GNNs, uncovering both alignment and decoupling regimes and demonstrating a practical denoising benefit of explanation‑guided subgraph training.