Discrete Diffusion-Based Model-Level Explanation of Heterogeneous GNNs with Node Features

Many real-world datasets, such as citation networks, social networks, and molecular structures, are naturally represented as heterogeneous graphs, where nodes belong to different types and have additional features. For example, in a citation network, nodes representing “Paper” or “Author” may include attributes like keywords or affiliations. A critical machine learning task on these graphs is node classification, which is useful for applications such as fake news detection, corporate risk assessment, and molecular property prediction. Although Heterogeneous Graph Neural Networks (HGNNs) perform well in these contexts, their predictions remain opaque. Existing post-hoc explanation methods lack support for actual node features beyond one-hot encoding of node type and often fail to generate realistic, faithful explanations. To address these gaps, we propose DiGNNExplainer, a model-level explanation approach that synthesizes heterogeneous graphs with realistic node features via discrete denoising diffusion. In particular, we generate realistic discrete features (e.g., bag-of-words features) using diffusion models within a discrete space, whereas previous approaches are limited to continuous spaces. We evaluate our approach on multiple datasets and show that DiGNNExplainer produces explanations that are realistic and faithful to the model’s decision-making, outperforming state-of-the-art methods.

💡 Research Summary

**
The paper introduces DiGNNExplainer, a novel model‑level explanation framework for heterogeneous Graph Neural Networks (HGNNs) that can generate realistic explanation graphs containing actual discrete node features. Existing post‑hoc explainers either operate at the instance level, ignore heterogeneous node attributes, or generate only continuous‑type explanations, limiting their applicability to real‑world heterogeneous data such as citation networks, social graphs, or molecular structures.

DiGNNExplainer addresses these gaps by employing two discrete denoising diffusion processes. For graph structure synthesis, it builds on DiGress, an existing discrete diffusion model that reconstructs graphs step‑by‑step in a discrete space. Because DiGress requires a fixed graph size, the authors train multiple DiGress models on subgraphs of varying sizes (typically 10–15 nodes) extracted from the original large graph using a forest‑fire sampling algorithm that preserves connectivity and respects the target node type. This yields a pool of small, human‑interpretable candidate graphs.

For node feature synthesis, the paper extends TabDDPM—originally designed for tabular data—to a fully discrete setting, creating DiTabDDPM. The forward diffusion adds discrete noise following DiGress’s cosine scheduler and marginal transition matrices. In the reverse process, a denoising network (based on the MLP‑Block architecture of TabDDPM) predicts a multinomial distribution over the discrete vocabulary for each feature at each timestep, from which the previous‑step features are sampled. The training loss is a cross‑entropy between predicted and true discrete values. Separate feature generators are trained per node type; for the class‑target node type, a distinct generator is learned for each class to capture class‑specific feature distributions.

After generation, each synthetic graph is checked against the dataset’s metagraph (the allowed type‑to‑type edge schema). Graphs violating the schema are discarded. The remaining graphs are fed to the trained HGNN that is being explained. For each class, the graph that yields the highest softmax probability is selected as the explanation for that class. This yields a set of explanation graphs—one per class—that reflect the model’s decision boundaries at a global level.

The authors evaluate DiGNNExplainer on several heterogeneous benchmarks (e.g., DBLP, PubMed, MUTAG). They assess realism using Maximum Mean Discrepancy (MMD) on graph statistics (degree distribution, clustering coefficient, spectral properties) and on node‑feature distributions. Faithfulness is measured in two ways: predictive faithfulness (the softmax score of the selected graph) and ground‑truth faithfulness (the overlap between the community structure of the explanation graph and that of real data). Compared to prior model‑level explainers—XGNN, GNNInterpreter, and D4Explainer—DiGNNExplainer consistently achieves lower MMD (more realistic graphs), higher predictive scores (≈0.94 vs. 0.84‑0.89), and higher ground‑truth faithfulness (≈0.90 vs. 0.65‑0.78). An ablation study shows that using the discrete DiTabDDPM is crucial: replacing it with the original continuous TabDDPM degrades discrete feature fidelity by over 20 %. Removing the metagraph validation also harms realism dramatically.

Strengths of the approach include: (1) genuine support for heterogeneous node attributes beyond one‑hot type encodings; (2) a fully discrete diffusion pipeline that preserves the combinatorial nature of graph data; (3) model‑level explanations that summarize global behavior rather than isolated subgraphs; (4) thorough quantitative evaluation covering both realism and faithfulness.

Limitations are noted: training multiple DiGress models for different graph sizes incurs significant computational overhead; the forest‑fire sampling strategy may bias the shape of explanation graphs and requires careful hyper‑parameter tuning per dataset; the current method focuses on purely discrete features, leaving mixed continuous‑discrete scenarios for future work; and while quantitative metrics are strong, the paper provides limited discussion of how end‑users interpret the generated explanation graphs, suggesting a need for better visualization and domain‑specific narrative tools.

In conclusion, DiGNNExplainer advances the state of the art in explainable GNNs by integrating discrete diffusion for both structure and feature generation, delivering realistic, faithful, and class‑specific explanation graphs for heterogeneous networks. Future research directions include extending the framework to mixed‑type features, scaling diffusion models to larger explanation graphs, and coupling the generated graphs with natural‑language or visual explanations to improve human interpretability in high‑stakes applications such as healthcare, finance, and legal decision‑making.