PolyGraph Discrepancy: a classifier-based metric for graph generation

Existing methods for evaluating graph generative models primarily rely on Maximum Mean Discrepancy (MMD) metrics based on graph descriptors. While these metrics can rank generative models, they do not provide an absolute measure of performance. Their values are also highly sensitive to extrinsic parameters, namely kernel and descriptor parametrization, making them incomparable across different graph descriptors. We introduce PolyGraph Discrepancy (PGD), a new evaluation framework that addresses these limitations. It approximates the Jensen-Shannon distance of graph distributions by fitting binary classifiers to distinguish between real and generated graphs, featurized by these descriptors. The data log-likelihood of these classifiers approximates a variational lower bound on the JS distance between the two distributions. Resulting metrics are constrained to the unit interval [0,1] and are comparable across different graph descriptors. We further derive a theoretically grounded summary metric that combines these individual metrics to provide a maximally tight lower bound on the distance for the given descriptors. Thorough experiments demonstrate that PGD provides a more robust and insightful evaluation compared to MMD metrics. The PolyGraph framework for benchmarking graph generative models is made publicly available at https://github.com/BorgwardtLab/polygraph-benchmark.

💡 Research Summary

Graph generative models (GGMs) have become central to many scientific domains, yet their progress is hampered by unreliable evaluation metrics. The dominant approach, Maximum Mean Discrepancy (MMD) computed on hand‑crafted graph descriptors, suffers from two fundamental drawbacks: (1) the score depends heavily on the choice of kernel and descriptor, so it lacks an absolute scale and cannot be compared across different feature sets; (2) with typical benchmark sizes of only 20–40 graphs, MMD estimates exhibit high bias and variance, leading to unstable model rankings.

The authors propose PolyGraph Discrepancy (PGD), a classifier‑based metric that directly approximates the Jensen‑Shannon (JS) distance between the true graph distribution and the distribution generated by a model. The JS distance is bounded in