On the Geometry of Discrete Exponential Families with Application to Exponential Random Graph Models

There has been an explosion of interest in statistical models for analyzing network data, and considerable interest in the class of exponential random graph (ERG) models, especially in connection with difficulties in computing maximum likelihood estimates. The issues associated with these difficulties relate to the broader structure of discrete exponential families. This paper re-examines the issues in two parts. First we consider the closure of $k$-dimensional exponential families of distribution with discrete base measure and polyhedral convex support $\mathrm{P}$. We show that the normal fan of $\mathrm{P}$ is a geometric object that plays a fundamental role in deriving the statistical and geometric properties of the corresponding extended exponential families. We discuss its relevance to maximum likelihood estimation, both from a theoretical and computational standpoint. Second, we apply our results to the analysis of ERG models. In particular, by means of a detailed example, we provide some characterization of the properties of ERG models, and, in particular, of certain behaviors of ERG models known as degeneracy.

💡 Research Summary

The paper investigates the geometric structure of discrete exponential families whose base measure is discrete and whose sufficient‑statistic support set P is a convex polytope. The authors’ central insight is that the normal fan of P — the collection of outward normal cones associated with each face of the polytope — fully determines the properties of the extended exponential family. By partitioning the natural‑parameter space into cones of the normal fan, they show that the location of a parameter vector within a particular cone dictates whether the expected sufficient statistic lies in the interior of P, on a face, or at an extreme point. This, in turn, governs the existence and uniqueness of the maximum‑likelihood estimator (MLE). If the parameter lies in the interior of a full‑dimensional cone, the expectation remains in the interior of P, guaranteeing a unique, finite MLE. Conversely, parameters on boundary cones force the expectation onto a lower‑dimensional face, causing the likelihood to be unbounded or the MLE to fail to exist. This geometric characterization refines the classical regularity conditions for exponential families and provides a concrete, visual tool for diagnosing MLE pathology before any numerical optimization is attempted.

Having established this general theory, the authors apply it to exponential random graph (ERG) models, a widely used class of network models that specify probability distributions over graphs via statistics such as edge count, triangle count, and two‑star count. In an ERG model the set of all possible graphs induces a high‑dimensional polytope P in the space of these statistics. The paper presents a detailed nine‑node example, explicitly constructing the polytope, computing its normal fan, and mapping each cone to a region of the parameter space. The analysis reveals that “degeneracy”—the phenomenon where the model places almost all probability mass on a few extreme graphs (e.g., the empty or complete graph)—corresponds precisely to parameters that fall on the boundary cones of the normal fan. When a parameter vector lies in such a cone, the expected sufficient statistics are forced onto a face of P, and the likelihood surface becomes flat or spikes toward the extremes, making standard MLE algorithms diverge or converge to nonsensical values.

The authors further discuss practical implications. By pre‑computing the normal fan, one can quickly test whether a proposed parameter lies in a “good” interior cone, thereby guaranteeing the existence of an MLE and avoiding costly failed optimizations. They propose an algorithmic framework that first identifies the cone containing the current iterate, then uses the pre‑computed expectation and variance information for that cone to guide a Newton‑Raphson or stochastic approximation step. This cone‑aware approach dramatically reduces the number of iterations needed and prevents the algorithm from wandering into degenerate regions.

Beyond ERG models, the paper suggests that the normal‑fan perspective can be extended to any discrete exponential family with a polyhedral support, such as contingency‑table models, hierarchical log‑linear models, and certain graphical models used in genetics and epidemiology. The geometric lens unifies questions of identifiability, MLE existence, and computational stability under a single combinatorial object.

In summary, the work makes three major contributions: (1) it establishes the normal fan of the convex support as the key geometric structure governing the extended exponential family; (2) it translates this abstract geometry into concrete diagnostics for MLE existence and degeneracy in ERG models; and (3) it leverages the fan to design more reliable, faster algorithms for likelihood‑based inference in discrete network models. The results deepen our theoretical understanding of discrete exponential families and provide actionable tools for practitioners dealing with complex network data.