Signal from Structure: Exploiting Submodular Upper Bounds in Generative Flow Networks

Generative Flow Networks (GFlowNets; GFNs) are a class of generative models that learn to sample compositional objects proportionally to their a priori unknown value, their reward. We focus on the case where the reward has a specified, actionable structure, namely that it is submodular. We show submodularity can be harnessed to retrieve upper bounds on the reward of compositional objects that have not yet been observed. We provide in-depth analyses of the probability of such bounds occurring, as well as how many unobserved compositional objects can be covered by a bound. Following the Optimism in the Face of Uncertainty principle, we then introduce SUBo-GFN, which uses the submodular upper bounds to train a GFN. We show that SUBo-GFN generates orders of magnitude more training data than classical GFNs for the same number of queries to the reward function. We demonstrate the effectiveness of SUBo-GFN in terms of distribution matching and high-quality candidate generation on synthetic and real-world submodular tasks.

💡 Research Summary

This paper investigates how to exploit structural properties of the reward function in Generative Flow Networks (GFNs). While prior GFN work assumes an arbitrary reward that can only be observed at terminal states, the authors focus on the practically important case where the reward is a submodular set function, i.e., it satisfies diminishing returns. Leveraging submodularity, they derive a simple upper‑bound formula: for any terminal set x, any intermediate subset s ⊂ x and any element a ∈ x \ s, the true reward satisfies
(R(x) ≤ UB(x|s,a) = R(s∪{a}) – R(s) + R(x\setminus{a})).
Because the marginal gain of adding a to a small set s overestimates its gain when added to the full set x, this bound becomes tighter as s approaches x.

The authors formalize how such bounds can be generated from observed trajectories. They define “parent trajectories” that pass through a parent of x (i.e., x \ {a}) and terminate elsewhere, and “compatible trajectories” that contain a state s ⊂ parent and then transition to s∪{a}. Each pair of a parent and a compatible trajectory yields one upper bound on R(x). By constructing a bipartite graph G(x) whose vertices are all possible trajectories and edges correspond to valid parent‑compatible pairs, they can count how many distinct bounds are expected after sampling m trajectories uniformly (or under an ε‑greedy policy).

Proposition 4.5 shows that the expected number of distinct bounds for any terminal state grows as
Ω(N C!·(1−C⁻¹)^{N(C−1)}·(1−e^{−m/(NC)})),
where N is the number of actions and C the cardinality constraint. Theorem 4.6 further proves that the probability of obtaining at least one bound for a given x is at least 1−exp(−Ω(Λ(m))) with Λ(m) a function of N, C, and m, using Janson’s inequality to handle dependencies between edges. These results give a rigorous guarantee that, even with modest sampling, a substantial fraction of the terminal space will be covered by useful upper bounds.

Building on this theory, the authors propose SUBo‑GFN (Submodular Upper Bound GFN). During training, whenever a state’s true reward is unavailable, the algorithm substitutes the tightest available upper bound as an optimistic learning signal. This “optimism in the face of uncertainty” (OFU) principle allows the forward and backward policies to be updated with many more pseudo‑rewards than the number of actual reward queries, dramatically increasing data efficiency.

Empirically, SUBo‑GFN is evaluated on synthetic submodular functions and on two real‑world tasks: sensor selection and influence maximization. Compared to standard GFN baselines, SUBo‑GFN generates orders of magnitude more training pairs per reward query, leading to better distribution matching (lower KL divergence), higher probability of sampling top‑reward candidates, and improved diversity of generated solutions. The experiments confirm the theoretical predictions: the number of distinct bounds grows with the number of sampled trajectories, and the coverage of the terminal space quickly approaches the theoretical lower bounds.

The paper’s contributions are threefold: (1) a novel use of submodular structure to derive provable upper bounds for unobserved objects; (2) a rigorous probabilistic analysis of how many bounds can be expected and how much of the solution space they cover; (3) a practical algorithm (SUBo‑GFN) that leverages these bounds to achieve superior sample efficiency and performance. Limitations include reliance on the submodular assumption (bounds become meaningless for non‑submodular rewards) and the need for sufficient intermediate reward observations. Future work may extend the approach to broader classes of structured rewards, develop adaptive tightening of bounds, or combine SUBo‑GFN with other exploration strategies such as Thompson sampling. Overall, the study opens a new direction for GFN research by showing that reward structure can be a powerful lever for both theoretical guarantees and practical gains.