Generating Probability Distributions using Multivalued Stochastic Relay Circuits

The problem of random number generation dates back to von Neumann’s work in 1951. Since then, many algorithms have been developed for generating unbiased bits from complex correlated sources as well as for generating arbitrary distributions from unbiased bits. An equally interesting, but less studied aspect is the structural component of random number generation as opposed to the algorithmic aspect. That is, given a network structure imposed by nature or physical devices, how can we build networks that generate arbitrary probability distributions in an optimal way? In this paper, we study the generation of arbitrary probability distributions in multivalued relay circuits, a generalization in which relays can take on any of N states and the logical ‘and’ and ‘or’ are replaced with ‘min’ and ‘max’ respectively. Previous work was done on two-state relays. We generalize these results, describing a duality property and networks that generate arbitrary rational probability distributions. We prove that these networks are robust to errors and design a universal probability generator which takes input bits and outputs arbitrary binary probability distributions.

💡 Research Summary

The paper “Generating Probability Distributions using Multivalued Stochastic Relay Circuits” extends the theory of stochastic relay circuits from the classical binary (two‑state) setting to a general N‑state framework, where each relay can assume any of the values {0,1,…,N‑1}. In this multivalued setting the conventional Boolean “and” and “or” operators are replaced by the min and max functions respectively, so a series connection yields the minimum of the component states and a parallel connection yields the maximum.

The authors first formalize stochastic N‑state switches (pswitches) as probability vectors v = (p₀,…,p_{N‑1}) and show how series and parallel compositions transform these vectors by enumerating all state pairs and applying min or max. They then prove a duality theorem: swapping series with parallel connections and replacing each switch by its dual (the distribution reflected about the midpoint) produces a circuit whose output distribution is the dual of the original. The proof proceeds by induction on series‑parallel constructions and mirrors the well‑known duality for resistor networks and binary stochastic relays.

The core technical contribution is an explicit construction algorithm for binary (dyadic) probability distributions on three states and, by extension, on any number of states. For a target distribution of the form (a/2ⁿ, b/2ⁿ, c/2ⁿ) the algorithm recursively inserts a ½‑pswitch (the vector (½, 0, ½)) and splits the probability mass at the midpoint of the unit interval. Depending on whether the cumulative mass of the first, first‑two, or all three components exceeds ½, the algorithm chooses one of three decomposition cases, each respecting the multivalued composition rules derived earlier. The recursion terminates when the denominator reaches 1, at which point a deterministic switch (a unit vector) completes the construction. The authors prove that at most 2ⁿ − 1 pswitches are required, matching the optimal bound known for binary circuits.

The method generalizes to N‑state distributions (a₀/2ⁿ,…,a_{N‑1}/2ⁿ) by repeatedly halving the interval and inserting an N‑state ½‑pswitch of the form (½, 0,…,0, ½). The depth of the resulting series‑parallel tree is bounded by N·n, leading to a total of O(N·2ⁿ) pswitches.

A significant practical concern—robustness to component errors—is addressed analytically. If each pswitch’s actual probability deviates by at most ε from its nominal value, the total variation distance between the intended output distribution and the realized one grows at most linearly with the circuit depth (≤ N·n·ε). Consequently, even with modest manufacturing tolerances the constructed distributions remain accurate.

Building on these foundations, the authors design a Universal Probability Generator (UPG). The UPG takes a stream of unbiased input bits and, using a fixed tree of ½‑pswitches and deterministic selectors, can output any binary distribution (p, 1 − p) where p is a dyadic rational. The input bits act as control signals that navigate the tree, effectively performing the same recursive splitting as the construction algorithm but driven by external randomness rather than internal stochastic switches. This hardware‑level generator offers an alternative to classic algorithmic transformations such as von Neumann’s extractor or the Knuth‑Yao tree, with the advantage of being realizable in purely physical substrates.

The paper also explores partially ordered state spaces, suggesting that when switches have constraints that prevent a total ordering of states, similar min‑max constructions can still be applied, opening avenues for modeling stochastic behavior in neural circuits and DNA strand displacement systems. The authors illustrate how multivalued stochastic relays can capture timing relationships in biochemical pathways, where the state of a component corresponds to the time of a reaction’s completion.

In summary, the work provides a comprehensive theoretical toolkit for designing multivalued stochastic relay networks that can synthesize arbitrary rational probability distributions. It establishes duality, presents optimal recursive constructions for binary and N‑state distributions, quantifies error propagation, and delivers a universal hardware generator. These contributions bridge the gap between algorithmic random number generation and structural, physically constrained stochastic computation, with potential impact on neuromorphic hardware, synthetic biology, and low‑power random number generators.