Linear Models of Computation and Program Learning

We consider two classes of computations which admit taking linear combinations of execution runs: probabilistic sampling and generalized animation. We argue that the task of program learning should be more tractable for these architectures than for conventional deterministic programs. We look at the recent advances in the “sampling the samplers” paradigm in higher-order probabilistic programming. We also discuss connections between partial inconsistency, non-monotonic inference, and vector semantics.

💡 Research Summary

The paper proposes a novel perspective on program learning by focusing on two computational architectures that admit linear combinations of execution runs: probabilistic sampling and generalized animation. The authors argue that these “linear models” make learning programs more tractable than in conventional deterministic settings because the underlying semantics are inherently linear and thus amenable to continuous optimization techniques.

In the probabilistic sampling domain, the authors describe how multiple samplers, each generating points from a distinct distribution, can be run in parallel at different speeds. By adjusting the relative speeds, one can obtain any convex combination of the target distributions. To extend beyond positive coefficients, they introduce a dual‑channel scheme consisting of a positive and a negative sampling stream. This mirrors biological evidence of signed neural signals (e.g., retinal ON/OFF pathways) and connects to the mathematical notion of signed measures, allowing “negative probabilities” or quasiprobabilities. The paper references historical work on Wigner quasiprobabilities, Kozen’s signed‑measure semantics for probabilistic programs, and recent quantum‑algorithmic applications, emphasizing that signed measures turn program denotations into continuous linear operators on Banach lattices.

Generalized animation is defined as a map from time to a “generalized monochrome image,” where an image is a function from an abstract point set to real numbers. Points may be pixels, graph vertices, grammar symbols, etc. Linear combination of images is performed point‑wise, and both positive and negative coefficients are naturally supported (as in audio/video mixing). The authors argue that such animation systems share with probabilistic samplers the property that complex behavior can emerge from very short programs, and that they tend to be non‑brittle under mutation and crossover, making them suitable for evolutionary search.

A substantial portion of the paper is devoted to the mathematical foundations that enable these linear models. By introducing partial inconsistency—e.g., extending interval numbers with “pseudo‑segments” where the lower bound exceeds the upper bound—the authors obtain a space equipped with two dual orderings: an informational order (⊑) based on reverse inclusion and a material order (≤) based on component‑wise comparison. This dual topology (Scott topologies in opposite directions) supports both upward (monotonic) and downward (anti‑monotonic) inference steps, providing a natural setting for non‑monotonic reasoning. Bilattices, bitopologies, and Hahn‑Jordan decompositions are presented as concrete algebraic structures that embody these ideas. The paper shows that with the “true minus” operation (component‑wise negation) the partially inconsistent interval numbers form a real vector space, enabling linear algebraic manipulation of program semantics.

From a software engineering viewpoint, the authors advocate data‑flow programming as the natural host for these models. By representing a data‑flow graph as a bipartite graph (nodes for transformations vs. nodes for linear combinations) it can be encoded as a real matrix. Continuous modification of matrix entries yields “almost continuous transformations” of the program while it is running, allowing the system to sample trajectories in program‑space rather than merely sampling discrete syntax trees. This bridges the gap between evolutionary program synthesis (which traditionally relies on discrete crossover/mutation) and gradient‑based learning.

The paper surveys recent advances in higher‑order probabilistic programming that embody the “sampling the samplers” paradigm. Notably, Perov and Wood’s work on compiling probabilistic programs into samplers that directly draw from posterior distributions (implemented in the Anglican engine with particle MCMC) and Lake’s compositional concept learning, where generative models emit other generative models, are highlighted as concrete steps toward program learning via higher‑order sampling. These systems demonstrate that a small number of examples can suffice to learn rich conceptual structures when the underlying computation supports linear combination and higher‑order sampling.

Finally, the authors discuss how partial inconsistency and vector semantics can enrich denotational semantics for AI systems, enabling robust handling of noisy or contradictory information and facilitating non‑monotonic inference. By embedding program meanings into vector spaces, one can apply familiar machine‑learning techniques (e.g., matrix factorization, gradient descent) to the program learning problem.

In conclusion, the paper argues that linear models of computation—probabilistic sampling with signed measures and generalized animation with point‑wise linear combination—provide a unified, mathematically rich framework that makes program learning more continuous, flexible, and amenable to both evolutionary and gradient‑based methods. This framework promises to overcome the brittleness of deterministic programs and to integrate evolutionary, Bayesian, and non‑monotonic reasoning within a single algebraic setting, opening new avenues for advanced AI and low‑power, noise‑tolerant implementations.