Tiny Moves: Game-based Hypothesis Refinement

Most machine learning approaches to scientific discovery frame hypotheses as end-to-end predictions, obscuring the incremental structure of scientific reasoning. We propose The Hypothesis Game, a symbolic formalism for hypothesis refinement in which LLM agents operate on a shared hypothesis state using a fixed grammar of reasoning moves. The framework is motivated by the observation that scientific progress often proceeds through small, localized revisions, grounded in domain context, rather than extensive rewrites. We instantiate a minimal game with LLM agents and evaluate it on pathway-level mechanistic refinement tasks. In the primary setting of corruption recovery, where hypotheses contain controlled errors, the game-based approach consistently removes more errors and achieves higher precision than strong prompting baselines, while preserving valid structure through incremental edits. In a secondary reconstruction setting from partial cues, it performs comparably to the strongest baseline, indicating that explicit move-based refinement remains competitive even when ground-truth recovery is difficult. These findings support game-based reasoning as a principled route to more controllable, interpretable, and transferable hypothesis refinement systems for scientific discovery.

💡 Research Summary

The paper introduces “The Hypothesis Game,” a symbolic framework that treats scientific hypothesis refinement as an iterative game played on a shared hypothesis state. A hypothesis is represented as a set of fragments (text claims, subject‑relation‑object triples, or optionally a graph). The core of the game is a fixed grammar of reasoning moves—prune, expand, retrieve, and debate—each formalized as a function that maps the current hypothesis (and optional context such as cell type or disease) to an updated hypothesis.

Game dynamics are governed by “modes” (e.g., discovery vs. validation) that bias the probability distribution over moves, and by a move‑budget k_max that limits how many moves can be applied in a single round. Two variants are defined: (1) Simple Hypothesis Refinement, which updates the entire hypothesis in each round, and (2) Localized Hypothesis Refinement, which selects a sub‑region of the hypothesis (via a selector σ) and applies moves only to that region while enforcing global consistency. This distinction mirrors the difference between wholesale rewrites and fine‑grained edits typical of real scientific work.

Implementation uses a central LLM controller called the Game Master, which diagnoses the current hypothesis, selects moves according to the prescribed mode, and dispatches specialized sub‑agents to execute the moves. The “retrieve_expand” move is instantiated in two ways: (a) searching an external corpus for evidence, and (b) using the LLM’s internal knowledge. Modes are realized simply by injecting a textual description of the mode into the initial prompt, thereby shaping the implicit policy π_M. No explicit scoring function drives the controller in the current prototype, but the paper outlines a vector of metrics (distance to known hypotheses, diversity, connectivity, traceability) that could be combined into a scalar utility for future autonomous agents.

To evaluate the approach, the authors construct two novel benchmark tasks based on curated human pathways from Reactome. In the “corruption recovery” task, controlled errors (incorrect reactions, missing steps) are injected into valid pathways, and the system must detect and correct them while preserving the correct structure. In the “reconstruction from partial cues” task, only a sparse subset of pathway statements is provided, and the system must rebuild the full mechanism. The datasets comprise 20 corrupted pathways (multiple error variants per pathway, yielding 2 880 experiments) and 100 pathways for reconstruction (820 experiments).

Metrics include error removal rate, precision, recall, and F1. Baselines consist of strong prompting strategies such as Chain‑of‑Thought, Self‑Consistency, and zero‑shot prompting. Results show that the Hypothesis Game consistently removes more injected errors and achieves higher precision than the baselines in the corruption‑recovery setting, while maintaining comparable recall, leading to a superior F1 score. In the reconstruction setting, performance is on par with the strongest baseline, indicating that the move‑based incremental approach remains competitive even when only partial information is available.

The contributions are threefold: (1) a formal compositional game model for hypothesis refinement that makes reasoning steps explicit and reusable; (2) a minimal implementation with LLM agents that produces transparent refinement trajectories; (3) empirical evidence that structured, move‑based editing improves controllability, interpretability, and transferability of scientific reasoning systems. Limitations include the absence of an automated scoring loop, a relatively small move set, reliance on the underlying LLM’s correctness, and modest dataset size, which together constrain generalization. Future work is suggested to incorporate reinforcement‑learning policies, richer move grammars, hybrid scoring that blends computational metrics with experimental feedback, and extensions to other scientific domains such as chemical synthesis planning or physical model discovery.