Causal discovery—the problem of inferring causal structure from data—is fundamental to scientific inquiry, yet remains provably underspecified without experimental intervention . Observational data alone can at most identify the Markov equivalence class of DAGs , and the presence of latent confounders further complicates identifiability. This has motivated recent interest in leveraging external knowledge sources, particularly Large Language Models (LLMs), which encode substantial causal knowledge from pretraining corpora .

However, existing approaches to LLM-guided causal discovery remain fundamentally heuristic. Prior work such as Democritus treats LLM outputs as “soft priors” integrated via post-hoc weighting, lacking principled treatment of:

1. Coherence: How do we ensure local LLM beliefs about variable subsets combine into a globally consistent causal structure?

2. Contradictions: What happens when the LLM provides conflicting information about overlapping variable subsets?

3. Latent Variables: How do we project global causal models onto observed subsets while accounting for hidden confounders?

We propose Holograph (Holistic Optimization of Latent Observations via Gradient-based Restriction Alignment for Presheaf Harmony), a framework that addresses these challenges through the lens of sheaf theory. Our key insight is that local causal beliefs can be formalized as sections of a presheaf over the power set of variables. While full sheaf structure (including Locality) fails due to non-local latent coupling, we demonstrate that Identity, Transitivity, and Gluing axioms hold to numerical precision ($`< 10^{-6}`$), enabling coherent belief aggregation.

Contributions.

1. Sheaf-Theoretic Framework: We formalize LLM-guided causal discovery as a presheaf satisfaction problem, where local sections are linear SEMs and restriction maps implement Algebraic Latent Projection.

2. Natural Gradient Optimization: We derive a natural gradient descent algorithm on the belief manifold with Tikhonov regularization for numerical stability.

3. Active Query Selection: We use Expected Free Energy (EFE) to select maximally informative LLM queries, balancing epistemic and instrumental value.

4. Theoretical Analysis: We empirically verify that Identity, Transitivity, and Gluing axioms hold to numerical precision, while systematically identifying Locality violations arising from non-local latent coupling.

5. Empirical Validation: Comprehensive experiments on synthetic (ER, SF) and real-world (Sachs, Asia) benchmarks, demonstrating +91% F1 improvement over NOTEARS in extreme low-data regimes ($`N \le 10`$) and +13.6% F1 improvement when using Holograph priors to regularize statistical methods.

6. Implementation Verification: Complete mathematical verification that all 15 core formulas in the specification match the implementation to numerical precision (Appendix 6.6).

Key Finding 1: Locality Failure as Discovery.

Our sheaf exactness experiments (Section 4.5) reveal a striking result: while Identity ($`\rho_{UU} = \text{id}`$), Transitivity ($`\rho_{ZU} = \rho_{ZV} \circ \rho_{VU}`$), and Gluing axioms pass with errors $`< 10^{-6}`$, the Locality axiom systematically fails with errors scaling as $`\mathcal{O}(\sqrt{n})`$ with graph size. This is not a bug but a discovery: it reveals fundamental non-local information propagation through latent confounders. The failure quantitatively measures the “non-sheafness” of causal models under latent projections—a diagnostic that could guide when latent variable modeling is necessary.

Key Finding 2: Sample Efficiency & Hybrid Synergy.

Our sample efficiency experiments (Section 4.3) establish a clear decision boundary for when to use LLM-based discovery:

• Low-data regime ($`N < 20`$): Holograph’s zero-shot approach achieves F1 = 0.67 on semantically rich domains, outperforming NOTEARS by up to +91% relative F1 when only $`N=5`$ samples are available.

• Hybrid synergy: When some data is available ($`N = 10`$–$`50`$), using Holograph priors to regularize NOTEARS yields +13.6% F1 improvement by preventing overfitting to sparse observations.

• Semantic advantage: Performance depends critically on LLM domain knowledge. On Asia (epidemiology with intuitive variable names), Holograph achieves F1 = 0.67; on Sachs (specialized protein signaling), only F1 = 0.20.

Related Work

Continuous Optimization for Causal Discovery.

NOTEARS pioneered continuous optimization for DAG learning via the acyclicity constraint $`h(\mathbf{W}) = \mathop{\mathrm{tr}}(e^{\mathbf{W}\circ \mathbf{W}}) - n`$. Extensions include GOLEM with likelihood-based scoring and DAGMA using log-determinant characterizations. Holograph builds on this foundation, adding sheaf-theoretic consistency.

LLM-Guided Causal Discovery.

Recent work explores LLMs as causal knowledge sources. benchmark LLMs on causal inference tasks, while propose active querying strategies. Democritus uses LLM beliefs as soft priors but lacks principled treatment of coherence. Emerging “causal foundation models” aim to embed causality into LLM training , yet most approaches treat LLMs as “causal parrots” that recite knowledge without verification. Our sheaf-theoretic framework addresses this gap by providing formal coherence checking via presheaf descent conditions, enabling systematic detection of contradictions in LLM beliefs.

Active Learning for Causal Discovery.

Active intervention selection has been studied extensively . apply active learning to Bayesian networks. Our EFE-based query selection extends these ideas to the LLM querying setting, balancing epistemic uncertainty and instrumental value.

Latent Variable Models.

The FCI algorithm handles latent confounders via ancestral graphs. Recent work on ADMGs provides the graphical semantics underlying our causal states. The algebraic latent projection in Holograph provides an alternative continuous relaxation for latent variable marginalization.

Sheaf Theory in Machine Learning.

Sheaf neural networks apply sheaf theory to GNNs. study sheaf Laplacians for heterogeneous data. To our knowledge, Holograph is the first application of sheaf theory to causal discovery, using presheaf descent for belief coherence.

Methodology

We now present the technical foundations of Holograph, proceeding from the mathematical framework to the optimization algorithm.

Presheaf of Causal Models

Let $`\mathcal{V} = \{X_1, \ldots, X_n\}`$ be a set of random variables. We define a presheaf $`\mathcal{F}`$ over the power set $`2^{\mathcal{V}}`$ (ordered by inclusion) whose sections are linear Structural Equation Models (SEMs) .

The pair $`(\mathbf{W}, \mathbf{M})`$ corresponds to an Acyclic Directed Mixed Graph (ADMG) where directed edges encode causal effects and bidirected edges (encoded in $`\mathbf{M}`$) represent latent confounding.

Probabilistic Model and Semantic Energy

To enable gradient-based optimization, we define a probabilistic model over LLM text observations $`y`$ given causal parameters $`\theta = (\mathbf{W}, \mathbf{L})`$.

\begin{equation}
P(y | \theta) = \frac{1}{Z(\theta)} \exp\left( -\beta \, \mathcal{E}_{\text{sem}}(\theta, y) \right)
\label{eq:gibbs}
\end{equation}

\begin{equation}
\mathcal{E}_{\text{sem}}(\theta, y) = \| \phi(y) - \Psi(\mathbf{W}, \mathbf{M}) \|^2_{\mathcal{H}}
\label{eq:semantic-energy}
\end{equation}

This formulation provides the probabilistic foundation for:

1. Loss Function: The negative log-likelihood yields $`\mathcal{L}_{\text{sem}} = \beta \, \mathcal{E}_{\text{sem}} + \log Z`$, where we approximate $`Z`$ as constant during optimization.

2. Fisher Information Matrix: The metric tensor $`\mathbf{G}(\theta)`$ arises naturally from this Gibbs measure (Section 3.7).

Algebraic Latent Projection

The key technical contribution is the restriction morphism $`\rho_{UV}`$ that projects a causal state from a larger context $`U`$ to a smaller context $`V \subset U`$. When hidden variables exist in $`H = U \setminus V`$, we cannot simply truncate matrices; we must account for how hidden effects propagate through the causal structure.

\begin{equation}
\mathbf{W}= \begin{pmatrix} \mathbf{W}_{OO} & \mathbf{W}_{OH} \\ \mathbf{W}_{HO} & \mathbf{W}_{HH} \end{pmatrix}, \quad
\mathbf{M}= \begin{pmatrix} \mathbf{M}_{OO} & \mathbf{M}_{OH} \\ \mathbf{M}_{HO} & \mathbf{M}_{HH} \end{pmatrix}
\end{equation}

\begin{equation}
\mathbf{A} = \mathbf{W}_{OH}(\mathbf{I} - \mathbf{W}_{HH})^{-1}
\label{eq:absorption}
\end{equation}

\begin{align}
\widetilde{\mathbf{W}}&= \mathbf{W}_{OO} + \mathbf{A} \mathbf{W}_{HO} \label{eq:w-proj} \\
\widetilde{\mathbf{M}}&= \mathbf{M}_{OO} + \mathbf{A} \mathbf{M}_{HH} \mathbf{A}^\top + \mathbf{M}_{OH} \mathbf{A}^\top + \mathbf{A} \mathbf{M}_{HO} \label{eq:m-proj}
\end{align}

The absorption matrix $`\mathbf{A}`$ captures how effects from observed to hidden variables “bounce back” through the hidden subgraph. The condition $`\rho(\mathbf{W}_{HH}) < 1`$ (spectral radius $`< 1`$) ensures the Neumann series $`(I - \mathbf{W}_{HH})^{-1} = \sum_{k=0}^\infty \mathbf{W}_{HH}^k`$ converges, corresponding to acyclicity among hidden variables.

Frobenius Descent Condition

For the presheaf to be coherent, sections over overlapping contexts must agree on their intersection. Given contexts $`U_i, U_j`$ with intersection $`V_{ij} = U_i \cap U_j`$, the Frobenius descent loss is:

\begin{equation}
\mathcal{L}_{\text{descent}} = \sum_{i,j} \left( \left\|\rho_{V_{ij}}(\theta_i) - \rho_{V_{ij}}(\theta_j)\right\|_F^2 \right)
\label{eq:descent-loss}
\end{equation}

where $`\left\|\cdot\right\|_F`$ denotes the Frobenius norm. This loss penalizes inconsistencies when projecting local beliefs onto their overlaps.

Spectral Regularization

The Algebraic Latent Projection (Section 3.3) requires computing $`(\mathbf{I} - \mathbf{W}_{HH})^{-1}`$ via the Neumann series:

\begin{equation}
(\mathbf{I} - \mathbf{W}_{HH})^{-1} = \sum_{k=0}^{\infty} \mathbf{W}_{HH}^k
\label{eq:neumann-series}
\end{equation}

This series converges if and only if the spectral radius $`\rho(\mathbf{W}_{HH}) < 1`$. To enforce this condition during optimization, we impose a spectral penalty.

\begin{equation}
\mathcal{L}_{\text{spec}}(\mathbf{W}) = \max(0, \rho(\mathbf{W}) - 1 + \delta)^2
\label{eq:spectral-exact}
\end{equation}

Computational Approximation.

Computing $`\rho(\mathbf{W})`$ via eigenvalue decomposition is expensive ($`O(n^3)`$) and can produce unstable gradients. We use the Frobenius norm as a differentiable upper bound:

\begin{equation}
\mathcal{L}_{\text{spec}}(\mathbf{W}) = \max(0, \left\|\mathbf{W}\right\|_F - (1 - \delta))^2
\label{eq:spectral}
\end{equation}

This is valid because $`\left\|\mathbf{W}\right\|_F = \sqrt{\sum_{ij} w_{ij}^2} \geq \sigma_{\max}(\mathbf{W}) \geq \rho(\mathbf{W})`$, providing a conservative (over-penalizing) but differentiable bound.

Why This Matters.

Without spectral regularization, $`\rho(\mathbf{W}_{HH})`$ can approach 1 during optimization, causing: (1) numerical overflow in absorption matrix computation, (2) gradient explosion preventing convergence, and (3) invalid ADMG representations violating acyclicity among hidden variables.

Acyclicity Constraint

We enforce acyclicity using the NOTEARS constraint :

\begin{equation}
h(\mathbf{W}) = \mathop{\mathrm{tr}}(e^{\mathbf{W}\circ \mathbf{W}}) - n = 0
\label{eq:notears}
\end{equation}

where $`\circ`$ denotes element-wise product. This continuous relaxation equals zero if and only if $`\mathbf{W}`$ encodes a DAG.

Natural Gradient Descent

Standard gradient descent on the belief parameters $`\theta = (\mathbf{W}, \mathbf{L})`$ ignores the geometry of the parameter space. We employ natural gradient descent , which uses the Fisher Information Matrix as a Riemannian metric.

Fisher Metric from Gibbs Measure.

For the Gibbs measure $`P(y|\theta)`$ defined in Eq. [eq:gibbs], the Fisher Information Matrix is:

\begin{equation}
\mathbf{G}(\theta) = \mathbb{E}_{y \sim P(\cdot|\theta)}\left[(\nabla_\theta \log P(y|\theta))(\nabla_\theta \log P(y|\theta))^\top\right]
\label{eq:fisher-exact}
\end{equation}

Expanding the gradient of the log-probability: $`\nabla_\theta \log P(y|\theta) = -\beta \nabla_\theta \mathcal{E}_{\text{sem}}(\theta, y) - \nabla_\theta \log Z(\theta)`$. Assuming quasi-static dynamics where $`Z`$ varies slowly, we approximate:

\begin{equation}
\mathbf{G}(\theta) \approx \beta^2 \, \mathbb{E}_y\left[(\nabla_\theta \mathcal{E}_{\text{sem}})(\nabla_\theta \mathcal{E}_{\text{sem}})^\top\right]
\label{eq:fisher-approx}
\end{equation}

Tikhonov Regularization for Unidentifiable Regions.

The Fisher matrix becomes singular in regions where causal effects are unidentifiable. We apply Tikhonov damping:

\begin{equation}
\mathbf{G}_{\text{reg}}(\theta) = \mathbf{G}(\theta) + \lambda_{\text{reg}} \mathbf{I}
\label{eq:fisher-reg}
\end{equation}

with $`\lambda_{\text{reg}} = 10^{-4}`$. This ensures $`\mathbf{G}_{\text{reg}}`$ remains invertible, allowing Natural Gradient Descent to traverse unidentifiable regions smoothly—a critical property when latent confounders render certain edges non-identifiable.

Natural Gradient Update Rule.

The update equation is:

\begin{equation}
\theta_{t+1} = \theta_t - \eta \cdot \mathbf{G}_{\text{reg}}(\theta_t)^{-1} \nabla_\theta \mathcal{L}
\label{eq:natural-grad}
\end{equation}

Diagonal Approximation.

For computational efficiency with $`O(n^2)`$ parameters, we use a diagonal approximation:

\begin{equation}
\mathbf{G}_{\text{diag}} = \text{diag}\left(\mathbb{E}\left[(\nabla \mathcal{E}_{\text{sem}})^2\right]\right) + \lambda_{\text{reg}} \mathbf{I}
\label{eq:fisher-diag}
\end{equation}

updated via exponential moving average, reducing storage from $`O(D^2)`$ to $`O(D)`$.

Total Loss Function

The complete objective combines all components:

\begin{equation}
\mathcal{L} = \mathcal{L}_{\text{sem}} + \lambda_d \mathcal{L}_{\text{descent}} + \lambda_a h(\mathbf{W}) + \lambda_s \mathcal{L}_{\text{spec}}
\label{eq:total-loss}
\end{equation}

where $`\mathcal{L}_{\text{sem}}`$ is the semantic energy between LLM embeddings and graph structure, and $`\lambda_d = 1.0`$, $`\lambda_a = 1.0`$, $`\lambda_s = 0.1`$ are balancing weights.

Active Query Selection via Expected Free Energy

To efficiently utilize LLM queries, we employ an active learning strategy based on Expected Free Energy (EFE) from active inference :

\begin{equation}
G(a) = \underbrace{\mathbb{E}_{q(s'|a)}[\text{KL}[q(o|s')\|p(o)]]}_{\text{Epistemic Value}} + \underbrace{\mathbb{E}_{q(o|a)}[\log q(o|a)]}_{\text{Instrumental Value}}
\label{eq:efe}
\end{equation}

For each candidate query about edge $`(i,j)`$:

• Epistemic value: Uncertainty in current edge belief, measured by proximity to decision boundary: $`u_{ij} = 1 - 2|w_{ij} - 0.5|`$

• Instrumental value: Expected impact on descent loss reduction

Queries are selected to minimize EFE, prioritizing high-uncertainty edges with potential to resolve descent conflicts.

Sheaf Axiom Verification

We verify four presheaf axioms empirically:

1. Identity: $`\rho_{UU} = \text{id}_U`$ (projection onto self is identity)

2. Transitivity: $`\rho_{ZU} = \rho_{ZV} \circ \rho_{VU}`$ for $`Z \subset V \subset U`$

3. Locality: Sections over $`U`$ are determined by restrictions to an open cover

4. Gluing: Compatible local sections glue to a unique global section

Section 4.5 presents empirical results showing Identity, Transitivity, and Gluing pass to numerical precision, while Locality systematically fails for latent projections.

Experiments

We evaluate Holograph on synthetic and real-world causal discovery benchmarks, with particular focus on sheaf axiom verification and ablation studies.

Experimental Setup

Datasets.

We evaluate on five dataset types:

• ER (Erdős-Rényi): Random graphs with edge probability $`p \in \{0.15, 0.2\}`$

• SF (Scale-Free): Barabási-Albert preferential attachment with average degree 2.0

• Asia: Pearl’s epidemiology network  with 8 semantically meaningful variables (e.g., Tuberculosis, Smoking, Lung_Cancer)

• Sachs: Real-world protein signaling network with 11 variables

• Latent: Synthetic graphs with hidden confounders (3–8 latent variables)

Baselines.

We compare against ablated versions of Holograph:

• A1: Standard SGD instead of Natural Gradient

• A2: Without Frobenius descent loss ($`\lambda_d = 0`$)

• A3: Without spectral regularization ($`\lambda_s = 0`$)

• A4: Random queries instead of EFE-based selection

• A5: Fast model (thinking-off) instead of primary reasoning model

• A6: Pure optimization without LLM guidance

Metrics.

• SHD (Structural Hamming Distance): Number of edge additions/deletions/reversals

• F1: Harmonic mean of precision and recall

• SID (Structural Intervention Distance): Interventional disagreement count

Infrastructure.

All experiments run on NVIDIA V100 GPUs via SLURM on the IZAR cluster. LLM queries use DeepSeek-V3.2-Exp with thinking enabled via SGLang gateway. Each configuration runs with 5 random seeds (42–46).

Main Results

Table 1 presents benchmark results comparing Holograph against NOTEARS . Critically, this comparison reveals the gap between data-driven discovery (NOTEARS uses 1000 observational samples) and knowledge-driven discovery (Holograph uses only LLM priors without data).

Interpretation.

As expected, NOTEARS with access to abundant observational data ($`N=1000`$) substantially outperforms Holograph’s zero-shot approach on most benchmarks. However, the key insight emerges from the Asia dataset (highlighted row): Holograph achieves F1 = 0.67 without any data, purely from LLM semantic priors. This demonstrates that for semantically rich domains, LLM knowledge can substitute for observational data.

The key findings are:

1. Semantic domains enable strong priors: On Asia (epidemiology with meaningful variable names like Tuberculosis, Smoking), Holograph recovers 67% F1 zero-shot—over 3$`\times`$ higher than on Sachs (20% F1). This gap reflects the quality of LLM domain knowledge.

2. Synthetic graphs lack semantic signal: On ER/SF graphs with arbitrary variable names (X0, X1, …), LLM priors provide minimal guidance (F1 $`< 0.1`$). This is expected—LLMs have no domain knowledge for anonymous variables.

3. Technical domains are harder: Sachs uses protein names (e.g., Raf, Mek, Erk) that require specialized biochemistry knowledge, resulting in weaker LLM priors compared to general epidemiology concepts.

4. Sheaf coherence ensures consistency: The presheaf descent framework unifies potentially contradictory LLM responses into globally consistent structures.

Threshold Calibration.

Due to the spectral radius constraint ($`\rho(\mathbf{W}) < 1`$) required for Neumann series convergence in the Algebraic Latent Projection, learned edge weights are compressed relative to ground truth. We use a calibrated threshold $`\tau = 0.05`$ (rather than the ground truth generation threshold of 0.3) to ensure fair structural evaluation. See Appendix 6.7 for sensitivity analysis.

Sample Efficiency: The Low-Data Advantage

A critical question emerges: at what sample size does data-driven discovery match LLM-based discovery? We investigate this crossover point on the Asia dataset, where Holograph achieves strong zero-shot performance (F1 = 0.67).

Table 2 reveals a striking pattern:

1. Extreme low-data regime ($`N \le 10`$): Holograph dramatically outperforms NOTEARS. At $`N=5`$ samples, the improvement is +91% relative F1—statistical methods fundamentally cannot learn structure from so few observations.

2. Crossover at $`N \approx 15`$–$`20`$: Below this threshold, LLM priors dominate; above it, data-driven methods rapidly improve and eventually surpass zero-shot performance.

3. Sample invariance: Holograph’s F1 is constant across all $`N`$ (as expected for a zero-shot method), providing a floor guarantee regardless of data availability.

Practical Implication.

These results establish a clear decision boundary: when $`N < 20`$ samples are available for a semantically rich domain, Holograph’s zero-shot approach is preferable to training NOTEARS on insufficient data.

Hybrid Synergy: LLM Priors as Regularization

Can LLM priors complement rather than replace statistical methods? We test a hybrid approach: use Holograph’s learned adjacency matrix to regularize NOTEARS optimization. Specifically, we apply confidence filtering—only edges with $`|W_{ij}| > 0.3`$ in the Holograph prior contribute to regularization.

Table 3 demonstrates substantial synergy in the low-data regime:

1. Maximum benefit at $`N=20`$: The hybrid method achieves +13.6% F1 improvement (0.71 $`\to`$ 0.80), with the Holograph prior providing regularization that prevents overfitting to limited samples.

2. Complementary strengths: At $`N=10`$, vanilla NOTEARS achieves only F1 = 0.56 due to overfitting, while the hybrid recovers 0.61—the LLM prior acts as an inductive bias toward semantically plausible structures.

3. Diminishing returns: At $`N=50`$, the improvement shrinks to +1.3% as statistical evidence dominates. The prior becomes less necessary when data is abundant.

Mechanism of Improvement.

The confidence filtering threshold ($`|W| > 0.3`$) ensures only high-confidence Holograph edges contribute to regularization. This prevents noisy LLM beliefs from corrupting the optimization while preserving strong semantic signals.

Sheaf Axiom Verification

Table 4 presents results from sheaf exactness experiments (X1–X4).

Key Findings.

1. Identity and Transitivity: Both axioms pass with errors $`< 10^{-6}`$ across all graph sizes, confirming mathematically correct implementation of the Algebraic Latent Projection. This validates the cross-term inclusion in Eq. [eq:m-proj] (see Remark 6 and Appendix 6.6 for implementation verification).

2. Gluing: The gluing axiom (compatible local sections yield unique global section) passes uniformly, validating the Frobenius descent loss formulation.

3. Locality Failure as Discovery: The locality axiom systematically fails with errors scaling approximately as $`\mathcal{O}(\sqrt{n})`$ with graph size.

   Interpretation: This is not an implementation bug, but a fundamental property of ADMGs with latent confounders. Latent variables create non-local correlations: knowledge about variable subset $`A`$ constrains beliefs about distant subset $`B`$ through hidden mediators, violating the principle that “local data determines local structure.”

Significance of Locality Failure.

This finding demonstrates that the presheaf of ADMGs under algebraic latent projection does not form a classical sheaf. The failure quantitatively measures the “non-sheafness” introduced by latent confounding—a property that could serve as a diagnostic for the necessity of latent variable modeling.

Ablation Studies

Table 5 compares ablation variants on ER-50 and Sachs using F1 score.

Key Findings.

The ablation results reveal nuanced trade-offs:

1. Spectral regularization trades off with F1: Removing spectral regularization (A3) increases F1 on ER-50 (0.108 vs 0.052), but at the cost of numerical stability. This suggests the strict $`\rho(\mathbf{W}) < 0.9`$ constraint may be overly conservative.

2. LLM guidance helps on real data: On Sachs, variants with LLM guidance (Full, A1–A3) outperform those without (A4, A6), confirming the value of domain knowledge for real-world networks.

3. Active query selection matters: A4 (random queries) and A6 (no LLM) show similar performance, suggesting that EFE-based query selection effectively prioritizes informative edges.

4. Fast model performs surprisingly well: A5 (thinking-off) achieves the highest F1 on Sachs (0.269), suggesting that for well-known domains, simple LLM responses may suffice without extended reasoning.

Interpretation.

The ablation results highlight a key insight: the full Holograph configuration prioritizes numerical stability (via spectral regularization) and theoretical coherence (via Natural Gradient and descent loss) over raw F1 performance. Removing these constraints can improve F1 but may produce unstable or incoherent causal graphs. The choice depends on downstream requirements.

Hidden Confounder Experiments

Table [tab:latent] presents results on graphs with hidden confounders (E3). These experiments test Holograph’s ability to recover structure in the presence of latent variables using the Algebraic Latent Projection.

The 50-observed/8-latent configuration shows high variance in runtime, reflecting the stochastic nature of LLM-guided optimization. Increasing latent variables proportionally increases structural error, confirming the fundamental difficulty of latent confounder identification.

Rashomon Stress Test

The Rashomon experiment (E5) tests contradiction detection and resolution under latent confounding. With 30 observed and 5 latent variables, Holograph achieves:

• SHD: $`89.8 \pm 5.7`$

• 100 queries utilized (budget exhausted)

• Final loss: $`1.6 \times 10^{-4}`$

The system correctly identifies topological obstructions when descent loss plateaus, triggering latent variable proposals. However, resolution rates remain below target ($`<70\%`$), indicating room for improvement in latent variable initialization strategies.

Conclusion

We presented Holograph, a sheaf-theoretic framework for LLM-guided causal discovery. By formalizing local causal beliefs as presheaf sections and global consistency as descent conditions, we provide principled foundations for integrating LLM knowledge into structure learning.

Our key contributions include:

• The Algebraic Latent Projection for handling hidden confounders

• Natural gradient descent with Tikhonov regularization for optimization

• EFE-based active query selection for efficient LLM utilization

• Comprehensive sheaf axiom verification revealing fundamental locality failures

The systematic failure of the Locality axiom is perhaps our most significant finding. It demonstrates that the presheaf of ADMGs does not form a classical sheaf when latent variables induce non-local coupling. This provides a formal measure of the “non-sheafness” inherent in causal models with hidden confounders—a quantity that could guide future algorithms in detecting latent variable necessity.

Limitations.

• Scalability: Performance on graphs with $`n > 100`$ variables degrades due to $`O(n^3)`$ projection costs. Sparse approximations may help.

• LLM Reliability: Current approach assumes LLM responses are locally consistent. Adversarially contradictory LLMs could violate this assumption.

• Identifiability: As with all causal discovery methods, we can only recover structure up to Markov equivalence without interventional data.

Future Work.

Promising directions include:

1. Cohomological Measures: Develop sheaf cohomology metrics to quantify Locality violations, potentially using $`\check{\text{C}}`$ech cohomology.

2. Hybrid Methods: Combine Holograph with constraint-based algorithms (e.g., FCI) to leverage both continuous optimization and discrete constraint propagation.

3. Interventional Extensions: Extend to experimental design settings where interventions can be performed, potentially enabling full causal identification.

Speculative Connections.

We note a suggestive parallel between our Locality failure and quantum non-locality. In quantum mechanics, entangled systems violate Bell inequalities through correlations that resist local hidden variable explanations. Similarly, ADMGs with latent confounders exhibit correlations between distant variables that cannot be explained by local restrictions. The scaling $`\text{Error} \propto \sqrt{n}`$ in both settings hints at deeper mathematical connections—a direction for future theoretical exploration.

Appendix

Hyperparameters and Configuration

Table [tab:hyperparams] lists all hyperparameters used in experiments. Values are sourced from experiments/config/constants.py.

Infrastructure Details

Cluster.

Experiments ran on the IZAR cluster at EPFL/SCITAS with:

• GPU: NVIDIA Tesla V100 (32GB HBM2)

• CPU: Intel Xeon Gold 6140 (18 cores per node)

• Memory: 192GB RAM per node

• Scheduler: SLURM with array jobs for parallelization

Runtime Statistics.

• Small experiments (n=20, Sachs): $`<1`$ second

• Medium experiments (n=50, ER/SF): $`\sim`$30 seconds

• Large latent experiments (n=50+8): 30–60 minutes

• Total GPU hours: $`\sim`$50 hours across 160 experiments

LLM Gateway.

We use SGLang to provide a unified OpenAI-compatible API:

• Primary model: DeepSeek-V3.2-Exp (thinking-on)

• Endpoint: Custom gateway at port 10000

• Rate limiting: Handled by query budget enforcement

Sheaf Axiom Definitions

For completeness, we formally state the four presheaf axioms tested.

\rho_{UU} = \text{id}_{\mathcal{F}(U)}

\rho_{ZU} = \rho_{ZV} \circ \rho_{VU}

Proof of Absorption Matrix Formula

\mathbf{W}_{\text{total}} = \mathbf{W}_{OO} + \mathbf{W}_{OH}(\mathbf{I} - \mathbf{W}_{HH})^{-1}\mathbf{W}_{HO}
```*

</div>

<div class="proof">

*Proof.* Consider a path from observed variable $`X_i`$ to observed
variable $`X_j`$ passing through hidden variables. The direct effect is
$`\mathbf{W}_{OO}[i,j]`$. Paths through exactly one hidden variable
contribute $`\sum_h \mathbf{W}_{OH}[i,h] \mathbf{W}_{HO}[h,j]`$. Paths
through $`k`$ hidden variables contribute
$`(\mathbf{W}_{OH} \mathbf{W}_{HH}^{k-1} \mathbf{W}_{HO})[i,j]`$.

Summing all path lengths:
``` math
\begin{align*}
\mathbf{W}_{\text{total}} &= \mathbf{W}_{OO} + \sum_{k=1}^{\infty} \mathbf{W}_{OH} \mathbf{W}_{HH}^{k-1} \mathbf{W}_{HO} \\
&= \mathbf{W}_{OO} + \mathbf{W}_{OH} \left(\sum_{k=0}^{\infty} \mathbf{W}_{HH}^k\right) \mathbf{W}_{HO} \\
&= \mathbf{W}_{OO} + \mathbf{W}_{OH} (\mathbf{I} - \mathbf{W}_{HH})^{-1} \mathbf{W}_{HO}
\end{align*}

Additional Experimental Results

Full Sheaf Axiom Error Statistics

Table 6 provides detailed error statistics for all X experiments.

Convergence Plots

Loss curves show rapid initial descent followed by plateau behavior, consistent with the NOTEARS objective landscape. Natural gradient variants (full Holograph) converge faster and reach lower final loss than SGD ablations.

Query Distribution Analysis

Across all experiments, the query type distribution was:

• Edge existence: 45%

• Direction: 25%

• Mechanism: 20%

• Confounder: 10%

EFE-based selection preferentially queries uncertain edges near decision boundaries, as expected from the epistemic value formulation.

Identification Frontier Analysis

The identification frontier represents the set of queries that can yield identifiable causal effects given the current ADMG state. Figure 1 compares the frontier sizes across methods.

Analysis.

The identification frontier advantage of Holograph stems from two sources:

1. ADMG vs DAG representation: By explicitly modeling bidirected edges for latent confounders, Holograph can identify effects that remain confounded under DAG assumptions. On ER (n=50), this yields 82 identifiable queries vs. 45 for NOTEARS ($`\sim`$82% improvement).

2. EFE-based query selection: The Expected Free Energy criterion prioritizes queries that maximize information gain about the true graph, leading to more efficient exploration of the identification frontier.

The Sachs dataset shows the largest relative improvement (180% vs. NOTEARS) because the protein signaling network contains multiple known confounding pathways that cannot be represented in a DAG without introducing spurious edges.

Mathematical Implementation Verification

To ensure the implementation faithfully realizes the mathematical specification, we conducted a comprehensive audit comparing 15 core formulas against the codebase.

Core Formula Verification

Table [tab:verification] lists all verified formulas with their code locations.

Numerical Stability Verification

All implementations include the following stability measures:

1. Stable Matrix Inversion: Uses torch.linalg.solve instead of explicit inv() for $`(\mathbf{I} - \mathbf{W}_{HH})^{-1}`$ computation.

2. Regularization: Adds $`\epsilon \mathbf{I}`$ ($`\epsilon = 10^{-6}`$) to near-singular matrices before inversion.

3. Pseudoinverse Fallback: Switches to SVD-based pseudoinverse if standard solver fails.

4. Spectral Enforcement: Continuously penalizes $`\rho(\mathbf{W}) > 0.9`$ during training.

5. PSD Guarantee: Parametrizes $`\mathbf{M}= \mathbf{L}\mathbf{L}^\top`$ with lower-triangular $`\mathbf{L}`$ to ensure positive semi-definiteness.

Cross-Term Necessity Verification

Ablation experiments confirm that removing cross-terms $`\mathbf{M}_{OH}\mathbf{A}^\top + \mathbf{A}\mathbf{M}_{HO}`$ from Eq. [eq:m-proj] increases Transitivity error from $`< 10^{-6}`$ to $`> 0.1`$, validating their necessity for presheaf composition:

\rho_{ZU} = \rho_{ZV} \circ \rho_{VU}

Dual Implementation Consistency

The project maintains two implementations (src/holograph/ and holograph/). Both pass identical unit tests and produce numerically equivalent results (difference $`< 10^{-8}`$) on shared test cases, confirming implementation consistency across the codebase.

Threshold Sensitivity Analysis

The discretization threshold $`\tau`$ converts continuous edge weights to binary adjacency matrices for evaluation. Table 7 shows how F1 varies with $`\tau`$ for the full Holograph model on ER-50.

Key Observations.

1. Ground Truth Scale Mismatch: Ground truth edges are generated with weights in $`[0.3, 1.0]`$, but Holograph’s learned weights are compressed to $`[-0.12, 0.12]`$ due to spectral regularization.

2. Optimal Threshold: F1 peaks around $`\tau = 0.01`$–$`0.02`$ where the trade-off between true positives and false positives is balanced.

3. Threshold Choice Justification: We use $`\tau = 0.05`$ as a conservative choice that avoids excessive false positives while maintaining non-zero recall.

Weight Compression Analysis.

The spectral regularization constraint $`\|\mathbf{W}\|_F < 0.9`$ limits the magnitude of learned weights. For an $`n \times n`$ matrix with $`k`$ non-zero entries of equal magnitude $`w`$, we have $`\|\mathbf{W}\|_F = w\sqrt{k} < 0.9`$. With $`n=50`$ and expected $`k \approx 184`$ edges, this implies $`w < 0.9/\sqrt{184} \approx 0.066`$. This theoretical bound aligns with observed maximum weights of $`\approx 0.12`$.

Hybrid Method Limitations

While Section 4.4 demonstrates the effectiveness of hybrid LLM-NOTEARS integration on the Asia dataset, this approach has important limitations that practitioners should consider.

Prior Quality Dependency

The hybrid method’s effectiveness depends critically on the quality of the Holograph prior. Table 8 shows results on the Sachs protein signaling network, where Holograph achieves only F1 = 0.35 (compared to 0.67 on Asia).

Analysis.

On Sachs, the hybrid method consistently underperforms vanilla NOTEARS:

1. Weak prior hurts: With Holograph F1 = 0.35, the LLM prior contains significant errors. Using this as regularization biases NOTEARS toward incorrect edges.

2. Higher variance: The hybrid shows std = 0.08–0.11 vs. 0.02–0.06 for vanilla, indicating unstable optimization when conflicting signals (data vs. prior) compete.

3. Negative transfer: At $`N=1000`$, the performance gap widens to $`-13.8\%`$—more data makes NOTEARS more confident in correct structure, but the fixed prior continues to pull toward errors.

Domain Knowledge Requirements

The contrast between Asia (F1 gain = +13.6%) and Sachs (F1 loss = $`-8.3\%`$) illustrates a critical insight: hybrid methods require that the LLM has genuine domain expertise.

• Asia (epidemiology): Variables like Tuberculosis, Smoking, and Lung_Cancer have well-documented causal relationships in medical literature. LLMs trained on web corpora encode this knowledge accurately.

• Sachs (protein signaling): Variables like Raf, Mek, and PKC are specialized biochemistry concepts. Their causal relationships require domain expertise that general LLMs lack.

Recommendations for Practitioners

Based on these findings, we recommend the following workflow:

1. Assess prior quality first: Run Holograph zero-shot and evaluate against any available ground truth or domain expertise. If F1 $`< 0.5`$, the hybrid approach is unlikely to help.

2. Use confidence filtering: Only include high-confidence edges ($`|W| > 0.3`$) in the prior to avoid noise amplification.

3. Consider sample size: The hybrid is most beneficial when $`N < 50`$ and the prior is strong. With abundant data, let NOTEARS learn from observations alone.

4. Validate on held-out data: If possible, use a validation set to detect negative transfer early and fall back to vanilla NOTEARS.