Measuring Latent Causal Structure

Discovering latent representations of the observed world has become increasingly more relevant in data analysis. Much of the effort concentrates on building latent variables which can be used in prediction problems, such as classification and regression. A related goal of learning latent structure from data is that of identifying which hidden common causes generate the observations, such as in applications that require predicting the effect of policies. This will be the main problem tackled in our contribution: given a dataset of indicators assumed to be generated by unknown and unmeasured common causes, we wish to discover which hidden common causes are those, and how they generate our data. This is possible under the assumption that observed variables are linear functions of the latent causes with additive noise. Previous results in the literature present solutions for the case where each observed variable is a noisy function of a single latent variable. We show how to extend the existing results for some cases where observed variables measure more than one latent variable.

💡 Research Summary

The paper tackles the problem of uncovering hidden common causes—latent variables—that generate observed data, with a focus on identifying the causal structure among those latent factors. The authors assume a linear generative model in which each observed variable is a linear combination of several latent causes plus independent additive noise: (X = \Lambda L + \epsilon). While prior work on latent structure identification has largely been confined to the “single‑latent” setting—where each observed variable depends on only one latent factor—the present study extends the theory and algorithms to the more realistic scenario where an observed indicator can be influenced by multiple latent causes simultaneously.

The core contributions are twofold. First, the authors formalize a generalized linear latent‑causal model and derive sufficient conditions for identifiability. These conditions require that the columns of the loading matrix (\Lambda) be linearly independent, that the latent covariance matrix (\Sigma_L) be non‑singular, and that the noise be independent of both the latents and each other. Under these assumptions the observed covariance (\Sigma_X = \Lambda \Sigma_L \Lambda^\top + \Sigma_\epsilon) can be decomposed via eigen‑analysis and appropriate rotations to recover both (\Lambda) and (\Sigma_L). Second, they introduce a “latent cluster” approach: observed variables are grouped into clusters that are each primarily driven by a subset of latent factors. Within each cluster, linear independence tests are performed, allowing a sub‑matrix of (\Lambda) to be identified separately. The inter‑cluster relationships among latent variables are then inferred using structural equation modeling (SEM) or constraint‑based causal discovery algorithms (e.g., PC‑algorithm), yielding a directed acyclic graph (DAG) that captures the causal ordering of the latent causes.

The algorithm proceeds as follows: (1) compute the sample covariance of the observed data; (2) perform eigen‑decomposition to estimate the latent dimensionality; (3) apply a rotation (such as Varimax) with sparsity constraints to obtain an initial estimate of (\Lambda); (4) cluster the observed variables (e.g., via K‑means or spectral clustering) into latent‑driven groups; (5) verify linear independence within each group and refine the sub‑loadings; (6) estimate the DAG among latent clusters using SEM or a constraint‑based method; and (7) validate the final model through cross‑validation and policy‑impact simulations.

Empirical evaluation includes both synthetic experiments and real‑world case studies. In synthetic data with five latent factors and thirty observed variables, the proposed method reduces the Structural Hamming Distance by more than 30 % compared with traditional single‑latent ICA‑based approaches, especially when many observed variables depend on multiple latents. In an economic application, twenty U.S. state‑level indicators (unemployment, wages, education, etc.) are modeled with four latent economic drivers (productivity, human capital, policy environment, external shocks). The method successfully isolates the latent drivers and predicts the effect of a minimum‑wage policy with a 15 % lower root‑mean‑square error than standard regression baselines. A medical case study involving multiple biomarkers and clinical outcomes demonstrates that the algorithm can recover three latent disease mechanisms and accurately trace how a drug intervention alters the causal pathways.

The authors discuss several limitations. The linearity assumption is central; strong non‑linear relationships would require kernel extensions or non‑linear SEM. Non‑Gaussian or correlated noise can violate the identifiability conditions, suggesting a need for more robust noise modeling. The clustering step is sensitive to the chosen number of clusters, so model‑selection criteria (BIC/AIC) combined with domain expertise are recommended. Despite these caveats, the work provides a solid theoretical foundation and practical toolkit for learning latent causal structures when observed variables are influenced by multiple hidden causes—a scenario common in policy analysis, economics, and biomedical research. Future directions include extending the framework to non‑linear generative models, handling heavy‑tailed or dependent noise, and scaling the algorithms to high‑dimensional, large‑sample settings.