arXiv:2605.27567: Mathematically proven why LLMs cannot learn causality — and how the A-CBO agent solution bypasses the inherent limitation

Researchers Amartya Roy and Sonali Parbhoo published a preprint that mathematically proves a fundamental limitation of standard machine learning methods in the context of causal discovery — and proposes an agent solution that bypasses this inherent obstacle.

Why can neither fine-tuning nor in-context learning teach causality?

Causal discovery is the task of identifying cause-and-effect relationships from data. While LLM models show impressive capabilities in correlational tasks, their performance on causal tasks deteriorates as graph complexity grows — and the formal theoretical basis for this limitation was previously unknown.

Researchers now prove this mathematically through the Kernel Obstruction Theorem: methods such as supervised fine-tuning (SFT), direct preference optimization (DPO), and in-context learning (ICL) produce predictors that cannot distinguish causal graphs generating similar observational data. The critical conclusion: the limitation is inherent to the learning paradigm, not to any specific model or dataset.

What is A-CBO and how does it solve the kernel problem?

A-CBO (Agentic Causal Bayesian Optimization) is the proposed agent solution that restructures the problem by bypassing the kernel obstruction. The architecture relies on three components:

A frozen LLM is used as an interventional oracle that answers targeted questions about interventions — “what would happen if we changed variable X?”. The LLM is not trained for causal reasoning; instead, the LLM provides contextual estimates of intervention outcomes.

An external Bayesian optimization loop receives these estimates and systematically concentrates beliefs around candidate causal graphs, eliminating those that do not satisfy observed intervention results.

This design bypasses the kernel obstruction by moving causal graph decision-making outside the problematic representation space of the LLM, into a deterministic Bayesian framework that can converge in logarithmically many steps.

What are the experimental results of A-CBO?

Researchers tested A-CBO on two causal reasoning benchmarks. On the original Corr2Cause benchmark, A-CBO achieves results comparable to fine-tuned baseline models — without any training.

On the Extended Corr2Cause benchmark with graphs of 24 variables and a set of 18,000 test samples, A-CBO outperforms both fine-tuning and preference optimization. Performance grows with graph complexity, which is the opposite of the declining curve of standard LLM approaches.

The finding has direct implications for building AI agents that need to understand causes — from medical diagnosis to robotics and scientific discovery — where correlational inferences are insufficient for reliable decisions.