arXiv: Arbor — tree search as a cognition layer for autonomous agents

Researchers published on June 10, 2026, a paper on arXiv introducing Arbor — a multi-agent framework that repositions structured tree search as the fundamental cognition layer for autonomous agents. The paper’s authors are Neha Prakriya, Chaojun Hou, Zheng Gong, Huasha Zhao, Xi Zhao, Mou Li, Zhenyu Gu, and Emad Barsoum.

Why were previous agents cognitively blind?

Previous approaches to autonomous agents share a common structural flaw: each agent evaluates individual goals in isolation and without state (stateless). When an agent finishes its evaluation, the context disappears — the next agent starts from scratch. In expansive, stateful action spaces, such as those typical of optimizing an entire inference stack, this approach quickly hits its limits.

Arbor responds to this problem by introducing an explicit search tree of scored hypotheses that functions as the shared working memory of all agents in the system. Instead of each agent working in a vacuum, the entire team of agents reads from and writes to the same structured representation of the problem state.

Architecture: two roles, clear division of responsibility

Arbor organizes agents into two coordinated roles:

The Orchestrator agent assumes the role of strategic coordinator. It drives the optimization process by delegating specific tasks through layers of the inference stack — from the application layer, through frameworks, compiler, kernel, all the way to hardware. The Orchestrator does not resolve technical details itself; its value lies in coordination and delegation.

The Critic agent assumes the role of stabilizer. Every proposal or change passes through the Critic’s root-cause analysis and validation. Without the Critic, the system is unstable — something the authors confirmed empirically.

Agent capabilities are divided into hard skills — domain expertise for specific layers of the stack — and soft skills — coordination protocols that enable multi-day autonomous optimization campaigns.

The role of the Critic in system stability

The authors conducted an ablation study that clearly demonstrates the importance of this architectural decision. A single agent without a Critic achieved only +33% throughput improvement — and then crashed. A system without the stabilizing role is incapable of sustained campaigns.

In contrast, the full Arbor system with Orchestrator and Critic achieves up to 193% improvement on the throughput-latency Pareto frontier compared to vendor-optimized baseline systems. This result is not hardware-specific — the authors confirm reproducibility across multiple hardware generations, and run-to-run variance remains within 2 percentage points.

Tree search as a general cognitive primitive

The authors put forward a broader thesis: a structured search tree is not just a tool for LLM inference optimization. They position it as a general cognition layer applicable to any complex action space where agents must coordinately explore, score, and discard hypotheses.

This generalization has practical importance. Today’s agentic systems often only solve well-defined, relatively shallow problems. Arbor suggests that adding an explicit search-tree memory — regardless of the domain — can dramatically extend the reach of autonomous agents.

Application: the full LLM inference stack

The concrete application described in the paper is the optimization of the full-stack LLM inference — a multi-layer problem spanning application code, frameworks like PyTorch, compiler pipelines, CUDA kernels, and the hardware itself. This is a space where no single expert can cover all layers simultaneously, and the combinatorial space of possible configurations is enormous.

Arbor solves this problem precisely through coordinated division of domain knowledge among agents, with the search tree as the shared coordination mechanism. The results confirm that such an approach not only works — it dramatically outperforms what any single vendor-optimized system could achieve.

The paper is available on arXiv under identifier 2606.12563.