arXiv: The illusion of multi-agent advantage — more expensive, yet worse than simple CoT-SC

Multi-agent systems are not automatically better

News shaking the AI research community: multi-agent systems (MAS) may not deserve their reputation as a superior approach after all. A new study titled “The Illusion of Multi-Agent Advantage” systematically examines the claim that coordinating multiple AI agents necessarily produces better results than single-agent approaches — and concludes that those claims do not hold, at least not for automatically generated architectures.

The authors evaluated across multiple task categories: traditional reasoning benchmarks, interactive multi-step workflows (including the BrowseComp-Plus benchmark), and a specially designed synthetic dataset that highlights situations where MAS should have a clear advantage.

What do the results show?

The finding is unexpectedly clear-cut. Automatically generated multi-agent systems consistently underperform Chain-of-Thought with Self-Consistency (CoT-SC) — and do so at costs that are up to 10 times higher. CoT-SC is an established single-agent technique in which a model generates multiple answers and selects the most consistent one by majority vote, without any complex coordination infrastructure.

The picture changes when expert-designed MAS enters the equation. On synthetic datasets specifically designed to test multi-agent capabilities, expert-designed systems outperform automatically generated architectures. But that advantage does not transfer to practical reasoning tasks — suggesting the problem lies specifically in the automated design process, not in the concept of multi-agent systems as such.

The weakness of automated design

Researchers identify a fundamental mismatch between how MAS are automatically constructed and the actual principles of multi-agent collaboration. Automated architecture design tools produce what the authors call “architectural bloat” — excess complexity that does not translate into functional benefit.

Concretely, automation often adds agent layers and coordination steps without clear functional motivation. The result is a system that looks sophisticated but does not actually exploit the parallelization, specialization, or verification mechanisms that multi-agent approaches theoretically offer.

When multi-agent systems still make sense

The study does not dispute the value of multi-agent systems as a concept — nor does it claim single-agent approaches are always better. The message is more precise: automated design of MAS architectures has not matured to the point where it can reliably outperform carefully tuned single-agent methods.

The diagnostic synthetic dataset the authors introduce can help researchers precisely identify weaknesses in individual architectures and compare systems while accounting for cost — a dimension that existing evaluation frameworks often neglect.

Implications for the industry

Multi-agent systems are currently experiencing intense popularity: numerous commercial products and research prototypes are based on automatically orchestrating multiple AI agents to solve complex tasks. This study places a clear warning: cost-to-benefit ratio and actual utility must be systematically evaluated, not assumed.

For practitioners, this means: before deploying complex multi-agent infrastructure, it is worth checking whether a well-configured single-agent approach with CoT-SC can achieve comparable or better results at lower cost. Automated design of multi-agent systems is promising, but according to this research it is not yet at the level that justifies premium costs across all use cases.