ArXiv: the hidden cost of tools in LLM agents — 'tool-use tax' reduces accuracy even when tools help

A team of researchers (Kaituo Zhang, Zhen Xiong, Mingyu Zhong, Zhimeng Jiang, Zhouyuan Yuan, Zhecheng Li, Ying Lin) published on April 30, 2026 a paper that challenges a widespread assumption: does calling tools (tool use) always improve LLM agent performance?

What is the “tool-use tax”?

Tool-use tax is the term the authors introduce for the hidden cost that arises when an agent calls a tool. The cost is not tied to the tool itself — but to the calling protocol: formatting the request, parsing the response, and the overhead that comes with that process. In the presence of semantic distractors (irrelevant but superficially relevant information in the query), this overhead can cancel out the benefit the tool provides.

In short: the tool may return a correct result, but the model fails to use it correctly because the protocol interferes.

How do the researchers measure the cost?

The authors develop a Factorized Intervention Framework that isolates three separate components:

1. Prompt formatting cost — how much the tool-call format itself confuses the model
2. Tool-calling protocol overhead — how much the communication layer degrades reasoning
3. Actual gain from tool execution — what the model gains from the concrete tool result

This decomposition reveals that the tool’s benefit often does not compensate for the first two costs — meaning native chain-of-thought (CoT) sometimes outperforms an agent with tools.

How does G-STEP mitigate the problem?

The proposed solution is G-STEP (inference-time gate) — a lightweight mechanism that decides at inference time whether an agent should call a tool for a given query. This avoids unnecessary overhead when the model can answer accurately on its own.

G-STEP delivers partial performance recovery without fine-tuning the model. The authors nonetheless emphasize that a complete solution requires improving the model’s fundamental capabilities for tool interaction — not just optimizing the protocol.

Why does this matter for agent development?

Industry is already intensively building tool-augmented agents: from OpenAI function calling to Anthropic MCP and Google’s agent framework. This paper warns that the mere availability of tools does not guarantee better results — the design of the protocol and when a tool is called are equally critical. For practitioners: evaluating an agent without isolating these costs can yield falsely optimistic conclusions.