TRIAGE: How to Assign Credit to the Right Tokens in Agentic Reinforcement Learning

Every time an AI agent solves a task, it generates a trajectory — a sequence of actions, tool calls, and intermediate results. Standard reinforcement learning algorithms such as GRPO treat that sequence uniformly: if the outcome is successful, all tokens receive a positive advantage; if not, all receive a negative one. The problem is that this assumption is incorrect.

Why Uniform Advantage Creates Bad Incentives

Imagine an agent that explores three dead ends, but succeeds on the fourth attempt. GRPO rewards all four sequences equally, including those three useful explorations that helped find the solution — but also a great deal of filler code that contributes nothing. In a failed run, it will conversely penalize the exploration sequence that was on the right track.

TRIAGE (Role-Typed Credit Assignment for Agentic RL), a paper published June 30, 2026, on arXiv (2606.32017), introduces a semantic axis alongside the existing outcome signal.

Four Roles, Four Levels of Credit

An LLM judge with a fixed structure evaluates each trajectory segment and assigns it one of four roles:

1. Decisive progress — actions that directly move the agent toward the goal. Rewarded proportionally to their contribution.

2. Useful exploration — actions that do not lead directly to success but eliminate dead ends or gather information relevant to subsequent steps. Penalized in failed runs under standard GRPO; recognized as a positive contribution in TRIAGE.

3. Infrastructure without progress — necessary but neutral actions: initialization, parsing, output formatting. Neither rewarded nor penalized beyond a proportional share of the outcome.

4. Regression — actions that move the agent farther from the goal, undo previous progress, or introduce errors. Penalized even when the final outcome is successful.

Role-conditioned reward assignment follows fixed rules — it is not the product of ad hoc heuristics. The authors prove that such assignment represents an optimal segment-level correction expressible from roles, defined as the projection of the per-segment advantage residual onto the role variable.

Results on Three Benchmarks

TRIAGE was tested on ALFWorld (navigation and manipulation in a text-based household environment), Search-QA (answer retrieval through web search), and WebShop (shopping on a simulated e-commerce interface).

Key finding: on completed rollouts, TRIAGE reduces the number of environment interactions by 10.4% to 14.8% compared to GRPO, while simultaneously achieving higher success rates. An agent with the same model solves tasks in fewer steps — which in practice is equivalent to lower costs and shorter response times.

What Does the Ablation Study Show?

The authors isolated the contribution of each of the four roles. Regression detection within successful trajectories proved to be the dominant factor in improvement. This is a counterintuitive finding: what matters most is not rewarding good exploration, but penalizing bad actions even when the outcome is positive.

Crediting useful exploration delivered consistent but secondary improvements — particularly pronounced in environments like WebShop where gathering product information is key to making the correct decision.

Positioning Within the Literature

TRIAGE does not modify the target model or introduce expensive additional training — the LLM judge can be a smaller, specialized model. The outcome signal (episode success/failure) remains the primary optimizer; TRIAGE adds a process layer that redistributes that signal within the trajectory according to the semantic contribution of each segment.

For practitioners working with agents that execute multi-step tasks in expensive environments — web, code, databases — a reduction in interactions of more than 10 percent translates directly into operational savings. The paper has been available on arXiv since today.