vLLM: Session-Aware Agentic Routing cuts model switches by 79 percent

The vLLM project has introduced Session-Aware Agentic Routing (SAAR), a new capability of its Semantic Router. SAAR makes model-selection decisions by taking into account the entire flow of an agentic conversation, instead of treating each message as an independent event.

What does SAAR actually solve?

Previous model routing could select a different model for each message, which leads to unnecessary and potentially unsafe switches in long-running agentic conversations. SAAR shifts control from the individual message to the level of the entire session, preserving conversational coherence. The mechanism distinguishes forbidden switches, for example during an active tool loop, from permitted ones, which happen after idle time or a change of task.

What are the test results?

Across 21,600 deterministic turns, spread over five seeds, 40 sessions per seed, and 18 turns per session, SAAR reduced the number of model switches by 79.29 percent, from 9,709 to 2,011. Unsafe switches, of which there were 3,836, were reduced to zero. Estimated costs fell by 78.71 percent. In real-world use across 2,896 requests on AMD ROCm, zero continuity violations were recorded.

What is SAAR made of?

SAAR adds five components around the existing routing process. Router Memory tracks the last used model, phase, and cache evidence, separately from the application’s own memory. Hard Locks prevent switching during active tool loops and non-transferable provider-managed state. Reset Boundaries reopen model selection after idle time or a change of decision. Switch Economics factor in the cost of losing the prefix cache, making switches asymmetric across model tiers, while Replay Traces log routing decisions for easier monitoring and debugging.

Why does a session-based approach matter?

Agentic systems increasingly run long, multi-step conversations in which a model calls tools and builds context across many turns. Treating each message separately can break that context and introduce inconsistencies, especially when the model changes mid-task. SAAR’s shift toward session-level coherence solves exactly this problem, delivering measurable cost savings and greater reliability of agentic flows at the same time.