vLLM: open-source inference engine takes first place on the Artificial Analysis leaderboard

vLLM, the open-source inference engine, has claimed first place on the Artificial Analysis leaderboard for three frontier models through targeted optimizations. The development team confirmed that vLLM now leads for DeepSeek V3.2, MiniMax-M2.5, and Qwen 3.5 397B — the result of three distinct approaches, one per model.

DeepSeek V3.2: aggressive kernel fusion

On DeepSeek V3.2, vLLM achieves 230 tokens per second output throughput — as the announcement states, “more than 4× what most inference providers report.” The key is aggressive kernel fusion that merges normalization, rotary embedding, and quantization operations. The number of GPU kernel launches was reduced from approximately 33 to just 10 per layer, eliminating launch overhead at small batch sizes and delivering a 1.28× speedup at batch size 1.

MiniMax-M2.5: custom EAGLE3 draft model

For MiniMax-M2.5, vLLM achieves 326 tokens/s at concurrency 1 using a custom EAGLE3 speculative decoding setup. EAGLE3 is a technique where a smaller “draft” model predicts several tokens ahead, which the large model then verifies in a single pass. vLLM engineers trained a specialized draft model via TorchSpec, allowing it to learn on the actual hidden states that vLLM produces — rather than on generic datasets.

Qwen 3.5 397B: attention path fusion

Qwen 3.5 397B ranks first among all 12 measured providers, with sub-second TTFT (time-to-first-token) on long prompts. The optimizations targeted the model’s specific linear-attention architecture and its normalization patterns, yielding “up to 6.69 requests/s at concurrency 256” compared to baseline.

What this means for the open-source ecosystem

The result is significant: vLLM, which anyone can run on their own hardware, leads production benchmarks for three frontier models. For organizations running self-hosted inference — for privacy, data sovereignty, or cost predictability — this is proof that an open stack no longer pays a structural performance penalty against proprietary services.