AMD: vLLM-ATOM plugin brings Instinct optimisations without changing vLLM code

What is vLLM-ATOM?

AMD presented on 7 May 2026 vLLM-ATOM, an open-source plugin that integrates optimisations for Instinct GPUs into vLLM, one of the most widely used production frameworks for serving large language models. The key characteristic is that the integration is achieved without a single change to the upstream vLLM source code — the plugin activates through the standard Python entry_points mechanism and registers two hooks: register_platform() and register_model().

Three-layer architecture

The plugin introduces a clear separation of concerns through three layers:

• The vLLM layer retains control over request scheduling, KV cache management, continuous batching and the OpenAI-compatible API.
• The ATOM plugin registers the platform, optimised model implementations and attention backend routing.
• AITER provides low-level GPU kernels optimised for Instinct hardware.

This division allows AMD to contribute optimisations without forking the vLLM repository — which is critical for sustainability in the open-source ecosystem.

Which models are supported?

The plugin covers both text (LLM) and multimodal (VLM) models through dense and MoE architectures:

• Kimi-K2.5 — multimodal MoE model (text/image/video)
• DeepSeek V3 and R1 with MLA+MoE variants, including FP8 and MXFP4 quantisation
• Qwen3 series in dense and MoE configurations
• GLM-4 and GPT-OSS with MoE support

Attention backend routing is automatic: AiterBackend for standard Multi-Head Attention, AiterMLABackend for Multi-head Latent Attention architectures.

Why is this strategically important?

NVIDIA dominates the inference market thanks to a mature software paradigm as much as its hardware. AMD’s move with vLLM-ATOM — plus AITER kernels for fused MoE and flash attention — shows the company is focusing on a “zero-friction” experience: install the plugin alongside vLLM and the optimisations switch on automatically. A live benchmark dashboard tracks throughput, latency and accuracy across model updates, enabling production verification before scaling. For the open-source community building infrastructure around Kimi-K2.5 and DeepSeek, this is a concrete step towards hardware diversity.