MiniMax M3 in vLLM: Day-0 Support for a Model with 1-Million-Token Context

The vLLM team on June 12, 2026, released full Day-0 support for MiniMax M3, a new model from Chinese AI startup MiniMax that combines 1-million-token context, native multimodality, and a Mixture-of-Experts (MoE) architecture. The announcement marks the first major implementation of a long-context MoE model within vLLM with full feature parity on launch day.

Why Is 1-Million-Token Context a Serving Challenge?

A standard KV cache grows linearly with context length — 1 million tokens in BF16 precision requires hundreds of gigabytes of GPU memory just to store intermediate attention steps. The traditional approach of materializing the full KV cache becomes infeasible on realistic hardware configurations. MiniMax M3 solves this through MiniMax Sparse Attention (MSA), an architectural innovation that redefines how the model accesses long sequences.

MSA divides the KV cache into blocks of 128 tokens. Each query — or GQA group of queries — scores all available blocks through lightweight indexer-score kernels and selects only the most relevant for actual attention computation. The result is dramatically reduced memory and compute burden: the model does not need to fully materialize 1 million tokens but selectively accesses relevant segments. The vLLM implementation also uses a KV-block-major prefill layout designed specifically for this data access pattern.

The Technical Stack: from EAGLE3 to Expert Parallelism

Support for MiniMax M3 comes with a complete set of vLLM features. Prefix caching and chunked prefill work without modification alongside the MSA mechanism. A tool-call parser and reasoning output parser enable native agentic use, while thinking-mode control allows dynamic enabling and disabling of chain-of-thought reasoning per request.

A particularly notable addition is support for EAGLE3 speculative decoding. A draft model for EAGLE3 is available on HuggingFace alongside the release, and measurements show speculative token acceptance of 67% with an average draft length of 3.0 tokens per step — in practice meaning significantly fewer calls to the main model and lower latency.

On the parallelization side, vLLM supports both tensor parallelism and expert parallelism for MoE layers, allowing the model to be distributed according to infrastructure needs.

Hardware Validation and Backend Specifics

MiniMax M3 is validated across six GPU platform classes. On the NVIDIA side:

• H200 (Hopper architecture) — uses the Marlin MXFP8 backend for quantized inference
• GB200 and B300 (Blackwell architecture) — use the DeepGEMM backend optimized for the new hardware generation

On the AMD side:

• MI300 and MI350 GPU series

The model is available in BF16 and MXFP8 precision, with the MXFP8 variant using the appropriate MoE backend depending on GPU architecture.

Measured Benchmark Performance

All results shown in the announcement were measured on a B300 configuration:

• GSM8K: 91.51% (strict match) / 91.66% (flexible match) — the standard mathematical reasoning benchmark for language models
• ShareGPT throughput at 256 concurrent requests: 8,530 tokens per second
• TPOT (Time Per Output Token): 56.0 milliseconds
• EAGLE3 acceptance: 67% with an average speculative draft length of 3.0 tokens

These results were achieved without sacrificing long-context capability — the model retains its full 1-million-token capacity alongside the stated throughput figures on a single B300 configuration.

Multimodality and Agentic Use Cases

MiniMax M3 is not a text-only model. Native support for images and video is integrated into the architecture, with no need for a separate vision encoder or additional processing steps. Combined with tool-call and reasoning parsers, the model is ready for complex agentic tasks that combine visual content analysis, multi-step reasoning, and external tool invocation.

Availability in BF16 and MXFP8 formats — together with Day-0 support for all listed vLLM features — makes MiniMax M3 practically usable from the first day without waiting for subsequent updates or feature restrictions. The vLLM community and the MiniMax team developed the integration jointly, and the announcement notes that close collaboration was key to achieving full feature parity on launch day.