AMD: GEAK Agent Automatically Optimized DeepSeek-V4 MLA Kernel on MI355 with Up to 9× Speedup

AMD has released results from applying the GEAK agent — an AI system for automatic GPU kernel generation and optimization — to the DeepSeek-V4 model running on AMD MI355 accelerators. The automated migration from PyTorch to Triton delivers speedups that manual optimization would struggle to match in comparable time.

What Are MLA and Triton — Key Concepts?

MLA (Multi-head Latent Attention) is a memory-efficient variant of the attention mechanism that DeepSeek-V4 uses to dramatically reduce KV-cache costs compared to classic Multi-head Attention — a key innovation of the model, but demanding for hardware optimization. Triton is a programming language for writing GPU kernels that directly targets the hardware characteristics of accelerators; unlike generic PyTorch, Triton code can exploit the specifics of AMD MI355’s memory hierarchy. The GEAK agent automates exactly this transition — without writing Triton code manually.

Concrete Speedups: Prefill, Decode, and End-to-End

Measurements on the AMD MI355 show clear gains across all inference phases. In the prefill phase (processing the input prompt), the speedup is 9.13× for Config 1 and 6.92× for Config 2 compared to the original PyTorch implementation. In the decode phase (token generation), the geomean speedup across 26 different configurations is 4.94×. End-to-end in the SGLang framework, GEAK delivers 2.10× higher throughput and 3.71× lower TTFT (time-to-first-token — the time until the first generated word), a critical parameter for user experience.

SGLang Integration and Accuracy Verification

Integration with the SGLang framework shows 16%–110% E2E improvement depending on concurrency level (2 to 32 simultaneous requests) — higher concurrency yields proportionally larger gains. Particularly important is the accuracy verification result: all 304 prefill and 4,748 decode test cases pass without any regressions. The speedups were not achieved through approximation or sacrificing precision, but through pure hardware kernel optimization. GEAK thus demonstrates that AI-assisted GPU code optimization can replace weeks of manual engineering work.