arXiv:2606.26758: EGG — multi-agent framework generates GPU kernels 2.13x faster than PyTorch

Why hand-writing GPU kernels is no longer scalable

A GPU kernel — low-level code that directly manages parallel computation on a graphics card — is critical for the speed and cost of LLM inference. Writing highly optimized kernels traditionally requires deep hardware knowledge and weeks of engineering work. EGG (Expert-Guided agent framework for kernel Generation), a paper submitted by Yaochen Han and collaborators on June 25, 2026, offers an automated path: a multi-agent system that combines expert knowledge with LLM code generation.

How does the two-stage approach work?

EGG decomposes the problem into two clearly separated steps. In the first step, agents design the algorithmic structure — defining mathematical operations and the computation graph. In the second step, specialized agents perform hardware-specific tuning: parallel thread mapping, tensor tiling (arranging data in matrix tiles for more efficient memory access), and memory access optimization for the specific GPU. Between stages, a multi-agent mechanism passes context, so each agent operates with a complete picture of the solution so far, not in a vacuum.

Results: 2.13x faster than PyTorch, ahead of RL-based systems

On KernelBench — the standard task set for evaluating automatically generated GPU kernels — EGG achieves an average 2.13x speedup over the PyTorch baseline. That is a measurable jump above both agent-based competitors that do not use expert guidance and approaches using reinforcement learning for kernel optimization. RL-based systems learn by trial and error; EGG instead embeds expert rules directly into the agent instructions, reducing the search space and accelerating convergence toward a correct and fast solution.

Practical implications for LLM deployment

Faster kernels translate directly into lower inference costs and shorter response times for production LLM systems. If EGG proves applicable beyond research benchmarks, teams deploying their own models could automate the part of optimization that currently consumes the most engineering hours — without needing specialists in hardware optimization.