PyTorch: Triton 3.7 brings plugin extensions and TLX — up to +15% throughput on AMD MI350

Triton gains a dynamic plugin system

Triton — the open-source programming language for writing GPU kernels (optimized GPU routines that perform specific computational operations) — introduces in version 3.7 a new extension mechanism via the TRITON_PLUGIN_PATHS environment variable. This is a dynamic plugin system that allows development teams to add their own compiler passes and optimizations without recompiling Triton and without forking the repository. Critically, there is no overhead from dynamic loading — codegen is identical to built-in implementations.

Meta’s TLX: measurable gains on H100 and AMD MI350

As the reference implementation of the new system comes TLX (Triton Language Extensions), Meta’s package available as a standalone utlx library for NVIDIA and AMD ecosystems. On H100 GPUs, Triton with TLX exceeds NVIDIA’s cuBLAS by +3.7% — reaching 573.2 TFLOPS on LLM workloads. cuBLAS is NVIDIA’s hand-optimized linear algebra library, long considered the gold standard of performance.

Even more significant are the gains on the AMD side: on AMD MI350 GPUs, TLX delivers +12–15% throughput compared to AMD’s rocBLAS. The GPUMode Trimul benchmark records a 1.61× speedup — a drop from 19.2 ms to just 12.0 ms for matrix multiplication.

What does this mean for the AI ecosystem?

The plugin system opens the door for projects like TLX that previously could not reach end users without complex distribution. The difference compared to previous solutions lies in approach: instead of teams forking Triton and manually integrating optimizations, they can now distribute a standalone package activated by a single environment variable. For the AMD ecosystem, which historically lags behind NVIDIA in optimized kernel libraries, gains of +12–15% on MI350 are a particularly relevant step toward performance parity in LLM inference and training scenarios.

---

Source: PyTorch Blog