AMD Explains Occupancy Math for the MI355X: High Occupancy Is Not Required for Peak Throughput

The AMD ROCm team has published a comprehensive technical guide on computing GPU occupancy for the CDNA4 architecture, specifically for the MI355X accelerator. The text targets GPU kernel engineers optimizing custom kernels for production inference or training — and delivers a finding that is directly counterintuitive for practitioners coming from the NVIDIA CUDA ecosystem.

What Is Occupancy and Why Compute It Manually?

GPU occupancy measures the fraction of available resources actively filled with wavefronts (in CUDA terminology: warps). High occupancy has long been considered a prerequisite for high throughput — more active wavefronts means more opportunities to hide memory access latency.

The guide’s authors take a different position: occupancy is fully derivable from known hardware constraints and the resources a kernel consumes, so it should be understood from first principles rather than treated as an opaque metric from profiling tools. The goal is to equip engineers who can predict occupancy before running a kernel.

Four Limiters and the CDNA4 Architecture

The MI355X is organized around 256 Compute Units distributed across 8 XCD chiplets, clocked at 2.4 GHz. Each CU contains 4 SIMD units of 64 lanes each, with a private register file.

The guide identifies four resources that set the upper bound on occupancy:

Vector registers (VGPR) — on CDNA4 the register file is 512 entries per lane, and registers are shared between regular and accumulator registers. This is a significant difference from CDNA3 where accumulator registers (AccVGPR) were a separate pool — AMD emphasizes that engineers carrying knowledge from the previous generation must update this assumption.

Scalar registers (SGPR) — approximately 800 per SIMD unit, used for values that are uniform across an entire wavefront.

Local Data Share (LDS) — increased from 64 KB on CDNA3 to 160 KB per Compute Unit on CDNA4.

Workgroup slots — a hardware limit on the number of simultaneously active workgroups per CU.

Counterintuitive Finding: Low Occupancy, Peak Throughput

The guide’s central finding runs directly counter to intuition from CUDA optimization experience. AMD’s microbenchmark for MXFP8 operations — for which the MI355X achieves approximately 5 PFLOP/s — shows that kernels with high instruction-level parallelism (ILP=8) sustain 4.82 PFLOP/s at just 12% occupancy. That figure exceeds everything an ILP=2 kernel achieves even at 96% occupancy.

The reason: the matrix units on CDNA4 are fast enough to saturate the pipeline with a small number of active wavefronts. When a kernel successfully hides latency through ILP rather than wavefront parallelism, high occupancy is not merely unnecessary — it can be impossible due to register file constraints while delivering no benefit whatsoever.

MXFP8 GEMM Example: Same Kernel, Two Generations

The guide includes a concrete example: an MXFP8 GEMM kernel with a 256-thread tile consuming 128 VGPRs and 32 KB of LDS.

On CDNA3 with 64 KB LDS: the kernel is LDS-limited and achieves 25% occupancy — only two such kernels can be simultaneously active per CU because together they exhaust the LDS.

On CDNA4 with 160 KB LDS: the same kernel is no longer LDS-limited. It is now register-limited, and occupancy rises to 50% — four instances per CU. Same source code, different limiter, twice the occupancy.

Context: AMD Closing the Documentation Gap Versus CUDA

This guide is part of AMD’s broader effort to improve ROCm software documentation and reduce the barrier for engineers migrating from the NVIDIA ecosystem. CUDA has held an advantage for decades in the depth of its technical documentation — details like these about resource limits and their impact on occupancy were well documented for NVIDIA architectures, while for AMD GPUs they required experimental investigation or insight into internal code.

Guides like this one, built from first principles, target precisely that gap: engineers who know what they are looking for can now derive occupancy analytically, without running a kernel, which accelerates the iterative optimization cycle for production AI models on AMD infrastructure.