CNCF: HAMi becomes Incubating project — GPU virtualization for Kubernetes AI workloads

What are GPU virtualization and Kubernetes?

GPU virtualization is a technology that splits a single physical GPU into multiple logical, shareable units that different applications or containers can use independently of one another. Without virtualization, a single AI task consuming only 20% of GPU capacity blocks the remaining resources — this fragmentation problem causes significant operational losses for expensive data centers. Kubernetes is the de facto standard for container orchestration: a system that automates the scheduling, scaling, and management of containerized applications in clusters. HAMi combines both — bringing GPU virtualization as a native Kubernetes mechanism.

From Sandbox to Incubating: rapid community growth

The CNCF Technical Oversight Committee approved HAMi (Heterogeneous AI accelerator Management interface) for Incubating status, marking the project’s maturity and a clear path toward future CNCF graduation. The project entered CNCF Sandbox in August 2024, and in less than two years it built an impressive community: 550 or more organizations using it in production, 2,687 GitHub contributors, and 16 released versions. For comparison, many mature CNCF projects needed much longer to reach similar adoption numbers.

Technical architecture and integrations

HAMi directly addresses resource fragmentation by slicing physical GPUs into shareable units — one of the key operational challenges for organizations running diverse AI workloads on shared clusters. Instead of each job requiring an entire GPU, HAMi enables granular capacity allocation according to actual needs.

The project integrates with two CNCF projects specialized in AI scheduling: Volcano, which focuses on batch AI and ML tasks, and Koordinator, which manages complex multi-resource priorities in heterogeneous clusters. Together, this trio of projects forms a cohesive stack for managing AI infrastructure within the Kubernetes ecosystem — from job scheduling to physical accelerator allocation.

Significance for open-source AI infrastructure

Incubating status in CNCF brings HAMi greater visibility, access to CNCF infrastructure, and a formal security audit. For organizations considering adoption, CNCF status serves as independent confirmation that the project has a sustainable governance structure and sufficient community. As AI workloads grow in cloud and on-premise clusters, tools like HAMi that increase the utilization of expensive GPU resources are becoming increasingly relevant to the operational economics of AI systems.