arXiv:2606.02800: NVIDIA Cosmos 3 — omnimodal world model for physical AI

On 1 June 2026, NVIDIA released an arXiv preprint titled “Cosmos 3: Omnimodal World Models for Physical AI”. The paper introduces Cosmos 3, a model that, within a single unified architecture, simultaneously processes and generates language, image, video, audio and action sequences. The goal is to build a foundational world model for physical AI — robots and embodied agents that operate in the real world. The author list names 294 contributors.

What does Cosmos 3 actually do?

Cosmos 3 brings into one framework what was previously separate: vision-language models, video generators, world simulators and action models. Instead of distinct systems for understanding and generation, a single model takes in and produces multiple modalities at once. It thereby covers both perception (understanding a scene) and prediction (how a scene will evolve after an action), which is essential for controlling a robot.

Mixture-of-transformers architecture

The system is built on a mixture-of-transformers design — an architecture in which multiple transformer components share a common framework and process different kinds of data, instead of a single monolithic model. According to the paper, this approach enables “highly flexible input-output configurations”: the model can take in text and an image and return video or an action sequence, depending on the task. The term omnimodal means that all five modalities — language, image, video, audio and actions — live inside the same model.

How good is it compared to the competition?

According to the preprint, Cosmos 3 achieves the best results (state-of-the-art) on a diverse set of understanding and generation tasks. The authors state that, at the time of writing, Artificial Analysis ranked Cosmos 3 as the best open-source Text-to-Image and Image-to-Video model, while RoboArena rated it the best policy model — that is, the model that decides which actions a robot takes. These claims are from the paper itself and refer to the leaderboards cited at the time of publication.

Open-source package

Alongside the paper, NVIDIA is opening up the entire package. The code and model checkpoints, curated synthetic datasets and an evaluation benchmark are available. The materials are published under the Linux Foundation’s OpenMDW-1.1 license, the repository is on GitHub (github.com/nvidia/cosmos), and the model hub on Hugging Face. The first version of the preprint was posted on 1 June 2026, and a revised version on 5 June 2026.

Why it matters

By releasing the complete package — code, checkpoints, data and benchmarks — NVIDIA lowers the barrier to physical-AI research beyond large labs. World models that equally well understand and generate multiple modalities are considered one of the key ingredients for scalable robotics and embodied agents, so the true performance of Cosmos 3 will become clear once the community starts testing it on its own hardware and tasks.