World models as the foundation of physical AI: new tutorial unifies robotics and autonomous driving

New research published on arXiv offers one of the first systematic guides to the field of world modeling and its role in the development of physical AI — a term encompassing intelligent systems that act in physical environments, such as robots and autonomous vehicles.

Author Il-Seok Oh, in the tutorial “A Tutorial on World Models and Physical AI,” argues that world modeling is the central unifying principle that brings together prediction, reasoning, and decision-making into a single framework.

Two schools of thought: explicit versus implicit models

The central distinction in the paper concerns two paradigms within world modeling.

Explicit world models learn structured environment dynamics. The system builds an internal simulation that allows it to “imagine” the outcomes of different actions — so-called rollout planning — without physically executing them. This approach is particularly useful in robotics where mistakes carry high costs.

Implicit world models work differently: the predictive structure is not a separate component, but is woven into the learned data representations themselves. The system does not build an explicit map of the environment — instead, that information is distributed within the model’s parameters, available as latent knowledge.

Oh does not position these approaches as rivals, but as complementary foundations. By combining both, the author argues, it is possible to build architectures that go beyond simple reactive systems and enable more complex, contextually aware intelligence.

Where we are heading: foundation models and the unification of perception, prediction, and action

The tutorial devotes special attention to the role of foundation models — large networks trained on enormous datasets like those powering modern language assistants — as a possible bridge toward unified physical AI systems.

The author argues that foundation models offer a path toward architectures that integrate perception, prediction, and action into a single system. Rather than separate modules relying on each other, the vision is a unified network capable of coherent reasoning about the physical world.

This perspective aligns with the dominant trend in the research community: the attempt to equip models that already understand language and images with robust behavior in real, physical environments — controlling a robotic arm, navigating an autonomous vehicle, or finding one’s way in an unfamiliar space.

Why is world modeling critical for robotics and autonomous driving?

The applied context of the paper focuses on robotics and autonomous driving — two domains where the ability to predict future states of the environment directly determines a system’s safety and usefulness.

In classic reactive systems, a robot or vehicle responds exclusively to current sensor data. The world modeling approach adds anticipation: the system can simulate what will happen if it turns left or accelerates, and only then make a decision.

For autonomous driving, this capability is especially valuable in rare but critical situations — such as an unexpected pedestrian or a failed traffic signal — for which training on examples is never sufficient.

Open challenges that remain

Despite progress, Oh clearly identifies three categories of unsolved problems that limit current systems:

First, hierarchical reasoning — the ability to decompose a problem into sub-problems at different levels of abstraction and integrate solutions from lower to higher levels.

Second, long-horizon planning — even today’s most advanced systems reliably plan only a few steps ahead. Situations requiring dozens or hundreds of steps remain out of reach.

Third, and perhaps most ambitious: autonomous goal formation — the ability of a system to determine on its own what it wants to achieve, without a human explicitly specifying it. This is a foundational requirement for what many researchers consider a genuine step toward general artificial intelligence.

Significance for the broader AI community

Il-Seok Oh’s tutorial arrives at a time when industry and academia are investing intensively in robotics with an AI core. Companies like Google, NVIDIA, and Tesla are presenting platforms that explicitly use world models for robot and vehicle planning.

This paper does not bring new experimental results, but something equally valuable at an early stage of a field: a coherent conceptual framework that helps researchers understand where they are, where they are going, and what the key obstacles are along the way.

For engineers building physical AI systems, the tutorial also offers practical guidance: when selecting an architecture for a specific problem, distinguishing between explicit and implicit approaches can be decisive for the system’s success in the field.