Is every video generation model a world model?

Not by most definitions. A world model is usually expected to support prediction conditioned on an action (what happens if I do X), useful for planning or control. A text-to-video generator that only produces a plausible clip from a prompt, with no action-conditioning or use in a decision loop, is closer to the video-diffusion mechanism than to a world model in the stricter sense, even though the underlying next-frame-prediction skill overlaps.

Do world models actually understand physics?

Not in the sense of having explicit equations. They encode statistical regularities that produce physics-like behavior often enough to be useful, learned entirely from correlations in training video. That's a different claim from deriving physical laws, and it's why performance degrades on situations underrepresented in training data.

MODEL TYPE

What Is a World Model?

A world model is a system trained to predict how an environment changes over time, given its current state and, often, an action taken in it. Video-generation models trained at large scale, Sora among them, are increasingly described as an early form of this: predicting the next frame well enough requires learning something like physics, object permanence, and cause and effect, even though nobody labeled any of that during training.

Predicting the next state, not just the next pixel

A plain video generator's job is to produce a plausible clip. A world model's job is narrower and stricter: given the current state of an environment and an action (move forward, turn left, drop the object), predict what the next state looks like. That distinction matters for what the model is trained to get right. A clip generator can get away with a plausible-looking guess. A world model used for planning or robotics has to get the consequence of an action right often enough to be useful for deciding what to do next.

In practice, the line blurs, because the same architecture, a model trained to predict future frames from past ones, can serve both purposes. Systems referenced as world models as of 2025, DeepMind's Genie, NVIDIA's Cosmos, Meta's V-JEPA 2, and video generators like Sora when used this way, all share this core mechanism even though they target different end uses.

How physics shows up without being programmed in

Nobody writes a gravity equation into a video generation model. What happens instead: a model trained on enough real video, where objects reliably fall, bounce, occlude each other, and keep existing when they leave frame, has to encode those regularities to predict the next frame accurately. Predicting a ball's position ten frames from now without any sense of downward acceleration produces a worse loss than a model that has implicitly learned it. Scale up the data and the training signal, and behavior that looks like physical understanding falls out of a purely predictive objective, the same way grammar and reasoning fall out of next-token prediction in a language model.

This is why OpenAI's own write-up on Sora frames it as an early step toward a general-purpose simulator, and equally why it's described as a step and not an arrival: this kind of physics is learned implicitly and statistically from correlations in training video, not derived from first principles, so it holds only as well as the training data covered the situation.

Where the implicit physics breaks

The failures are consistent, not random. Long-range object permanence is weak: something that leaves frame and re-enters can come back changed, because the model never had a real persistent internal state for it, only a statistical tendency to keep drawing something similar. Rare physical configurations, an object balanced at an odd angle, an unusual collision, get less accurate treatment than a well-represented scenario like a ball bouncing on flat ground, because the model learned physics from examples, and rare situations have few examples.

The field has not settled on one definition of what a world model must do to count as one. A 2026 perspective paper from researchers surveying the space notes there's still no consensus on what a world model fundamentally is, what it should predict, or how it should be built, which is part of why the term gets applied to systems as different as a video generator and a robotics planning model.

FAQ

Is every video generation model a world model?: Not by most definitions. A world model is usually expected to support prediction conditioned on an action (what happens if I do X), useful for planning or control. A text-to-video generator that only produces a plausible clip from a prompt, with no action-conditioning or use in a decision loop, is closer to the video-diffusion mechanism than to a world model in the stricter sense, even though the underlying next-frame-prediction skill overlaps.
Do world models actually understand physics?: Not in the sense of having explicit equations. They encode statistical regularities that produce physics-like behavior often enough to be useful, learned entirely from correlations in training video. That's a different claim from deriving physical laws, and it's why performance degrades on situations underrepresented in training data.

Sources

Last updated 2026-09-13

What Is a World Model?

Predicting the next state, not just the next pixel

How physics shows up without being programmed in

Where the implicit physics breaks

FAQ

Sources

Related