A world model is a learned predictor of what happens next, given the current state and a proposed action. That lets a system rehearse consequences internally instead of discovering them in reality. The value is not the prediction itself but the ability to evaluate an action before committing to it.
The question worth asking is not “what is a world model” but: what can a system do once it can imagine outcomes that it fundamentally cannot do otherwise?
The answer is planning. Without a predictive model, a system must either follow rules written in advance or learn by acting and observing the results, which is expensive when actions have physical consequences.
At a glance
- Prediction error is the training signal, requiring no human labelling
- Errors compound over rollout steps, which caps the useful horizon
- Latent-space prediction scales better than pixel prediction
- The model need not be accurate, only accurate enough to rank actions
- Language models learn a weak world model as a side effect
- Simulation-to-reality transfer is the persistent hard problem
How does a world model learn?
By predicting the next observation and being corrected by what actually happens.
That loop is self-supervising. Reality supplies the label, so no annotation is required, which is why world models can be trained on quantities of interaction data that supervised approaches could never afford to label.
Observation and action in, predicted next observation out, prediction error back. The mechanism is simple; everything difficult about it lies in what happens when you chain predictions together.
Why does the horizon collapse?
Figure 1 — One-step prediction versus rollout
One step ahead
The model predicts from a real observation it was just given.
Accuracy is high, because the input is ground truth and only one step of error is introduced.
Reliable
Twenty steps ahead
Each prediction feeds the next, so the model is soon predicting from its own imagined states rather than from reality.
Small errors compound multiplicatively and the trajectory drifts into states that never occur, often while remaining superficially plausible.
Unusable
This is the defining constraint of the field. Systems handle it by planning over short horizons and replanning constantly from fresh observations, rather than trusting a long imagined trajectory.
The practical pattern that follows is short-horizon planning with frequent re-grounding: predict a handful of steps, act on the first, observe reality, and predict again. Reality corrects the drift before it accumulates.
What is predicted, and in what space?
Early approaches predicted raw sensory input, which is wasteful because most pixels are irrelevant to any decision.
Modern approaches predict in a learned latent space instead: compress observations into a compact representation, then predict the next representation. This discards detail that does not affect outcomes and makes rollouts far cheaper.
| Prediction target | Cost | Trade-off |
|---|---|---|
| Raw pixels | Very high | Wastes capacity on visually salient but decision-irrelevant detail |
| Learned latent state | Moderate | Efficient, but the encoder can discard something that mattered |
| Object and relation state | Low | Highly interpretable, requires a structured representation to exist |
| Reward or outcome only | Lowest | Enough to rank actions, no explanation of why |
The last row is worth taking seriously. If the only purpose is choosing between actions, predicting their relative outcomes is sufficient and far cheaper than predicting the world in detail. Many deployed systems need nothing more.
Do language models have world models?
Weakly, implicitly, and the honest answer is contested.
Predicting text about the physical world requires representing something about how that world behaves, so a language model that reliably completes “she dropped the glass and it” has encoded a regularity. Whether that constitutes a world model or a statistical reflection of one is a live disagreement, and this article does not resolve it.
What is less contested is the practical limitation: such knowledge is not grounded in sensory experience, so it fails on precisely the physical specifics that matter for action. A system can describe what happens when a glass falls and cannot predict where the pieces land.
Watch for this
Fluent description of physical processes is easily mistaken for predictive capability. A system can produce a textually correct account of a mechanism while being unable to predict a specific outcome accurately enough to act on. If you are relying on physical prediction, test it against measured outcomes rather than against whether the explanation reads well.
Where does this actually get used?
- Robotics. Rehearsing a manipulation before attempting it, where a failed attempt costs time or hardware.
- Autonomous systems. Predicting how other agents will move over the next few seconds.
- Industrial control. Modelling a process to plan setpoint changes without disturbing production.
- Simulation-based training. Generating experience for a policy far faster than real time allows.
- Agent planning. Predicting the effect of a tool call before making it, which is the software analogue.
The last one connects this to ordinary systems work. An agent estimating whether an action will succeed before executing it is doing exactly what a robot does when it rehearses a grasp, with the same compounding-error limits over long plans.
What is the transfer problem?
A model trained in simulation encounters reality and finds the differences matter.
Simulators approximate friction, deformation, sensor noise and timing. A policy trained against those approximations learns to exploit them, and exploits that work in simulation fail physically.
The mitigation that works is domain randomisation: vary simulation parameters aggressively during training so the policy cannot depend on any particular value. The resulting behaviour is more conservative and transfers better, which is a deliberate trade of peak performance for robustness.
Where do teams go wrong?
Optimising prediction accuracy rather than decision quality
A model that predicts poorly but ranks actions correctly is more useful than one with lower prediction error that ranks them wrongly. Measure the decisions, not the pixels.
Planning over horizons the model cannot support
Long imagined trajectories look impressive and are usually fiction past a certain depth. Measure where rollout accuracy degrades and plan strictly inside that boundary.
Trusting the model outside its training distribution
World models extrapolate badly and rarely signal that they are doing so. Novel objects, unusual configurations and unseen dynamics produce confident predictions with no accompanying warning.
What do experienced teams do differently?
They measure rollout accuracy as a function of horizon length and treat the resulting curve as a hard system parameter.
Knowing that predictions stay useful for roughly five steps and become unreliable past twelve turns planning depth from a guess into a specification, and it prevents the whole class of failures caused by trusting imagined futures too far.
They also keep a fallback that requires no prediction at all. When the model’s confidence drops or the situation looks unfamiliar, reverting to a conservative reactive policy is safer than acting on an extrapolation, particularly where actions have physical consequences that cannot be undone.
A short glossary
- World model
- A learned predictor of future states given the current state and an action, used to evaluate actions before taking them.
- Rollout
- Chaining predictions forward so each output becomes the next input, producing an imagined trajectory.
- Compounding error
- The multiplicative growth of prediction error across rollout steps, which caps the useful planning horizon.
- Latent space
- A compressed learned representation in which prediction is cheaper than in raw observation space.
- Domain randomisation
- Varying simulation parameters during training so a policy cannot depend on any specific approximated value.
Where to go deeper
Three directions, in ascending order of difficulty.
Begin with model-based reinforcement learning, which is where the planning-with-a-learned-model machinery is developed and where the compounding-error problem is treated rigorously. Then look at representation learning for control, since what a world model predicts over matters more than how it predicts. Finally, read the simulation-to-reality transfer literature, which is the least solved and most practically consequential of the three.
The connecting thread: a world model is only ever a tool for choosing actions, so every design decision should be judged by whether it improves the choice rather than the prediction.
Key takeaways
- A world model predicts next states so a system can evaluate actions before committing to them.
- Prediction error is a self-supervising signal, requiring no human annotation.
- Errors compound across rollout steps, which is the defining constraint on planning horizon.
- Short-horizon planning with constant re-grounding in real observations is the standard mitigation.
- Predicting in latent space beats predicting raw pixels, and sometimes predicting outcomes alone is enough.
- Language models encode weak, ungrounded physical regularities that fail on specific quantitative prediction.
- Judge a world model by the quality of decisions it enables, not by its prediction accuracy.
Frequently asked questions
What is a world model in AI?
A learned predictor of what happens next given a current state and a proposed action. It lets a system rehearse consequences internally rather than discovering them by acting, which matters most where actions are slow, costly or physically irreversible.
How are world models trained?
Through self-supervision on prediction error. The system predicts the next observation, reality supplies the correct answer, and the difference is the training signal. No human labelling is required, which is why they can be trained on far more interaction data than supervised methods.
Why do predictions degrade over long horizons?
Because each prediction feeds the next, so the model is soon predicting from its own imagined states rather than from reality. Small errors compound multiplicatively and the trajectory drifts into states that never occur, often while still looking superficially plausible.
Do large language models have world models?
Weakly and implicitly, and whether that counts is genuinely contested. Predicting text about the world requires encoding some regularities about it. What is clearer is the limitation: that knowledge is not grounded in sensory experience and fails on specific physical prediction.
Should a world model predict pixels or something else?
Usually something else. Raw pixel prediction wastes capacity on visually salient but decision-irrelevant detail. Predicting in a learned latent space is far cheaper, and where the goal is only to rank actions, predicting relative outcomes alone can be sufficient.
What is the simulation-to-reality gap?
The failure of policies trained in simulation to work physically, because simulators approximate friction, deformation, sensor noise and timing. Policies learn to exploit those approximations, and the exploits do not survive contact with reality.
How does domain randomisation help?
By varying simulation parameters aggressively during training so the policy cannot depend on any particular value. The resulting behaviour is more conservative and transfers considerably better, trading peak simulated performance for robustness in the real environment.
How do I evaluate a world model?
On decision quality rather than prediction accuracy. A model with higher prediction error that ranks candidate actions correctly is more useful than a more accurate one that ranks them wrongly. Also measure rollout accuracy against horizon length, and plan strictly inside where it holds.
References
- Ha, D., and Schmidhuber, J. World Models. arXiv:1803.10122.
- Hafner, D., et al. Dream to Control: Learning Behaviors by Latent Imagination. arXiv:1912.01603.
- Tobin, J., et al. Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World. arXiv:1703.06907.
- Schrittwieser, J., et al. Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model. arXiv:1911.08265.
