Aiodic

Back to the table

113 Emerging Wm World Models Frontier

World Models

AI that learns how environments behave.

What it is

Put a full glass of water near the edge of a table and nudge it with your elbow. Before it has moved an inch, you already know what is about to happen: it will tip, fall, hit the floor, and probably break, and there will be water everywhere. You didn't need to see it happen. You ran it in your head first, fast enough to lunge for the glass.

That mental simulator is what AI researchers mean by a world model. It's an internal model of how an environment behaves: if things are like this now, and this action happens, here is what things will probably look like next. People carry one around all day. It's how you know a ball thrown upwards comes down, that a door opens towards you if it has a handle on your side, and that the car in front will slow down because its brake lights just came on.

The idea is old. In 1943 the Scottish psychologist Kenneth Craik suggested in his book The Nature of Explanation that a mind might carry a "small-scale model" of the outside world and of its own possible actions, so it can try out alternatives in its head before committing to one. Modern AI research on world models is, in a sense, an attempt to build Craik's small-scale model out of a neural network.

A world model in AI has two jobs:

Where does the model come from? There are two main sources of learning.

Watching. A system can learn a great deal about how the world behaves by watching enormous amounts of video. Things fall down, not up. Liquids spread. Objects that go behind a sofa usually come out the other side. None of this needs labels; the future frames are the answer key for the present ones. This is a form of unsupervised learning: the data teaches itself.

Doing. Watching tells you what tends to happen, but not what happens because of you. For that, a system has to act and see the consequences, whether in a game, a simulator, or a real robot. This is where world models meet reinforcement learning, the branch of AI that learns by trial and error. Most successful world models combine both: learn the general shape of the world from watching, then learn how your own actions change it from doing.

It's worth being clear about what a world model is not. It's not a physics engine written by a programmer, like the one inside a video game, where someone typed in the rules for gravity. A world model learns the rules from experience, which means it can pick up things nobody thought to program, and also that it can get things confidently, subtly wrong. And it's not the same as a video generator, although the two are converging. A video generator makes a plausible clip. A world model is judged by whether its predictions are right when something acts in it.

Why it matters

It is how you plan without breaking things. Most AI today reacts: a question comes in and an answer goes out. Planning means thinking several steps ahead and choosing between options by their likely results, which you can only do if you can predict results. A world model lets a system ask "what if I did this?" and "what if I did that instead?" and compare the imagined outcomes. This is the missing piece between the planning that chatbots do with words and the planning a robot needs to do with its hands.

Real-world practice is slow, costly, and sometimes dangerous. A robot that learns only by trying things will drop a lot of glasses. A self-driving car cannot practise swerving around a child by actually swerving around a child. A world model is a practice space: once it's good enough, a system can rehearse thousands of situations in imagination for every one it tries for real. It is a close cousin of synthetic data, which is why self-driving companies were among the first to invest.

It's how AI might move from screens into the physical world. Language models learned from text, and text is a thin slice of reality. A lot of what you know about the world (how heavy a full suitcase feels, what a slippery floor looks like) was never written down. Researchers working on embodied AI, meaning AI with a body such as a robot, widely see learned world models as one of the things robots most need.

It sits at the centre of a live argument about the future of AI. Some prominent researchers, most famously Yann LeCun, argue that today's language models lack a proper understanding of the physical world and that world models are the path forward. Others point to evidence that language models already build internal models of a kind. Who is right shapes where billions of dollars of research money go. It's also one of the questions people have in mind when they argue about AGI.

For creators, it's changing what video and games can be. The same technology that lets a robot imagine the next second of its day is producing video that responds to the viewer and game worlds that are generated as you walk through them. These are early and imperfect, but they're arriving in products now.

A word of caution before the examples. This is an "Emerging" element for a reason. A lot of what's written about world models is prediction and ambition, some of it from companies with a product to sell. The examples below are chosen to separate what has been shown from what is hoped.

Real-world examples

An agent learns to dodge fireballs inside its own dream (2018)

In March 2018, the researchers David Ha and Jürgen Schmidhuber posted a paper called simply "World Models". They built an AI agent from three parts: a vision part that squashed each video frame into a small summary, a memory part that learned to predict the next summary, and a tiny controller that chose actions. The memory part was the world model.

In a simple top-down car racing game, the agent used its world model's predictions to drive, and reached an average score of 906 over 100 trials, which the authors reported as the first time that task had been solved by this kind of method. The more striking experiment used a version of the classic game Doom called "Take Cover", where the goal is to survive by dodging fireballs. The researchers trained the agent entirely inside the world model, a simulated version of the game the model had learned from watching play, which they described as the agent's "dream". Then they dropped the agent into the real game. Its skills transferred: it survived long enough to pass the game's test.

The paper is also remembered for an honest failure. When the dream was too predictable, the agent learned to exploit flaws in it: moves that worked in the imagined game and nowhere else. The fix was to make the dream deliberately noisier and more uncertain during training, which made the agent more cautious and produced better results in the real game. That's a lesson that still applies. Anything that learns inside an imperfect model will find the imperfections.

Learning the rules instead of being told them: MuZero and Dreamer (2020 to 2025)

DeepMind's AlphaZero had famously taught itself chess, shogi, and Go, but it was given the rules of each game (the reinforcement learning article tells that story). On 23 December 2020, DeepMind published MuZero in the journal Nature. MuZero was not given the rules. It learned its own model of each game, and a deliberately limited one: DeepMind explained that MuZero models only what matters for planning, namely how good a position is, which move looks best, and what reward an action brings. It matched AlphaZero's superhuman play in the board games and set a new state of the art on 57 Atari video games, while planning with a model it had built itself.

A second line of work, led by Danijar Hafner, pushed the idea into a much messier world. The Dreamer algorithms learn a world model from experience and then improve their behaviour by imagining future scenarios inside it, rather than practising everything for real. The third version, DreamerV3, was posted in January 2023 by Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap, and published in Nature in April 2025. Its headline result was in Minecraft: it was the first algorithm to collect diamonds from scratch, with no human demonstrations and no step-by-step curriculum. Getting a diamond in Minecraft takes a long chain of steps (wood, then tools, then stone, then iron, then digging deep) with no reward until near the end, which is exactly the kind of long-horizon problem where imagining ahead pays off. The same settings worked across more than 150 different tasks without retuning.

What both show is demonstrated, not speculative: a system that learns a model of its environment can then plan and practise inside that model, and it can beat systems that don't. The catch is that these environments are still games, with simple goals and a limited set of actions.

Genie: worlds you can walk around in, and a car company that uses them (2024 to 2026)

Google DeepMind's Genie project took the world-model idea in a visual direction. On 4 December 2024 it announced Genie 2, which could turn a single image into a playable 3D environment that responds to keyboard and mouse. DeepMind said the worlds stayed consistent for up to a minute, with most examples lasting 10 to 20 seconds, and called the research early, with substantial room for improvement.

In August 2025 it announced Genie 3, which generates navigable worlds from a text description in real time at 720p and 24 frames per second, stays consistent for a few minutes, and supports what DeepMind calls "promptable world events", such as changing the weather mid-walk. DeepMind listed its limits plainly: a narrow range of actions, trouble with several agents interacting, imperfect accuracy for real places, and interaction lasting minutes rather than hours. On 29 January 2026 Google opened a prototype called Project Genie, running on Genie 3, to its top-tier AI Ultra subscribers in the US; at launch, each world could be explored for up to 60 seconds, and Google warned that physics might be imperfect and controls laggy.

The use that tells you most about where this is heading came from Waymo, the self-driving car company owned by Google's parent Alphabet. On 6 February 2026 Waymo announced the Waymo World Model, built on Genie 3 and adapted for driving. It generates both camera and lidar (laser-radar) data, and Waymo says it uses it to simulate rare events that are almost impossible to collect on real roads, from tornadoes and floods to wrong-way drivers and, in Waymo's own examples, an elephant in the road. It also uses it to replay past situations and ask "what if" the car had done something different. The synthetic data article covers Waymo's earlier simulator; the change here is that the simulated world is now generated by a learned model rather than built by hand.

NVIDIA is betting the same way. At CES in January 2025 it launched Cosmos, a family of what it calls "world foundation models" for training robots and self-driving cars, with robot makers such as 1X, Agility, and Figure AI among the early adopters it named.

Meta's V-JEPA 2 and Yann LeCun's bet (2022 to 2026)

Yann LeCun, a Turing Award winner and for years Meta's chief AI scientist, has argued for a long time that predicting the next word is not enough to reach human-level intelligence and that AI needs world models learned from observing the world. In a 2022 position paper, "A Path Towards Autonomous Machine Intelligence", he proposed an approach called JEPA (joint embedding predictive architecture). The key idea is that the model should predict an abstract summary of what happens next, not every pixel. Predicting pixels wastes effort on things that don't matter, such as the exact pattern of leaves on a tree, when what matters is that the ball is heading for the window.

On 11 June 2025 Meta released V-JEPA 2, a 1.2-billion-parameter world model built this way. It first learned from over a million hours of video, then from just 62 hours of robot data showing how actions change things. Meta reported that robots using it could pick up and place objects they hadn't seen, in environments they hadn't trained in, with success rates of 65 to 80 percent, planning by imagining the result of candidate actions. Meta also released new tests of physical common sense, and was candid that on one of them, which asks whether a video is physically possible, current models performed close to chance while humans scored 85 to 95 percent.

In November 2025 LeCun announced he was leaving Meta to start a company to pursue what he calls Advanced Machine Intelligence: systems that understand the physical world, have persistent memory, can reason, and can plan. The company, AMI Labs, has been widely reported to have raised more than a billion dollars in early 2026. That is a very large wager on one researcher's view. It's worth holding it as exactly that, a view and a wager, rather than a settled conclusion.

Does a language model have a world model? Two studies that point in different directions (2022 to 2024)

The sharpest version of the debate is about language models. They are trained only to predict the next word. Do they build an internal picture of the world as a side effect, or are they matching surface patterns?

In October 2022, Kenneth Li and colleagues from Harvard and other universities trained a small GPT-style model on nothing but lists of moves from the board game Othello. The model never saw a board and was never told the rules. It learned to suggest legal moves. The researchers then looked inside it and found an internal representation of the board: which squares held which colour. More convincingly, when they edited that internal board, the model's move suggestions changed to match the edited position. The paper, often called "Othello-GPT", was presented at the ICLR conference in 2023 and became the main evidence that next-word prediction can produce a world model.

In 2024, a team led by Keyon Vafa at Harvard, with colleagues at MIT and Cornell, pushed back with a careful test. They trained models on taxi rides in New York City. The models gave turn-by-turn directions with nearly perfect accuracy. But when the researchers reconstructed the map implied inside the model, it contained streets that don't exist and impossible flyovers. And when they closed just 1 percent of the streets, as a detour would, accuracy fell from nearly 100 percent to 67 percent. The work was presented at the NeurIPS conference in 2024.

Put together, the honest reading is: language models can build internal models of the things they're trained on, sometimes surprisingly good ones, and those models can be patchy and incoherent in ways that stay hidden until conditions change. Good performance on familiar cases doesn't prove understanding.

Try it yourself

You can't build a world model on your laptop this afternoon, but you can get a feel for what one is, and for where AI's grasp of the physical world is strong and weak. Allow about 30 minutes.

  1. Warm up your own world model. Before you touch a computer, write down what you think happens in each of these: a full glass of water is pushed off a table; a ball is rolled towards a rug on a wooden floor; a stack of five books has the bottom one pulled out quickly; an ice cube is left on a sunny windowsill for an hour. Notice how quickly and confidently you answered. That speed is your world model at work.
  2. Run a "predict then check" test with a chatbot. Open any chatbot and paste this prompt:
I'm going to describe a physical situation. Tell me what happens next,
step by step, and say how confident you are. Then tell me one thing
that could make your prediction wrong.

Situation: A full glass of water sits on a table. I push it slowly
towards the edge until about a third of its base is hanging over
the edge.
  1. Compare with what you know. Does its answer match yours? (A glass with a third of its base over the edge stays put, because its centre of gravity, the middle of the base, is still over the table. Did the chatbot say it falls? Then ask about three quarters over the edge, where it should topple.) Try the other situations from step 1.
  2. Make it harder with a change of conditions. Language models are usually good at common situations, because those are described often in text. The New York taxi study suggests the weak point is changed conditions. Try this:
Same glass, same table, but now the table is on a boat that tilts
10 degrees towards the edge the glass is on, then tilts back.
What happens? Walk through it moment by moment.

Then try a situation that is unusual but checkable at home, such as stacking coins so the top one overhangs the bottom one, and test it yourself. Keep a note of where the chatbot's predictions match reality and where they don't.

  1. Ask it to plan by imagining. Planning with a world model means imagining outcomes and choosing between them. Try:
I need to carry a full cup of coffee, a laptop and a plate of
biscuits up a flight of stairs in one trip. List three ways to do it.
For each, imagine what could go wrong, step by step, and then pick
the safest.

This is the planning loop from this article, done in words. Notice whether the imagined failures are ones you'd also foresee.

  1. Watch an agent drive inside its dream. Open the free interactive version of the 2018 paper at worldmodels.github.io. It runs in your browser with no account. Scroll to the car racing and Doom sections and watch the demos, including the one showing what the agent "sees" inside its own world model. Notice how blurry the dream is: the model only keeps what matters for the task.
  2. Optional: a generated world. Google's Project Genie lets you walk around worlds generated from a text prompt, but at the time of writing it requires a paid Google AI Ultra subscription and is limited to certain countries. Check Google's current terms if you want to try it. If you do, try to break it: walk away from something and come back, and see whether it's still there.

Common mistakes and misunderstandings

Thinking a realistic video means the AI understands physics. A clip can look right while getting physics wrong. When OpenAI unveiled Sora in February 2024 it published a report titled "Video generation models as world simulators", and in the same report listed that the model did not accurately handle basic interactions such as glass shattering. Looking plausible and predicting correctly are different tests.

Assuming "world model" means a model of the whole world. Most world models cover one narrow environment: a game, a kitchen counter, a stretch of road. MuZero was deliberately built to model only what matters for its decisions. A system that plans brilliantly in Minecraft has no idea what happens when you push a real glass.

Treating demos as deployments. Many headline results come with carefully chosen examples, short time limits, and a "research preview" label. Genie 2's worlds stayed consistent for up to a minute; Genie 3's for a few minutes. That's remarkable progress and also a long way from a robot working an eight-hour shift. Read the limitations section, which the better announcements include.

Forgetting that learners exploit flaws in their model. An agent trained inside an imperfect world model will find and use the imperfections, as Ha and Schmidhuber's Doom agent did. Anything trained in simulation needs testing in reality, which is why self-driving companies still drive real miles.

Treating one side of the language-model debate as settled. You'll hear confident claims that language models "have no understanding of the world" and equally confident claims that they "have built a model of reality". The evidence supports neither extreme. Othello-GPT showed a real internal model; the taxi study showed a brittle one. Attribute strong claims to the people making them, and ask what evidence they're relying on.

Mixing up world models and physics engines. A video game's physics engine is hand-written and exact within its rules. A world model is learned and approximate. That makes world models more flexible, because they can pick up behaviour nobody programmed, and less reliable, because they can hallucinate physically impossible things with total confidence.

Assuming more video is all it takes. Watching shows what tends to happen, not what happens because of your actions. V-JEPA 2 needed robot data on top of a million hours of video to plan actions. Learning from interaction remains slow and expensive, and it's one of the main bottlenecks for robotics.

Believing a prediction about the future of AI because the person making it is famous. World models attract bold forecasts about robots, driverless cars, and human-level AI. Some come from brilliant researchers; some come from people raising money; often they're the same people. Treat forecasts as forecasts.

Related elements