Hallucination in World Models Is Predictable and Preventable
Modern generative world models can generate realistic, action‑controllable futures but frequently produce hallucinated rollouts that deviate from true dynamics. These hallucinations remain visually fluent, which can mislead downstream planning and policy learning. Researchers trained a 350 million‑parameter model on 210 tasks and showed that hallucination is both predictable and largely preventable.
- ▪Generative world models render strikingly realistic futures yet often hallucinate, staying visually fluent while drifting from ground‑truth dynamics.
- ▪Hallucinated rollouts can cause incorrect decisions when used for planning or policy learning.
- ▪A 350 million‑parameter generative world model was trained on a dataset covering 210 tasks to investigate hallucination behavior.
- ▪The study found that hallucination can be predicted and largely mitigated by addressing underlying issues.
Opening excerpt (first ~120 words) tap to expand
Hallucination in world models Modern generative world models render strikingly realistic, action-controllable futures. But the rollouts they produce frequently hallucinate: they stay visually fluent and superficially plausible while drifting away from the ground-truth dynamics. When used downstream for planning or policy learning, model hallucination leads to incorrect decisions. In this work, we train a 350M-parameter generative world model on a large dataset spanning 210 tasks and show that, even at this scale, hallucination is both predictable (we can predict when it will happen) and preventable (the underlying issue is, to a great extent, fixable).
Excerpt limited to ~120 words for fair-use compliance. The full article is at Hallucination in World Models.
Discussion
0 commentsMore from Hallucination in World Models
