The paper investigates whether performance gains from world model post-training (where agents learn to predict next observations) actually come from accurate world prediction or from other training effects. In controlled experiments on ALFWorld and ScienceWorld, the authors compared ground-truth next-observation targets with mismatched targets (same distribution but wrong transitions) and with random coin-flip rewards. They found that mismatched targets reduced prediction accuracy by 15.3-61.6% relative to ground-truth, yet preserved substantial task gains over a no-training baseline. Even training with independent random rewards expanded task coverage (pass@64). These findings generalize to VisualWebArena, where random-reward training raised pass@64 by 14.3% relative to baseline without needing observation-matching rewards or an external multimodal teacher. This is interesting because it suggests that the benefits of world model post-training may stem from training dynamics like action diversification rather than learning accurate environment dynamics, which simplifies the requirements for agent training pipelines.
Accurate World Predictions Are Not Required for Agent Training Gains
Controlled experiments found that mismatched observations and random rewards could improve agents despite providing no correct information about environment transitions.
Research Lab
Xinyu Che · Hang Yan · Yanchen Liu · Haochen Liu · Ruifeng Li · Anran Shi · +2 more
Xi’an Jiaotong University · University of Southern California · University of the Chinese Academy of Sciences · East China Normal University
Research Digest··2 min read
In one line
World model post-training improves agent performance through training effects other than accurate world prediction, as mismatched targets retain task gains despite lower accuracy.
Before you read this
Three ideas this paper builds onworld model for agentsA predictive environment model that agents use to simulate outcomes, enabling planning, reasoning, and long-horizon task execution.115 papers →reinforcement learningA machine learning method where an agent learns to make sequences of decisions by receiving rewards or penalties for its actions.296 papers →group relative policy optimizationA reinforcement learning method that optimizes policy by computing advantages based on group-level reward statistics.188 papers →
Read it as
Che et al. audited whether world-model post-training improves agents because they learn to predict what happens after an action, or because training changes their behavior in other ways.
The authors ran controlled reinforcement-learning experiments on ALFWorld and ScienceWorld.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.