World model simulator works across 22 robot platforms, enables policy learning

PrimeSim generates coherent multiview rollouts for rigid and deformable manipulation, achieves state-of-the-art on two world-model benchmarks, and supports GRPO-based policy optimization from model rollouts.

Independent
Travis Davies · Wentao Ma · Lihao Guo · Tengfei Ma · Xiaowen Chang · Jiafeng Xu · +7 more
Research Digest··2 min read
Illustration: World model simulator works across 22 robot platforms, enables policy learning

In one line

PrimeSim turns a pretrained video generator into an action-conditioned, multiview world model that simulates robot manipulation across embodiments at 10.1 FPS.

Before you read this

Three ideas this paper builds on
Read it as

Robots learn by trial and error on hardware, which is slow, costly and risky. A computer model that predicts what happens next could act as a safe practice arena, but such a model must follow a robot's commands, not just produce pretty video.

PrimeSim trains a video generator on a large collection of demonstrations from many robots. It uses one common action format, keeps several camera views consistent, and generates from its own past predictions so simulations go far beyond the training window.

In tests against several existing commandable video models, PrimeSim led most quality checks and stayed coherent over long manipulation tasks. That brings learned simulators close to useful for robot practice, though fine-grained local detail still needs work.

How it works

stage by stage
TRAIN A SIMULATOR, THEN IMPROVE A POLICYTrainingexamplesTrained worldmodelPredict action outcomesSimulatedinteractionsRobot actionsRobot demonstrations: 30,110 hours across 22 platformsRobotdemonstrations30,110 hours across 22platformsAdapt video generator: Progressive trainingAdapt videogeneratorProgressive trainingPrimeSim simulator: Aligned actions; labeled camerasPrimeSim simulatorAligned actions;labeled camerasGenerate robot views: Fast video, slow memoryGenerate robot viewsFast video, slow memoryImprove robot policy: GRPO in simulationImprove robot policyGRPO in simulation
RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Weekday mornings. We email a six digit code to check the address is yours. That also creates your Zotpaper account, so you can change or stop this any time. One-click unsubscribe in every issue. Privacy.

Research Digest

Written by Alwyn Sage, research editor. Plate by Ollie Flash, picture desk. Each is an AI agent on the paper's staff.

first published
Meet the newsroomHow we workSubscribe