RAPID automatically generates, verifies, and refines robot programs from a single visual human demonstration. It infers a testable task specification, action primitives, and an interactive simulation environment from the demonstration, then uses an agentic loop to iteratively improve the program. The system represents programs in an object-centric relational form, focusing on the strategy's structure rather than specific motions, enabling generalization across variations in object pose, shape, material, and environment. Evaluated on eight contact-rich nonprehensile manipulation tasks and LIBERO-Pro benchmarks, RAPID showed strong performance in both simulation and real-world tests with a Franka arm.
One demonstration can bootstrap reusable robot manipulation programs
RAPID infers task goals, action primitives and a verification environment from a visual demonstration, then iteratively refines executable robot code.
Big Tech
Yuyao Liu · Jiayuan Mao · David Hsu · Leslie Pack Kaelbling · Tomás Lozano-Pérez
Massachusetts Institute of Technology · National University of Singapore · University of Pennsylvania · NVIDIA
Research Digest··2 min read
In one line
RAPID generates reusable robot programs from a single human demonstration using an iterative agentic coding loop.
Before you read this
Two ideas this paper builds onRead it as
Liu et al. present RAPID, a system that converts a single visual human demonstration into a robot manipulation program.
RAPID reconstructs an interactive simulation environment from a demonstration and infers three components needed for agentic code refinement: a testable task specification, robot action primitives and a program that composes those primitives.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.