ZeroBot is a framework that lets a robot learn to manipulate a novel object into a goal pose with zero human demonstrations, zero policy pre-training, and zero known object models. It uses a single image of the object to generate a complete 3D mesh via an image-to-3D generative model, builds a simulation from that mesh, and trains a policy using massively parallel . A novel action space leverages the generated geometry to sample contact states and uses the learned value function to select a high-value contact point, accelerating training.
In real-world tests on grasping, pushing, articulated object interaction, and multi-stage tasks, ZeroBot achieved an 87% success rate with an average training time of just 119 seconds. This is interesting because it dramatically reduces the human effort and time required to adapt a robot to a new object and task, enabling rapid, autonomous learning in settings where demonstrations or pre-trained models are unavailable.