Humanoid robots lack training data that pairs camera views and commands with movements. Collecting this by having people remotely control robots is too slow.
The researchers built HALO, which creates training episodes entirely in computer simulation. It generates many virtual homes and uses AI to produce motions for tasks like sitting and walking through doors. On the real robot, a planner breaks down language commands, a generator makes motion plans, and a controller executes them while staying balanced.
When tested in six real homes it had never seen, the robot succeeded 91% of the time. This shows simulation can replace human data collection, enabling robots to work in new environments without prior scanning.