Adapting large vision-language-action (VLA) models to specific robot tasks is challenging. Supervised fine-tuning on static demonstrations suffers from covariate shift and treats all data equally. HIL-UMI addresses these issues with a robot-free human-in-the-loop approach using handheld Universal Manipulation Interface (UMI) grippers.
During demonstrations, the current policy is queried on the same observation stream without executing its actions. An Energy Score compares human and policy trajectories to detect out-of-distribution states, triggering targeted data collection. A separate advantage estimation loop identifies essential segments for policy improvement, and the policy is retrained with advantage-conditioned behavioral cloning on a mix of original and new data.
On four real-world manipulation tasks (including long-horizon and precise tasks), HIL-UMI consistently outperformed supervised fine-tuning. On the Clean Up Table task, it achieved better performance than HG-DAgger with 5.63x faster per-frame collection time. This decoupling from physical robots makes interactive post-training more scalable across operators and locations.