Robot arms need to learn how to pick up and move objects, but collecting enough training data by having robots practice each task is slow and expensive. Humans do these tasks all day, and their hand movements can be recorded by a camera worn on the head. This kind of video shows close-up hand-object interactions and how tasks unfold over time.
This paper organizes methods that turn egocentric human video into useful training for robot control. It describes four approaches: converting human movements into robot-format demonstrations, extracting task labels from the video, using the video to pre-train visual systems, and learning hidden models of actions and object changes from the video alone.
The main finding is that the value of human video depends not on its quantity but on how much task-relevant interaction information it preserves and how that information is grounded in the robot's actual body and sensors. This helps researchers choose which approach to use based on what robot data they already have.