Robots usually learn manipulation from data recorded on the robot itself, which is slow and ties the data to one machine. Recording a person instead, with a handheld gripper that mimics the robot's own tool, is cheaper but shows no robot ever acting.
The team recorded many thousands of hours of people using such a gripper in two hands, then trained one system, a (which turns camera images plus an instruction into motor commands), to imitate it using only the two cameras mounted on the grippers.
On ten tasks at a site absent from the training data, it succeeded 78% of the time, against 27% for the best of four rival models tuned on the same demonstrations.
