A robot hand turning an object in its fingers needs to know the object's orientation. Cameras often see mostly the hand, hiding the object. Common systems estimate the object's position and orientation, but this is hard and needs object-specific setup.
They trained one controller mapping camera images and joint-angle feedback directly to finger motions, with goals given as target images rather than pose coordinates. Training copied a simulated expert with extra state information and added signals to reconstruct hidden variables.
In simulation it beat pose-based methods and controllers built on image models pretrained on unrelated data. One controller ran on real hardware across several objects, succeeding 80% versus 27% for the best pose baseline, showing pose estimation can be skipped.