Many robot control systems that combine camera images and text instructions first shrink the image into a compact summary using a vision encoder. This can lose fine details like object edges and depth that are crucial for precise movement.
Researchers designed NEO-VLA, which processes raw pixels directly with the text in a single network. The part that decides robot actions can read visual information from multiple stages of this network.
NEO-VLA performed better than traditional systems on standard benchmark tasks. The key visual information for actions was found in early network stages, before being fully abstracted. This shows that keeping early visual details accessible leads to more robust robot control.