that generate code from images often rely on their own built-in ability to see and interpret images, which limits performance when that ability is weak or absent. This approach works poorly for agents that cannot process images at all, and even vision-capable agents are capped by their own perception.
The authors propose a new approach called Vision-as-a-Tool. Instead of making the agent do all the visual work itself, the agent can call separate, general-purpose services to perform specific visual tasks, such as transcribing the layout of an image, finding a particular element, or comparing two images. These services are provided by a framework called VHarness, which offers five basic visual actions.
In tests across three different tasks, turning images into web code, vector graphics, and 3D models, agents using this tool-based approach produced results with better visual accuracy and geometric correctness. The key finding is that separating visual perception from code generation into reusable tools is both flexible and effective, enabling text-only agents to match the performance of native vision agents.
