(RLVR) improves vision-language benchmark scores even when training without images, recovering much of the real-image gain. Prolonged training with images can erode grounding while benchmark gains persist, revealing that images in prompts are not necessarily part of the learning signal. The authors propose a design rule called visual resolvability: require visual evidence for correct answers while keeping the task learnable. They test this on counterfactual coordinate scenes where the question is fixed and the target must be found in the image. Using standard GRPO, a 7B model raises discovery accuracy from 0.425 to 0.875 on held-out scenes denser than any training. Controls show that images are essential: test with gray canvases drops accuracy to zero, and training on gray canvases yields none of the gain. The skill transfers to other grounding tasks, and adding an answer-containing caption cuts the gain by nearly two thirds. The lesson is that what reward requires determines what RL actually learns.
Changing reward requirements teaches vision-language models to actually use images
Experiments with counterfactual scenes show that reinforcement learning improves visual grounding only when correct answers require extracting evidence from the image.
Research Lab
Haocun Ye · Xinlong Jiang · Qile Chen · Bingyu Wang · Teng Zhang · Shubai Chen · +3 more
University of the Chinese Academy of Sciences · Institute of Computing Technology, Chinese Academy of Sciences
Research Digest··3 min read
In one line
Multimodal reinforcement learning with verifiable rewards fails to learn visual skills unless tasks require visual evidence.
Before you read this
Three ideas this paper builds onreinforcement learning with verifiable rewardsReinforcement learning that uses rewards from objectively checkable signals like correct answers or verification feedback.167 papers →visual groundingAssociating visual information in images with corresponding language descriptions or abstract concepts.6 papers →evidence blindnessHaving access to relevant evidence does not prevent an LLM agent from making incorrect or unsafe decisions.36 papers →
Read it as
Ye and colleagues test whether reinforcement learning with verifiable rewards improves visual perception or merely teaches models to exploit nonvisual shortcuts.
The authors first examined image-based reinforcement learning on Geometry3K, comparing normal training against runs where every training image was replaced by a gray canvas.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.